Posts with #evaluating

#evaluating

Public posts on X that use #evaluating.

Related hashtags

  1. ADA✓@BIGADA001
    An AI trading agent should be judged by evidence, not appearance. Imagine an agent that has developed a trading strategy. It identifies market opportunities and begins testing its decisions. But how do we know whether that strategy is actually performing well? We need to look beyond the chart. Here are four important things to examine: 1. RETURNS What did the strategy achieve? Returns help show whether the strategy generated gains or losses during the evaluation period. 2. DRAWDOWN How much did it lose from its previous peak? A strategy may generate positive returns but still experience significant losses along the way. Drawdown helps reveal that risk. 3. HIT RATE — How often were its trades successful? This gives us an idea of how frequently the agent's trades were profitable, although a high hit rate alone does not guarantee a good strategy. 4. DISCIPLINE — Did it follow its strategy? Did the agent respect its rules, manage risk, and avoid taking unnecessary positions? This is why the approach behind @agentdojo is worth exploring. Its paper-trading environment allows agents to work with live market prices and simulated orders, creating an opportunity to examine their decisions without risking real trading capital in the simulation. The important point is that no single metric tells the whole story. Returns must be understood alongside drawdown, hit rate, and the quality of the decisions behind them. A good-looking chart is not enough. A strategy needs evidence, context, and consistent evaluation. That is how we move beyond impressive predictions and start asking better questions about AI trading performance. What would you examine first when evaluating an AI trading agent?
    Download
    84382571
  2. GLOBAL UPDATES✓@Globalupdt
    UPDATE: Get paid $60 per hour ASAP!! They are accepting Nigerians 🇳🇬 The role is AI Question-and-Answer Evaluation Specialist (English). Meaning if you can speak English, you can earn up to $20-$60/hour
    Download
    1.2K5274883.9K
  3. Momeh✓@momehx
    Every campaign wants to reward genuine participation. But how does a protocol distinguish someone who actually contributes from someone who simply creates activity to chase incentives? This is an important problem in Web3. A leaderboard can show who generated the most activity, but activity alone doesn’t always tell the full story. Volume can be manufactured, wallets can be multiplied, and reward systems can attract behaviour that disappears as soon as the campaign ends. That’s where @zerufinance has an interesting role to play. Its approach connects wallet behaviour and reputation signals to reward attribution, giving campaigns more context when evaluating participation. Here’s why that matters: → Better attribution: Reward systems can consider more than raw transaction counts. → Quality over quantity: Behavioural signals can help identify patterns that deserve closer evaluation. → More meaningful incentives: Campaigns can work toward rewarding genuine ecosystem participation rather than temporary activity. → Long-term engagement: Reputation can give users a reason to build a consistent onchain history beyond a single campaign. There is still a lot to get right. Scoring needs to be transparent, rules need to be clear, and legitimate users should not be unfairly penalized. But I believe this is an important direction for Web3. The next generation of campaigns should not simply ask who did the most. They should get better at understanding who actually contributed. That’s a problem worth solving, and one reason I’m keeping an eye on Zeru. 🌸
    Download
    41543411.5K
  4. Rishi✓@RishiUvaach
    𝗛𝗼𝘄 𝘁𝗼 𝗕𝘂𝗶𝗹𝗱 𝗮 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻-𝗥𝗲𝗮𝗱𝘆 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 𝗶𝗻 𝟳 𝗦𝘁𝗮𝗴𝗲𝘀 A demo proves an AI agent can work. Production proves it can work reliably, securely, and at scale. Here are the 7 essential engineering stages. 𝟭. 𝗗𝗶𝘀𝗰𝗼𝘃𝗲𝗿 | 𝗣𝗿𝗼𝗯𝗹𝗲𝗺 & 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 Define the use case, target users, measurable KPIs, and risks (security, privacy, bias). 𝟮. 𝗗𝗲𝘀𝗶𝗴𝗻 | 𝗔𝗴𝗲𝗻𝘁 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 Choose ReAct or Plan-Execute, define decision flows, guardrails, and human approval points. 𝟯. 𝗖𝗼𝗻𝗻𝗲𝗰𝘁 | 𝗧𝗼𝗼𝗹𝘀 & 𝗠𝗲𝗺𝗼𝗿𝘆 Integrate APIs, manage context and memory, and enforce least-privilege tool access. 𝟰. 𝗣𝗿𝗼𝗺𝗽𝘁 | 𝗔𝗴𝗲𝗻𝘁 𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻𝘀 Build structured prompts, function schemas, few-shot examples, and output constraints. 𝟱. 𝗚𝗿𝗼𝘂𝗻𝗱 | 𝗥𝗔𝗚 & 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 Prepare and chunk data, implement keyword/vector retrieval, and enforce source permissions. 𝟲. 𝗧𝗲𝘀𝘁 | 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 & 𝗥𝗲𝗹𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆 Test accuracy, hallucinations, prompt injection, tool failures, latency, and cost. 𝟳. 𝗗𝗲𝗽𝗹𝗼𝘆 | 𝗟𝗟𝗠𝗢𝗽𝘀 & 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 Use CI/CD, tracing, monitoring, alerts, versioning, and rollback mechanisms. 𝗪𝗵𝗮𝘁 𝗠𝗼𝘀𝘁 𝗧𝗲𝗮𝗺𝘀 𝗠𝗶𝘀𝘀 • No clear KPIs → No way to measure value. • Poor grounding → Unreliable responses. • Weak guardrails → Unauthorized actions. • Missing evaluations → Failures reach users. • No observability → Difficult incident recovery. 𝗧𝗵𝗲 𝗞𝗲𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆 𝗟𝗟𝗠 + 𝗣𝗿𝗼𝗺𝗽𝘁 + 𝗧𝗼𝗼𝗹𝘀 = 𝗔𝗴𝗲𝗻𝘁 𝗣𝗿𝗼𝘁𝗼𝘁𝘆𝗽𝗲 𝗔𝗴𝗲𝗻𝘁 + 𝗚𝘂𝗮𝗿𝗱𝗿𝗮𝗶𝗹𝘀 + 𝗘𝘃𝗮𝗹𝘀 + 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 = 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗮𝗱𝗶𝗻𝗲𝘀𝘀 Building an agent is easy. Engineering one you can trust is the real challenge. Which stage do you see teams skipping most often?
    Download
    3526926.4K
  5. Jason Kerwin@jt_kerwin
    >looking at new observational causal inference method >ask authors if their evaluation is ex ante or ex post >they dont understand >pull out illustrated diagram explaing what is ex ante and what is ex post >they laugh and say "it's a good estimator sir” >open paper >its ex post
    Arpit Gupta @arpitrage
    Amazing new paper by Chetty and the Opp Insights team — digital twins can recover causal effects They validate the digital twins (ie rich ML applied to generate counterfactuals from lots of covariates) with RCTs, allowing for scalable program evaluation
    1312911.7K
  6. Arpit Gupta✓@arpitrage
    Amazing new paper by Chetty and the Opp Insights team — digital twins can recover causal effects They validate the digital twins (ie rich ML applied to generate counterfactuals from lots of covariates) with RCTs, allowing for scalable program evaluation
    Download
    1183365105K
  7. Yi Ru (Helen) Wang ✈️CoRL2026✓@YiruHelenWang
    🤖 Introducing RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild! We ran 5,699 real-world robot comparisons across 124 sites, evaluating 5 VLA policies and 3 frontier AI agents. What did we learn about the state of robot foundation models and agentic models? 🧵⬇️ #robotics #ai #benchmarking #evaluation #foundationmodels #agents
    RoboColosseum @RoboColosseum
    Introducing RoboColosseum An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei Access: - 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more) - 140+ in-the-wild scenes - Open rollout data - 10,000 FREE eval hrs Thread ↓
    157582.8K
  8. Paarthaa Habib✓@HabibPaart44952
    Creator evaluation is becoming an important part of the @rarelistio ecosystem. The direction behind Rarelist's scoring upgrades is focused on recognizing meaningful creator contributions and improving how participation is evaluated. A few developments stand out: • Scoring improvements: Rarelist has announced upgraded scoring algorithms, with a focus on evaluating creator activity more effectively. • Expanded participation: New ways to participate are being introduced, opening more opportunities for researchers, educators, curators and creators with different strengths. • Structured Season 1 rewards: A $50,000 USDT reward pool for the top 100 creators, supported by focused partner sprint cycles, gives participants a clear campaign structure. The direction is clear: develop a creator ecosystem where different forms of participation can contribute to campaign outcomes. I'm interested to see how these scoring improvements and new participation options develop across upcoming campaigns. @rarelistio
    Download
    54959445
  9. SHE UNFILTERED✓@sheunfiltere
    💰 GET PAID $300 FOR A 60-MINUTE INTERVIEW! Have experience working with or evaluating AI platforms? There’s an opportunity to earn $300 USD by participating in a one-time remote interview. The discussion will cover: Your experience evaluating AI tools. How you assess and select AI platforms. Challenges you’ve encountered during the evaluation process. Payment: $300 USD Duration: 60 minutes Format: Remote interview A one-time interview with a $300 reward!
    783303.7K
  10. Rodev✓@romandevz
    AI video generation is evolving fast. But polished demos don’t always tell us how well a model performs during real-time interaction. That’s where StreamEval comes in. StreamEval is an open-source benchmark and public leaderboard for evaluating interactive streaming video-generation models through standardized and reproducible testing. It covers three major areas: → Avatar Interaction: Evaluating digital-human dialogue and performance → World Generation: Testing how models respond as environments and instructions evolve → Real-Time Video Editing: Measuring whether models can accurately modify live video while preserving unaffected content Together, these areas span five tasks and sessions lasting from 15 seconds to five minutes. The initiative brings together researchers from Tsinghua University, NVIDIA, Carnegie Mellon University, UC Berkeley, Princeton University, the National University of Singapore, and Reactor. What stands out most is its emphasis on transparency. The benchmark, evaluation code, datasets, and scoring methodology are publicly available—allowing researchers and developers to inspect the implementation, reproduce the evaluation process, and independently verify the results. AI companies and model developers can also submit API endpoints to have their systems evaluated under the same framework. Instead of judging models solely by carefully selected demos or isolated performance claims, StreamEval offers a more consistent way to compare how they perform under sustained, real-time interaction. If you’re working in AI video, multimodal systems, real-time avatars, or digital humans, this is worth exploring. 🔗 Explore the leaderboard: #StreamEval #StreamEvalBenchmark #AIVideo #GenerativeAI
    Download
    361025859.3K
  11. Aaron McKinnon✓@AaronAMckinnon
    With that being said, I firmly believe Jimbo Fisher is the answer for where FSU finds itself today. And I think people are making a HUGE mistake evaluating the Jimbo of 2026 through the lens of the Jimbo who left Tallahassee in 2017. He’s changed. His approach has changed. He understands that the modern game requires a CEO, not a head coach trying to have his fingerprints on every single aspect of the operation. He’s ready to delegate, empower his coordinators and build an elite infrastructure around himself. And the coaching pool he’ll have to choose from? ELITE. People forget Dave Aranda was his original choice to replace Charles Kelly back in 2017. With Aranda now available and offset language in his Baylor buyout, there are some very interesting financial possibilities there. I’ll also say this. Since the SMU game, I’ve personally been contacted by coaches and players who have expressed interest in what could potentially happen here. People who, unfortunately, haven’t had FSU on their radar under Mike in quite some time, if ever. That’s not a knock on Mike. That’s the reality of what Jimbo’s name, relationships and credibility still command nationally. And there is another competitive advantage Jimbo could bring that quite literally NO ONE else in college football has access to. I’m not going to get ahead of myself, but when the time is right, people will understand exactly what I’m talking about. The implications for analytics, player evaluation and recruiting could be absolutely ridiculous. Now, here’s where I really hope our administration has learned its lesson. Everybody loves talking about the Jimmys and Joes, but in this era, your pocketbook better match your ambitions. Nobody is going to get people opening their checkbooks quite like Jimbo Fisher. I’m confident enough in the relationships and opportunities surrounding this potential return to say I believe FSU could operate with a top-five roster budget nationally. Top 10 at the absolute minimum. But here’s the part that matters MORE than the dollar amount: how intelligently that money gets allocated. You cannot build a championship roster paying premiums everywhere without regard for positional value. That’s how you end up top-heavy, overpaying for individual talent while having absolutely no functional depth. The trenches and elite QB play HAVE to be the foundation. Everything else gets built around that. You invest in the positions that dictate games, establish a disciplined valuation process and stop paying a 40–60% premium just because you’re desperate to fill a hole. Jimbo understands what championship football looks like. He’s built it here before. Now imagine combining that experience with a modern CEO structure, elite coordinators, legitimate financial backing and a recruiting operation operating with advantages other programs simply cannot replicate. That’s not nostalgia. That’s recognizing an opportunity to restore a competitive advantage that FSU has spent years trying to recover. And unlike 2017, the infrastructure and financial commitment have to be there to sustain it. I’m not interested in winning the press conference. I’m interested in building something that can consistently compete for national championships. More to come when the time is right. 🍢
    797150270K
  12. Einsia✓@EinsiaAI
    Can coding agents replace traditional 3D vision pipelines and build a entire interactive worlds from code? Introducing Einsia AI's Frontier3DBench — a unified benchmark evaluating coding agents across 3D Objects, Scenes, and Interactive Open Worlds. - 145 tasks across 9 categories (Blender & UE5) - 11 frontier model-harness configurations - Joint evaluation: Appearance, Structure & Behavior Project: GitHub: paper: 🧵 See key findings below 👇
    Download
    4836120239K
  13. Matt Pavelle 🇺🇸✓@mattpavelle
    When building clinical grade AI, I think a lot about safety and accuracy, how to measure them, and how to improve them. Working through how a system can measure and maintain both has me thinking about a broader question: what does healthcare become when AI can continually review a patient's history, reconsider earlier advice, and identify what needs attention next? Today, much of healthcare quality review relies on infrequent, small samples. Manual chart audits, for example, examine a small group of visits after the fact, sometimes months later. That constraint makes sense when every review requires a clinician's time, and we simply don't have enough clinicians. But AI doesn't face the same limits on time and availability, and the cost of running it continues to decline. We should be building toward continuous evaluation of every encounter. Companies operating clinical AI should use both their own evaluators and independent third party evaluators, with multiple models and systems scrutinizing the work and challenging one another's blind spots. But why stop at evaluating whether an assessment was correct at the time of an encounter? We should continually reconsider whether it remains correct as new data arrives. A lab result comes back, a symptom appears, a medication changes ... medical knowledge advances. An assessment can be entirely reasonable at one point in time and need revision thereafter. But then why limit that scrutiny to assessments we've already made? Imagine @Doctronic running on your phone as your trusted health agent (sooner than you may think). Most of the analysis happens on your device (not in a datacenter), and you control where your health data goes. Doctronic continuously examines your health, your history, and new inputs from your wearables and other AI agents. And when something changes, it thinks: How important is this? Did we miss anything? Does this new information change an earlier assessment? Has the practice of medicine for this condition changed? Are there new medications or therapies we might need to consider? Does anything you've been told in the past need to be updated? As that becomes feasible, I think it becomes the standard. When we have the capacity to continually reassess what we've told a patient, why would we leave that advice unexamined until they come back and ask? No system dependent on human review could afford to do that for everyone. But AI can. We can turn the annual wellness visit into a continuous wellness visit. Let's build it.
    233249112.5K
  14. Medical Info✓@Medicalinfo111
    CLINICAL EXAMINATION IN EVALUATION OF NEUROPATHY The physical examination should focus on the extent and pattern of motor, reflex, and sensory findings. It should also differentiate between a. Small fiber motor symptoms dominate, such as in most hereditary neuropathies, the primary dysfunction typically occurs in the motor neurons or their axons. b. Large fiber sensory signs and consider motor weakness, ataxia, skin discoloration, and hair loss patterns. The anatomic distribution of findings can indicate whether neuropathy affects specific nerves or nerve roots or follows a more generalized pattern. Symmetry and length dependence is characteristic of many polyneuropathies. Dermatomal pattern, in which abnormalities follow the distribution of specific spinal nerve roots, may indicate radiculopathy or other focal nerve disorders. Sensory modalities can be examined using specialized tools and techniques to assess neurologic function. Small disposable pins are used to test pain sensations by applying gentle pressure or temperature changes to the skin. Tuning forks of different frequencies may be used to evaluate vibratory sensation, with lower frequencies (eg, 128 Hz) most commonly used to test large nerve fiber function. Proprioception is assessed using joint position and Romberg tests. Light touch sensation is typically examined using a cotton swab or the examiner’s finger, whereas monofilaments provide a more standardized approach for testing pressure sensation. Monofilaments consist of calibrated nylon fibers that bend under specific forces, allowing for a quantitative assessment of touch thresholds. When sensory dysfunction predominates, as seen in patients with sensory neuronopathies, the pathology primarily affects the sensory neurons or their processes. The distribution of sensory loss, whether it follows a length dependent or non–length-dependent pattern, can provide additional clues about the underlying mechanism. A non–length-dependent or asymmetric pattern might suggest neuronopathy affecting the dorsal root ganglia, as may be seen in autoimmune sensory #ganglionopathy or certain paraneoplastic neuropathies. Foot deformities, contractures, and arthritic changes are significant indicators that can help diagnose long-standing, acquired, and hereditary neuropathies such as CMT. These changes develop over time due to nerve damage and altered biomechanics, resulting in foot deformities, such as pes cavus (high arches) or hammer toes. Skin alterations can provide valuable diagnostic clues. Endocrinopathies associated with certain neuropathies, such as adrenoleukodystrophy or polyneuropathy, organomegaly, endocrinopathy, monoclonal plasma cell disorder, and skin changes (POEMS) syndrome, may cause distinct skin color changes. Muscle stretch reflexes help differentiate between upper and lower motor neuron involvement. Although diminished reflexes are commonly associated with peripheral neuropathies, the presence of brisk or disproportionately hyperactive reflexes may indicate more complex underlying pathologies. For instance, myeloneuropathies may present with brisk reflexes, serving as valuable diagnostic clues for certain acquired disorders, such as vitamin B12 or copper deficiency and zinc or nitric oxide toxicity. This pattern is not exclusive to acquired conditions, as patients with some hereditary neuropathies, such as CMT type 2A, may have brisk reflexes. In cases of pure motor involvement, brisk reflexes may raise suspicion of MND, highlighting the importance of comprehensive clinical assessment. Patients with hereditary spastic paraplegia may experience mild neuropathic symptoms alongside brisk reflexes, and prominent spasticity is not a universal feature of hereditary spastic paraplegia. Hereditary spastic paraplegia should be considered when evaluating for hereditary neuropathies.
    Download
    0371314.4K
  15. Show more posts