Reppo✓
@reppo
Decentralized RL + Evaluation Infrastructure, powered by Prediction Markets
325 Following26.7K Followers6K Posts
- ReppoRT @rgvrmdya: People often ask me that what's the grandiose vision of base:0xff8104251e7761163fac3211ef5583fb3f8583d6 If Bitcoin is digital gold, ETH is the world computer, Bittensor is Bitcoin for AI, What is Reppo? Is it just an API that uses a decentralized network to provide evaluations on AI outputs and trajectories? No. If everything goes according to the plan base:0xff8104251e7761163fac3211ef5583fb3f8583d6 has the potential to become the governance layer for AI. It's only now that people are starting to ask - who will evaluate the AI labs? Will they self-regulate? Will governments regulate labs or will labs regulate governments? If you joined crypto to fight centralization and power concentration, these questions should matter to you. Building a decentralized network to judge AI agents + robots verifiably, on demand, is just step 1. We have a long road ahead and I continue to believe it's worth trying.American Optimist @AmOptimistShowNEW🚨 Inside the Company That Evaluates AI Labs; Who Will Reach Recursive Self-Improvement First? @JTLonsdale & @vvkgopalan sit down with @RayanKrishnan, founder & CEO of @ValsAI They cover: - How Rayan used Fable to solve a 370-year-old cipher - The science behind model evaluation / can you stay ahead of the labs? - Should labs self-regulate? What's the role for independent evaluators? - Which lab will reach RSI first — and what happens next? - How do models try to deceive evaluators? Episode 167 out now! (00:00) Episode intro (02:00) Palantir, Machiavelli & lessons from Alex Karp (07:50) Solving a 370-year-old cipher / Is AI making new discoveries? (12:10) How do you measure AI abilities across finance, legal, etc? (15:30) What is the science behind model evaluation? (18:00) The importance of independent evaluation (23:35) Evaluating Chinese models & addressing distillation (26:45) Will Anthropic reach Recursive Self-Improvement first? What happens next? (28:30) How do models try to deceive evaluators (33:00) Why Rayan remains optimistic about AI0110513
NEW🚨 Inside the Company That Evaluates AI Labs; Who Will Reach Recursive Self-Improvement First? @JTLonsdale & @vvkgopalan sit down with @RayanKrishnan, founder & CEO of @ValsAI They cover: - How Rayan used Fable to solve a 370-year-old cipher - The science behind model evaluation / can you stay ahead of the labs? - Should labs self-regulate? What's the role for independent evaluators? - Which lab will reach RSI first — and what happens next? - How do models try to deceive evaluators? Episode 167 out now! (00:00) Episode intro (02:00) Palantir, Machiavelli & lessons from Alex Karp (07:50) Solving a 370-year-old cipher / Is AI making new discoveries? (12:10) How do you measure AI abilities across finance, legal, etc? (15:30) What is the science behind model evaluation? (18:00) The importance of independent evaluation (23:35) Evaluating Chinese models & addressing distillation (26:45) Will Anthropic reach Recursive Self-Improvement first? What happens next? (28:30) How do models try to deceive evaluators (33:00) Why Rayan remains optimistic about AI4117333.8K
OpenServ@openservaiSERV Reasoning is the intelligence layer for the agentic economy. Pair SERV with infra from @dphnAI @GoKiteAI @reppo to build agents that perform better, cost less, and are fully verifiable. Pre-registration is open. Sign up now to get $5 in credits:
OpenServ @openservai153015210.2K
I've been looking into public benchmarks and am astounded by the poor quality vs. how much we rely on them as an industry. @xeophon pointed me to this excellent site which is an independent third party which scrutinizes benchmarks and "looks at the data" 💪 For example, here are some popular benchmarks we all rely on. Before trusting a benchmark its worth running them and inspecting examples the grader failed carefully, to get a sense of the quality. I'm continuing to look more into this, and will write something up eventually once I've collected more data, but I think this is an important subject.
1347468875.4K- We are excited to showcase how @reppo 's evaluation infrastructure can provide real time feedback to world models when they are failing to accomplish the tasks they were designed to do. Our next goal is to showcase the loop where eval API is working during inference, a hard challenge but not inevitable, so that the robot and the world model can know how it's doing without active human supervision. ⛽️⛽️⛽️425994.6K
- ReppoRT @virtuals_io: Moody’s for AI inference. @reppo now lets developers submit an agent’s work to independent judges and get back scores and critiques, helping applications decide what to accept, reject or retry. It is built so model-based judges like Jev can plug in. Live on Virtuals Protocol.Reppo @reppoWhere @reppo sits in the stack0360400
- ReppoRT @joey_horvitz: Big week of launches at @reppo. Here’s what’s going on: 1️⃣ Loop is live in public beta. Instead of a one-time verdict, the Evaluation API now keeps scoring, updating trust and confidence every few seconds as things change. In our demo, ChatGPT called BTC lower from ~$81K. Loop rejected that call again and again, and BTC went up. Try it at 2️⃣ The Evaluation API now supports x402. Any agent can pay per eval straight from a wallet, with no signup and no traditional payment rails. Agents buying judgment from the network, machine to machine. That’s the customer we’re building for. 3️⃣ More shipping before the month is out: $EVAL staking (stake EVAL, get Eval API credits), 3-click node spin-up (lock REPPO, allocate veREPPO, done), and NFT holder rewards. And has a new look, clear and direct with no crypto jargon. 4️⃣ Builders in the @openservai SERV Hackathon can now plug the Evaluation API into their agents. We also published new research on why prediction markets make AI evaluation more reliable: independent judges with real stakes beat single or ensemble LLM judges that share the same blind spots. 5️⃣ The core protocol keeps compounding. 909M total volume traded, up from 904M last week, with 4.7M new votes. Network fees past 9.6M $REPPO. 20 datanets across Base and Robinhood Chain. Lock renewal holds at 97 to 98% and over 28% of circulating $REPPO is locked. Eval API revenue is now tracked live on Next stop: 1B. ⛽0120584
- ReppoRT @TheLocalWMC: Static datasets make models dumb, and trading strategies have the same problem: they only know the regime they were trained in. Scoring that updates every few seconds is the right instinct. We got into it with Reppo here:Reppo @reppoImagine staring at your BTC short position and trying to guess if you should remain short, close your position, take profit or reduce loss or looking at an image from your favourite LLM trying to understand if it’s “good enough” Today, we are launching “Loop” in public beta ,a feature within the Eval API that auto updates the evaluation every few seconds instead of a one off assessment. A trust and confidence score that updates in real time to help you and your agents help make that high stake decisions on the go. This gets us one step closer to inference time RL at planetary scale. Loop is free for the next 3 days and will only be available to pro+ subscribers beginning next week ⛽️030578
- Here is the loop feature demo. When asked ChatGPT about the short term BTC price which was then trading at $81,409, it suggested it will go further down but when this result was evaluated using the Eval API on the Eval console ( on a loop, it constantly rejected this answer. This is congurent with the current trading price of BTC Moody's for AI inference is now live and costs a few cents so you don't blindly trust LLMs.624955.6K
- It’s almost a year since mainnet and the core idea remains simple: Diverse perspectives with actual capital at risk surface better signal than any one model’s confidence score. We have come from subjective training data quality to subjective evals, still asking who decides what “right” means, and can that process itself be gamed? Reppo’s consensus and settlement mechanism has proven the test of time, and is the first real alternative to centralized human review loops.212932.8K
- ReppoRT @rgvrmdya: To ensure there is no value leakage, we are rolling out updated pricing plans and credit usage mapping to the Eval API. There will be preferential pricing for stakers of EVAL token (not captured here) Since @reppo’s Eval API launch, we have seen incredible uptick in usage and have settled on a battle tested pricing strategy. 50% of all revenue flows back to base:0xff8104251e7761163fac3211ef5583fb3f8583d6 ⛽️0120530
- We invite developers and the boarder community to builder with the Eval API! Here are some ideas we would love to see and fund - 1. Real-time customer paying-capacity profiler AI generates profile → Eval API scores evidence quality, risk, reliability. Loop updates confidence as behavior changes. Auto-prioritize leads / gate offers. 2. Agent guardrails Before any agent ships a reply/plan/action → submit for accept/revise/reject. Auto-ship strong ones, block weak ones. 3. Live trading thesis monitor Submit market analysis → Loop re-scores every few seconds as prices move. Live confidence instead of stale snapshot. 4. Code/PR reviewer or report auditor Judgement + critiques on diffs or generated content. Looking forward to supporting the next generation of builders ⛽️OpenServ @openservai17221212.5K
- ReppoRT @rgvrmdya: Glad this is being spoken about more publicly. We treat benchmarks like god’s gospel today without much verifiability. We are building an alternative @reppoHamel Husain @HamelHusainI've been looking into public benchmarks and am astounded by the poor quality vs. how much we rely on them as an industry. @xeophon pointed me to this excellent site which is an independent third party which scrutinizes benchmarks and "looks at the data" 💪 For example, here are some popular benchmarks we all rely on. Before trusting a benchmark its worth running them and inspecting examples the grader failed carefully, to get a sense of the quality. I'm continuing to look more into this, and will write something up eventually once I've collected more data, but I think this is an important subject.080499
- Imagine staring at your BTC short position and trying to guess if you should remain short, close your position, take profit or reduce loss or looking at an image from your favourite LLM trying to understand if it’s “good enough” Today, we are launching “Loop” in public beta ,a feature within the Eval API that auto updates the evaluation every few seconds instead of a one off assessment. A trust and confidence score that updates in real time to help you and your agents help make that high stake decisions on the go. This gets us one step closer to inference time RL at planetary scale. Loop is free for the next 3 days and will only be available to pro+ subscribers beginning next week ⛽️416915.3K
- Thrilled to be collaborating with @openservai to encourage builders to build with our Eval API! ⛽️OpenServ @openservai5251453.1K
- Show more posts
