Braintrust✓

Your team deserves more from your agent observability platform - a single, connected place for instrumentation, investigation, and measurement, enhanced with intelligence. In Braintrust, you can start with an open-ended prompt about agent behavior and carry the investigation into code, evals, automation, and monitoring. New tools like Patterns and Debugger make it easier to find recurring behavior, get to root causes, and act on what you learn. The context you build in one step automatically follows to the next. Work in the platform with an enhanced Loop experience, or move to your coding agent of choice. Read more →

Your agents have patterns. Braintrust helps you find them. Learn how at re:Invent. Karaoke, after parties, and all your questions about agent observability, answered. See a demo, meet with the team, then join Braintrust and friends off the show floor to relax and unwind.

@cartesia @llama_index @p0 @modal @turbopuffer Agent Afters Let loose with the strip’s only all-vinyl, all-the-time DJ. Hosted with @browserbase, @modal, and @NotionHQ. Superfrico, at The Cosmopolitan. Wed, Dec 2, 8–11 PM. Register →

@cartesia @llama_index @p0 @modal @turbopuffer @browserbase @NotionHQ And don’t forget to stop by our booth for demos, swag, and a friendly team to answer your questions. See you there.

RT @imabhiprasad: You should be evaling how your coding agents use skills! You can do some really cool context optimization with it. I wrote a skill to do this leveraging coding agent traces. The skill is also built to be iterated on (via braintrust datasets/evals), so you can run the skill with an agent in a cron and keep improving your skills incrementally over time.

We tested three databases on finding the only input containing a specific phrase across 4 million spans. Brainstore did it in 2.25 seconds. Read more →

@ Jev, Clef, Luna, Mammoth-1: is this content harmful? We tested decision models, standard LLMs, and specialized safety classifiers on thousands of human labeled examples from three public safety benchmarks to see how well they recognize harmful content. Jev stood out for its performance at low cost. Mammoth-1 raised the fewest false alarms on safe answers among the decision models. Luna caught more harmful answers than Jev but was 3x as likely to flag false alarms. Read more →

Which AI assistant is the best personal shopper? It depends on how you shop. Dots is best if you care about about price, Muse is the most careful product researcher, and Instinct is the most well-rounded across tasks. Read more →

RT @ankrgyl: We now automatically partition and summarize trajectories when you view them in Braintrust. It's surprisingly cheap with the most affordable LLMs, and makes it much easier to navigate long multi-turn traces.

Bubble recently launched a completely new version of its core app-building Agent. Braintrust helped make it 1.5x more accurate than its predecessor at visually representing user prompts, while also performing faster and more cost efficiently. Read more →

Phrase search with no matches. We tested Nitro against other databases. See the benchmarks →

What's new: - Trace your team's coding agents to track costs, usage, and recurring issues - Review agent requests, actions, and responses in a chat conversation style thread view - Prioritize the most important issues in your traces with Patterns severity labels - Send alerts from Loop automations directly to PagerDuty - Ask questions about your dashboards and build new charts with Loop - Trace Cursor agent sessions from the CLI or desktop app Read more →

Run evals at half the cost by using AI provider batch APIs in Braintrust. Large regression suites, model comparisons, and scoring jobs can be batched at a discount if your deadline is flexible. Submit the requests to the provider, collect the results when finished, and track everything in one Braintrust experiment. Now available for both TypeScript and Python →

Export your OpenTelemetry traces from the Claude platform directly to Braintrust. Measure production behavior, eval agent quality, and continuously improve performance. Read more →

RT @daRubberDuckiee: being able to use Codex to automate: text into Muse -> grab results -> save them to Braintrust -> continue until done with all 25 prompts... insane.

We gave AI assistants Muse, Instinct, and Dots the same personal shopping tasks and let them loose on the web. We evaled their product recommendations on response quality, adherence to budget, and how often they recommended the same products. Instinct performed best overall in our eval, but there are tradeoffs depending on how you shop. Dots stood out for finding good options at lower prices, while Muse was best for shoppers with strict guidelines. Read more →

Send your application logs to Braintrust. Connect app logs to agent traces to understand what went wrong and why. Debug with the context from both in one view without switching tools. Learn more →

Trace, inspect, and improve your coding agent's work. See what's driving costs, find recurring problems across your team's sessions, review suggested fixes, and measure the impact of your changes. Coding agent insights is now in public preview. Read more →

Understand where your agent went wrong with the updated thread view in Braintrust. Follow your agent's trajectory as a back and forth chat conversation, with tool calls and sub-agent work in context. Works for complex and custom agents built with any framework. Now available in public preview. Read more →

We ran evals on Opus 5.5 and new GPT-6 models to assess both problem solving and writing quality. - Opus 5.5 had the strongest observed results, but it comes at a cost. - GPT-6 Luna is the standout on value, and is best for running an agent at volume. - GPT-6 Sol balances writing, task performance, and latency, without Opus prices. Read more →