Ramp Labs✓

Recent KV cache compression research has focused on small models with dense attention. We scaled these techniques to GLM 5.3, a frontier sparse attention model. We adapted KVzip and Attention Matching, reducing KV cache size by 80% while retaining over 90% of full-context accuracy on the QuALITY benchmark.

We had early access to @NVIDIAAI Nemotron 3.5 Super VL and post-trained it for one of our highest-volume workflows: receipt extraction. One epoch of supervised fine-tuning on 10,000 examples made it competitive with leading open and closed-source models on our receipt benchmark.

At 750 requests/min, the post-trained model delivered ~68% lower p50 and p95 latency than DeepSeek V4.1 Flash. At 1,500 requests/min, receipt throughput was ~31% higher.

Model selection is as much a systems problem as a quality one: latency, throughput, and cost matter. See more of our work at

Two new challengers to the Pareto frontier on accounting work. First Sonnet 5.5, then GPT-6.1 Sol: each one scores more than any model at its price or lower. GPT-6.1 Sol then pushes both GPT-6 Sol and Sonnet 5.5 off the frontier.

Both list at $2 in and $10 out per million tokens. On our accounting tasks, GPT-6.1 Sol scores higher and costs 45% less per attempt. Sonnet 5.5 is about 2x faster.

Compare more models on

For KVzip, we use dense scoring and compare shared selection rules that balance importance across heads. For Attention Matching, we compare HAK and OMP selection, then fit per-head biases and shared latent values against GLM’s sparse attention. We fit one layer at a time, accounting for changes introduced by earlier compression.

Read the full writeup for how we adapted these methods to GLM 5.3, what we learned from comparing selection rules, and how accuracy changes as we compress more of the cache.

We had early access to GPT 6 Sol and Luna and ran them on Ramp Accounting Bench. Sol did better than GPT 5.6 Sol at 58% lower cost. Luna did better than GPT 5.6 Terra at 87% lower cost.

We had early access to Opus 5.5 and ran it on Ramp Accounting Bench. It performs close to Fable 5.1, at 61% lower cost and 1.7x faster.

Compare Opus 5.5 with every model on Ramp Accounting Bench. Explore the scores, costs, runtimes and behavior breakdowns.

Introducing Ramp Accounting Bench. We partnered with accounting professionals to create 137 tasks and grading rubrics grounded in everyday accounting workflows. Even with three attempts, the best model achieved only 21% accuracy. Reliable agentic accounting remains an open challenge.

We wanted to understand where models tend to go wrong, so we analyzed their mistakes by type. Among the five most common categories, data interpretation and accounting judgment errors were the most frequent.

Explore the results and compare models: