Agents & Developer Experience Ecosystems Lead EMEA at Google. O'Reilly author. Local AI, fine-tuning. Building tool-eval-bench & awesome-local-ai. Views my own.
I couldn't keep track of all the local AI tools flying around anymore. New engines, harnesses, tooling, benchmarks... every week.
So I started awesome-local-ai: a list sorted by what you want to do, not by hype.
The twist: it's built for agents too. Point your coding agent at it and ask it to recommend a setup for your hardware.
It's early and DGX Spark-heavy. Help me fill the gaps:
Pascal: "I have made this longer than usual because I have not had time to make it shorter."
AI gave us the time. We spend it generating more.
Add to AGENTS․md:
1. Answer first.
2. Cut every sentence that doesn't change what the reader knows or does.
3. No preamble, no recap.
Welcome, Le Chonk! Glad to see a European model competing with the open weights frontier.
10 style points for the name and graphics 🔥
Share
Copied
Mistral AI@MistralAI
Today, we release the weights of a new and improved Le Chonk, advancing the open frontier for agentic coding and cybersecurity.
💻 Built to build: Le Chonk ranks #1 on @cognition’s FrontierCode among open models, and sets a new benchmark for Pareto efficiency on @artificialanalysis.
🛡 Built to defend: State-of-the-art in cyberdefense, put through the paces by the world's leading cyber vendors, including @CrowdStrike, @cloudflare, @rubrik, @zscaler, and others.
Built for sovereignty: Make it yours. Download the weights on @huggingface , customize it with Mistral Forge, and deploy it wherever you need it.
Read more from our team and customers 👇:
Today, we release the weights of a new and improved Le Chonk, advancing the open frontier for agentic coding and cybersecurity.
💻 Built to build: Le Chonk ranks #1 on @cognition’s FrontierCode among open models, and sets a new benchmark for Pareto efficiency on @artificialanalysis.
🛡 Built to defend: State-of-the-art in cyberdefense, put through the paces by the world's leading cyber vendors, including @CrowdStrike, @cloudflare, @rubrik, @zscaler, and others.
Built for sovereignty: Make it yours. Download the weights on @huggingface , customize it with Mistral Forge, and deploy it wherever you need it.
Read more from our team and customers 👇:
Enjoying a couple of days off with the family in Crete. My highlight so far? A loop up to the Lasithi Plateau: 91 km, 1,609 m of elevation. My biggest climb in one go outside of my Zwift Everesting.
Time off to recharge, gather new ideas and invest in wellbeing. The work is better for it when I'm back.
Vacation season, and I'm enjoying how quickly @GenkitFramework lets me turn ideas into agentic apps. It's an open-source SDK for TypeScript, Go, Dart and Python, works with any model, and its local Developer UI shows every tool call.
Agent skills and the MCP server let your coding agent run your flows and read the traces.
Great post on measuring a quant's quality by @mhelcig and @adotdad.
TL;DR: Tuning only the quantization scales of a 3-bit model on the eval text took its KLD from 0.205 to 0.048, the same as the 4-bit model's. On held-out text it was 0.149, and GSM8K was 85.6% against 90.8% for 4-bit.
Worth reading before you trust a quant ranking. KLD is a diagnostic, not a leaderboard. Your own tasks decide.
Introducing a new little project of mine: LabTunnel.
A macOS menu bar app that forwards a port from an SSH host to localhost on your Mac. Flip a switch, open the page. Handy for MVPs and for login flows on headless devices.
Agent-friendly by default.
For scripts and agents, there's a labtunnel CLI with JSON output and fixed exit codes, plus a bundled SKILL.md:
labtunnel add --name Grafana --host homelab --remote-port 3000 --connect
PyTorch 2.15 makes 4-bit NVFP4 decoding on Blackwell cheaper and therefore faster by removing one kernel launch per layer: the dequantization scale now folds into the GEMM.
Why NVFP4 is not just another 4-bit format: 16 values share one FP8 scale (MXFP4 uses 32 values and a power-of-two scale), plus one FP32 scale per tensor. Finer scales mean less quantization error.
Share
Copied
PyTorch@PyTorch
PyTorch 2.15 is here, with 3,117 commits from 583 contributors and updates across distributed training, NVGEMM, Apple Silicon, and accelerator platform support.
The release makes the rewritten NCCL backend the default and adds one-sided transports with a NIXL backend. FSDP2 and pipelining gain finer memory and communication control. It also brings faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs, FP8 and faster decode on Apple Silicon, and broader platform support.
On Thursday, November 5, at 11 AM PST, join Tristan Rice (@rice_fry), Natalia Gimelshein, Huy Do, and Jane Xu (@janeyx99) (@Meta) for a live PyTorch 2.15 Q&A, moderated by Chris Gottbrath (Gottbrath Tech). The speakers will share an overview of the release and answer community questions about PyTorch and the new capabilities in this release.
🔗 Read the release blog and register for the live Q&A:
PyTorch 2.15 is here, with 3,117 commits from 583 contributors and updates across distributed training, NVGEMM, Apple Silicon, and accelerator platform support.
The release makes the rewritten NCCL backend the default and adds one-sided transports with a NIXL backend. FSDP2 and pipelining gain finer memory and communication control. It also brings faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs, FP8 and faster decode on Apple Silicon, and broader platform support.
On Thursday, November 5, at 11 AM PST, join Tristan Rice (@rice_fry), Natalia Gimelshein, Huy Do, and Jane Xu (@janeyx99) (@Meta) for a live PyTorch 2.15 Q&A, moderated by Chris Gottbrath (Gottbrath Tech). The speakers will share an overview of the release and answer community questions about PyTorch and the new capabilities in this release.
🔗 Read the release blog and register for the live Q&A:
I'm pretty excited about Android MCP. Android is leaning into becoming the agent-friendly OS!
Share
Copied
Mishaal Rahman@MishaalRahman
✨ This is huge 👀: We're launching Android MCP, powered by the AppFunctions API!
This lets your app integrate with @GeminiApp on Android so your users can ask questions that pull info from your app or take actions in it.
To support this, we're also launching a preview of the Connected Apps Platform.
This is the announcement I'm most excited about from #AndroidDevSummit!
✨ This is huge 👀: We're launching Android MCP, powered by the AppFunctions API!
This lets your app integrate with @GeminiApp on Android so your users can ask questions that pull info from your app or take actions in it.
To support this, we're also launching a preview of the Connected Apps Platform.
This is the announcement I'm most excited about from #AndroidDevSummit!
Welcome, Gemini 4 Carbon!
Massive improvements over Argon -- just two weeks after Argon was released.
Give it a try. I'm sure you'll love it! 🚀
Share
Copied
Logan Kilpatrick@OfficialLoganK
Introducing Gemini 4 Carbon, it's a big jump over Argon which we released just 2 weeks ago.
Gemini 4 Carbon is SOTA on cyber, long horizon tasks, ranks #1 on Artificial Analysis V5, significantly improves on usability in domains like coding, and is more aligned.
Introducing Gemini 4 Carbon, it's a big jump over Argon which we released just 2 weeks ago.
Gemini 4 Carbon is SOTA on cyber, long horizon tasks, ranks #1 on Artificial Analysis V5, significantly improves on usability in domains like coding, and is more aligned.
I like to look at claims like this in case I can improve Ghostty. I grabbed your source and immediately spotted the issue: your read loop overwrites a 64 KiB ring. Everything older is discarded unparsed per tick. You're faster because you're not actually terminal emulating.
It means the number isn't a throughput number. In a kitten benchmark run at the rate you're claiming, roughly 6.6 MB arrives per 60 Hz tick and 64 KiB of it gets parsed. You're read(2)-ing 100% of the data and parsing about 1% of it.
There are other minor issues (a small fixed scrollback in a preallocated ring buffer, no grapheme clustering, throwing away data like OSC 8 URIs, etc.), but the major one is simply that you're discarding ~99% of the input.
For plain text, e.g. your cat benchmark, the final screen ends up looking right, but the scrollback is missing 99% of the file and each retained chunk starts mid-line. As soon as there are escape sequences in the dropped spans, your emulator is fully out of sync and invalid: a cut inside a CSI prints the parameters as text, and any SGR, mode, title, or alt-screen change in the gap is just... gone.
FYI, this is why I don't like when people point at refterm as a terminal benchmark. refterm implements a TINY subset of what a terminal emulator must support, it's closer to a fast file reader, not a terminal emulator.
I also don't mean any shade by this. I like to look at technology objectively to try to learn something and improve my own craft and work. So I'm sharing what I found.
I modified your code very slightly to parse on every full buffer (rather than per tick). And the results are attached. Both use main commit in each repo at the time of this post.
Note the graph is in log scale, the discrepancy between performance for things like Kitty images is measured in GB/s.
I like to look at claims like this in case I can improve Ghostty. I grabbed your source and immediately spotted the issue: your read loop overwrites a 64 KiB ring. Everything older is discarded unparsed per tick. You're faster because you're not actually terminal emulating.
It means the number isn't a throughput number. In a kitten benchmark run at the rate you're claiming, roughly 6.6 MB arrives per 60 Hz tick and 64 KiB of it gets parsed. You're read(2)-ing 100% of the data and parsing about 1% of it.
There are other minor issues (a small fixed scrollback in a preallocated ring buffer, no grapheme clustering, throwing away data like OSC 8 URIs, etc.), but the major one is simply that you're discarding ~99% of the input.
For plain text, e.g. your cat benchmark, the final screen ends up looking right, but the scrollback is missing 99% of the file and each retained chunk starts mid-line. As soon as there are escape sequences in the dropped spans, your emulator is fully out of sync and invalid: a cut inside a CSI prints the parameters as text, and any SGR, mode, title, or alt-screen change in the gap is just... gone.
FYI, this is why I don't like when people point at refterm as a terminal benchmark. refterm implements a TINY subset of what a terminal emulator must support, it's closer to a fast file reader, not a terminal emulator.
I also don't mean any shade by this. I like to look at technology objectively to try to learn something and improve my own craft and work. So I'm sharing what I found.
I modified your code very slightly to parse on every full buffer (rather than per tick). And the results are attached. Both use main commit in each repo at the time of this post.
Note the graph is in log scale, the discrepancy between performance for things like Kitty images is measured in GB/s.
I look forward to seeing what @MiaAI_lab and @ashxhart will do with this. Still counts as local AI, doesn’t it?
Share
Copied
Wall St Engine@wallstengine
A 300-rack AI supercomputer powered by 9,600 Nvidia A100 GPUs is listed on eBay for $59.7M.
That’s 1,200 GPU compute servers with 384TB of combined GPU memory and roughly 1.4 petabytes of system RAM.
Each compute server connects eight A100 GPUs through $NVDA's NVSwitch technology.
More than six years after A100 launched, The seller is offering these systems refurbished for new AI training and inference deployments.
A 300-rack AI supercomputer powered by 9,600 Nvidia A100 GPUs is listed on eBay for $59.7M.
That’s 1,200 GPU compute servers with 384TB of combined GPU memory and roughly 1.4 petabytes of system RAM.
Each compute server connects eight A100 GPUs through $NVDA's NVSwitch technology.
More than six years after A100 launched, The seller is offering these systems refurbished for new AI training and inference deployments.
Google Research gave an LLM a sleep phase, and it stays near-perfect on BABILong up to 10M tokens.
How it works: while awake, new facts sit in short-term memory (context, KV cache). During sleep, the model unlocks pre-allocated low-rank MoE blocks and distills that short-term memory into them, so old weights stay untouched. Then it "dreams": random experts generate practice examples, filtered by gradient, and it trains on the best ones.
The part I find interesting: no fixed knowledge cutoff, and near-zero forgetting when learning Manchu and Kalamang in stages.
Share
Copied
Google Research@GoogleResearch
Today on the blog, learn about our new instance of a “sleep” phase in continual learning, which enables LLMs to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting or static knowledge cutoffs.
More →
Today on the blog, learn about our new instance of a “sleep” phase in continual learning, which enables LLMs to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting or static knowledge cutoffs.
More →
I love it when people push the boundaries of what is possible with models. Looking forward to @mmastrac releasing his take on EmbeddingGemma2 - another great use case for @googlegemma 💎
Share
Copied
Matt Mastracci@mmastrac
It's been a while since I showed any Jev stuff but I've been cooking on a way to use @GoogleDeepMind's EmbeddingGemma2 as an insanely fast real-time /v1/systemone endpoint (20ms on text, 100ms on images).
It's trainable in real-time, can distill an upstream teacher model, and is multi-modal by default. Hard to believe a @googlegemma model of this size can do so well with the right harness!
It's been a while since I showed any Jev stuff but I've been cooking on a way to use @GoogleDeepMind's EmbeddingGemma2 as an insanely fast real-time /v1/systemone endpoint (20ms on text, 100ms on images).
It's trainable in real-time, can distill an upstream teacher model, and is multi-modal by default. Hard to believe a @googlegemma model of this size can do so well with the right harness!
For the first time, most of the Artificial Analysis Intelligence Index runs on private test sets or held-out answers: 60% of v5, per AA. 70% of the weight sits in evals where the agent uses tools.
Private sets keep scores out of training data, which is why tool-eval-bench lets you bring a private tool catalog. The cost is that nobody outside can check them, so a big rerank is hard to verify. Good reason to run your own private eval on the task you care about.
Share
Copied
Artificial Analysis@ArtificialAnlys
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04