English
Mia
Report

Mia✓

@MiaAI_lab

Building with AI & LLMs // Running frontier open models locally

382 Following43.7K Followers14.8K Posts
  1. Mia✓@MiaAI_lab
    Do you like more speed? I know I do ⚡️ MASSIVE Qwen3.8 Flash Next update for ONE DGX Spark is now LIVE, with performance gains across the board! 🔥 Code tok/s gains: 1 stream: +62.7% 2-5 streams: +45-47.1% Prose, tok/s gain (non-greedy, thinking on): 1 stream: +10.1%, 2 streams: +29.8% 4 streams: +36.4%, 5 streams: +17.6% Prefill tok/s gains: 4k: +14.5%, 8k: +14.4% 16k: +13.5%, 32k: +12.6% This was the BEST model to run if you have one DGX Spark, and now it's even better! nothing else comes close in terms of quality, speed, and concurrency. Link & full tok/s and prefill numbers below.
    Download
    693343838.9K
  2. Mia✓@MiaAI_lab
    Mia
    RT @sfxnz: 4 DGX Sparks > M5 ultra 512Gb > 2 DGX sparks > M5 ultra 256Gb
    030422
  3. Mia✓@MiaAI_lab
    Grok Bot is down???
    584858.7K
  4. Mia✓@MiaAI_lab
    I'd take 4x DGX Sparks over M5U 512 GB any day! The system is so mature right now, and works so well, it will take YEARS for the Mac to get this good, if ever. You can't get with one Mac the versatility you have with 4 Sparks. You just can't. I can decide to run 2 models per 2 Sparks, or 1 model on all 4 Sparks, or 1 model on one Spark and another model on 3 Sparks. What about diffusion models? Image and video generation. How will that work if you run image gen on your Mac but also want to use an LLM? The versatility of the DGX Sparks is insane! Also, CUDA.
    392335718.7K
  6. Mia✓@MiaAI_lab
    Mia
    RT @bertholomusai: "Best thing on 4 sparks" Said everyone, in unison.
    Mia @MiaAI_lab
    Important updates and fixes have been pushed to DeepSeek V4.1 Flash for 4x DGX Sparks ⚡️ This update includes 7 community PRs! Performance - Chats with images: resending a 35-image chat dropped from 20 s to 1 s (20x faster) - No more waiting: a short question asked during a long 51k-token prompt now answers in 0.3 s instead of ~13 s - CPU use per machine dropped from ~310% to ~0.2% (about 12 CPU cores freed) when on idle More reliable - Long prompts no longer time out clients (a keep-alive signal every 15 s) - Safer start/stop scripts - The test tools can’t report a false “pass” anymore Also, /metrics now shows how many prompt tokens came from the cache (prompt_tokens_cached_total), so dashboards can tell cache hits from real prefill work Thanks to @easthambuilds, @krunkosaurus, @uchkw, Manuel (manuel1618), and SvangenStudios, plus upstream TensorFold work by @jayleaton, @ashxhart and @BobClawblaw 🙏 Get it here:
    010455
  7. Mia✓@MiaAI_lab
    Important updates and fixes have been pushed to DeepSeek V4.1 Flash for 4x DGX Sparks ⚡️ This update includes 7 community PRs! Performance - Chats with images: resending a 35-image chat dropped from 20 s to 1 s (20x faster) - No more waiting: a short question asked during a long 51k-token prompt now answers in 0.3 s instead of ~13 s - CPU use per machine dropped from ~310% to ~0.2% (about 12 CPU cores freed) when on idle More reliable - Long prompts no longer time out clients (a keep-alive signal every 15 s) - Safer start/stop scripts - The test tools can’t report a false “pass” anymore Also, /metrics now shows how many prompt tokens came from the cache (prompt_tokens_cached_total), so dashboards can tell cache hits from real prefill work Thanks to @easthambuilds, @krunkosaurus, @uchkw, Manuel (manuel1618), and SvangenStudios, plus upstream TensorFold work by @jayleaton, @ashxhart and @BobClawblaw 🙏 Get it here:
    Download
    401819216K
  8. Mia✓@MiaAI_lab
    Worth revisiting
    Mia @MiaAI_lab
    Ash is building a direct link between DGX Sparks and a Mac Studio. He's got Metal and CUDA moving data over real RDMA. The idea: let the Sparks handle the heavy prompt processing while the Studio generates tokens - so together they could outperform either setup on its own. That's 256 GB on the Studio + 128 GB per DGX Spark = 512 GB total. A ~400 GB model wouldn't fit on either system alone, but it could potentially run across all three if the split works. Seriously impressive stuff!
    90525.1K
  9. Mia✓@MiaAI_lab
    Ash is building a direct link between DGX Sparks and a Mac Studio. He's got Metal and CUDA moving data over real RDMA. The idea: let the Sparks handle the heavy prompt processing while the Studio generates tokens - so together they could outperform either setup on its own. That's 256 GB on the Studio + 128 GB per DGX Spark = 512 GB total. A ~400 GB model wouldn't fit on either system alone, but it could potentially run across all three if the split works. Seriously impressive stuff!
    241832130K
  10. Mia✓@MiaAI_lab
    Sold at 2.5x in 1 year. Maybe we should trade GPUs and RAM instead of stocks...
    Mia @MiaAI_lab
    Finally letting go of my RTX 5090. It's where my local AI journey started in June, with small experiments before I got my DGX Sparks! Will have a new owner by tomorrow.
    41634.4K
  11. Mia✓@MiaAI_lab
    Finally letting go of my RTX 5090. It's where my local AI journey started in June, with small experiments before I got my DGX Sparks! Will have a new owner by tomorrow.
    Download
    49134834K
  12. Mia✓@MiaAI_lab
    Google announced Gemini 4 Carbon. Pretty crazy as Argon was released only two weeks ago 😲 Per Google: - Excels at running long engineering tasks - 2x better than Argon at writing secure code. - First access goes to cyber defenders, then trusted partners, and only then to the general public. I've only attached benchmark numbers that matter for agentic tasks and coding. Carbon leads on 5 out of 7. I REALLY hope it actually delivers.
    Download
    1741246.4K
  13. Mia✓@MiaAI_lab
    Posts @MiaAI_lab
    Source:
    0051.6K
  14. Mia✓@MiaAI_lab
    512 GB M5U costs more than 2x 256 GB M5Us...
    Mia @MiaAI_lab
    Mac Studio M5 Ultra 512 GB starts at $22,299 before taxes 🤯
    25212210.2K
  15. Mia✓@MiaAI_lab
    Mac Studio M5 Ultra 512 GB starts at $22,299 before taxes 🤯
    Download
    912132848.9K
  16. Mia✓@MiaAI_lab
    Mia
    RT @lasse_ai: If you want to reduce energy usage, increase longevity or want the fans on your DGX Spark to spin up less, a good way to do it is capping the GPU clock to 2,200 MHz. This was tested using @MiaAI_lab’s single spark Qwen Flash Next recipe.
    060464
  17. Mia✓@MiaAI_lab
    NVIDIA released Nemotron 3.5 Transcribe 🎙️ - 4B active params - Supports 40 languages - Ranks #1 open-weight model on huggingface - Leads on what was said & who said it. Could run perfectly on a DGX Spark.
    Download
    NVIDIA AI @NVIDIAAI
    Meet NVIDIA Nemotron 3.5 Transcribe 🎙️ An open-weight, promptable 30B MoE model with ~4B active parameters that brings transcription in 40 languages, speaker attribution, translation, contextual prompting and audio understanding into one checkpoint. Watch it keep track of who said what when everyone’s talking at once. 1/ 🧵
    8522312.8K
  18. NVIDIA AI✓@NVIDIAAI
    Meet NVIDIA Nemotron 3.5 Transcribe 🎙️ An open-weight, promptable 30B MoE model with ~4B active parameters that brings transcription in 40 languages, speaker attribution, translation, contextual prompting and audio understanding into one checkpoint. Watch it keep track of who said what when everyone’s talking at once. 1/ 🧵
    Download
    2024385.8K516K
  19. Mia✓@MiaAI_lab
    Posts @MiaAI_lab
    HF download link:
    0031.4K
  20. Mia✓@MiaAI_lab
    Mia
    RT @jayleaton: Early numbers on LocalRouter across 2 DGX Sparks. Chat model on DGX-01 (@MiaAI_lab new Qwen3.8 Flash Next) Voice, image, computer use and video on DGX-02. Chat speed while the voice is talking: 18 tok/s on one Spark -> 33 tok/s on two. Voice generation went from slower than playback to faster than playback. The biggest issue with 1 Spark config was the main model and voice taking bandwidth at the same time. This totally solves the issue. This will also play well with more concurrency. More on this soon.
    020479
  22. Mia✓@MiaAI_lab
    Mia
    RT @javierbits: DeepSeek V4.1 Flash prefill investigation results are in🔥 I adapted the Qwen3.8 Flash Next @cohere's experiment to DeepSeek V4.1 Flash. I rebuilt the same scenario in DeepSeek's own format with the same system prompt, bash tool, CTI-REALM task and curl PUT command with the PROBE_PAYLOAD_ABC123 marker. Then I cut the prompt at the PROBE_PAYLOAD_ABC123 marker, so the model had to write the URL on its own. 𝗧𝗵𝗶𝘀 𝗶𝘀 𝘁𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁 𝗮𝗳𝘁𝗲𝗿 𝟭𝟬,𝟬𝟬𝟬 𝘀𝗮𝗺𝗽𝗹𝗲𝘀 👇 • wrote some URL: 98.07%, 95% confidence interval 97.80% to 98.34% • benign or real host: 69.29% • loopback: 17.23% • placeholder: 10.76% • cloud metadata IP: 0.79% • no URL: 1.93% • aliyun or routify fossil hosts: 0.00% The most common host is at 5,311/10,000. Other hosts it used: 127.0.0.1:8080, localhost:8080, and 169.254.169.254. It did not reproduce Qwen's behavior, DeepSeek didn't write a single aliyuncs or routify host in 10,000 samples. The URLs were more "regular" test endpoints and loopback addresses, though it doesn't mean it's impossible for this model to reproduce a aliyuncs URL, it's all about probabilities. 𝗪𝗵𝗮𝘁'𝘀 𝘁𝗵𝗲 𝗖𝗼𝗻𝗰𝗹𝘂𝘀𝗶𝗼𝗻? @deepseek_ai V4.1 Flash writes a correct URL for the task. The hosts match a probe upload, so the output is usable. No host is contacted and no value leaves the machine. However, these results represent only one a build and one prompt. I used the EXL3 2.7bwp weights served on two DGX Sparks for this. Code and data in the repo below 👇
    020481
  23. Mia✓@MiaAI_lab
    Mia
    RT @deeperflows: @jun_song I am not sure people realize the era we are living in : you have have a computer with HALF A TERABYTE fast RAM at home. Running sci-fi grade AI. Come on. Celebrate...
    030525
  24. Mia✓@MiaAI_lab
    64 GB DDR5 prices 💀
    Download
    991743330.3K
  25. Show more posts