David Zats✓

RT @anurahk: More and more, image models get ranked by human vote. A vote can miss that your font changed. Today @DavidZats and I are releasing VybBench, an open benchmark that scores design fidelity with numbers instead of votes. Ask a model to change one headline and it can nudge your font, shift your blue and redraw a corner of your logo too. Each edit changes a little more, until the design isn't yours. VybBench scores each piece: 145 fonts, matched pixel by pixel against the real font 100 hex codes, scored on how far each color drifts 200 small edits, scored on how many pixels changed, with a judge checking the edit was made We are not including model scores in the post, but you can see how each model scores on the Hugging Face. Every prompt, output, judge verdict and the scorer is on Hugging Face, link in the reply. Full Disclosure: At @cocoailabs we also build Vyb - our first design model built to preserve design fidelity and taste.

RT @anurahk: The method, the data and the scorer are on Hugging Face: The full write-up:

RT @anurahk: I launched the first image and video generation models at Google, and kept asking why all AI designs look the same. AI has brought us the WORST era of design. It made content creation effortless, but it also made 90% of it look painfully generic. My co-founder and I spent 2 years building in stealth to change that. Meet Vyb, the first AI design model that understands your taste like a designer would. RT + comment your website URL and we’ll make your first design.

"it should never look like it was made by AI. It has to feel handcrafted." That was the design standard at until now. Praveen Das, the company's CMO and co-founder, shared it in September. Then he added: "I'm starting to question that." Isha Kumar, a product designer at Linear, runs only about 20-30% of her net-new design work through AI. She does not use it to critique her designs or to write her UI copy. Her reason: "It wants to affirm me, not provoke me" But they show where design stays human even when AI helps. It stays in the taste, in the final call, and in knowing which parts of the work to keep for yourself.

Games Workshop makes Warhammer. In a report to its shareholders, in writing, it said: "we do not allow AI generated content or AI to be used in our design processes" Wizards of the Coast is the company behind Magic and D&D. Its FAQ on generative AI art says: "we require artists, writers, and creatives contributing to the Magic TCG and the D&D TTRPG to refrain from using AI generative tools to create final Magic or D&D products" Dove is a Unilever brand. In April 2024, marking 20 years of its Real Beauty campaign, Dove committed "to never use AI-generated content to represent real women in its advertisements" These look like small details. I think they will matter more as AI helps with everything around that human part.

RT @anurahk: The only way to edit an AI-generated design is to generate another one. The text in it is not really text. It is part of the picture, so a one-word fix comes back as a whole new image, different everywhere. People try to get around that by pulling the layers back out. The model never drew the background behind the logo, so it has to invent what was underneath. It reads the text off the picture and picks a font that looks close. You end up with a copy of the design you already had.

RT @anurahk: Design generation is not the same as image generation. An image model makes one flat layer of pixels, the same way it would make a photograph. It looks like a design. But the logo in it cannot be moved and the headline cannot be rewritten, because they are not separate things. They are pixels. So the only thing left to do is ask for another one, which is not the right process for designers. Design generation means building the design out of a company's own logo, fonts and photos, and placing every part of it on purpose. A better image model does not get you there.

RT @anurahk: Design is judged on novelty, so companies throw out ads that still look good and pay for new ones. A model gives you back the patterns it has seen most, which means you can pay for a design and get one that could belong to your competitor. Nobody has worked out how to get new design work into a model. I would not say it cannot be done. I think it is the open question underneath all of this.

Reliable agentic frameworks are hard to build. Because LLMs can answer almost anything, there is a tendency to hand them an entire task at once and let it run. Suppose a design takes 5 steps, and each one works 9/10 times. The finished design works a little more than half the time, because the failures accumulate. By splitting up complex tasks like design into the well-defined, well-tested steps that a designer would take, we can increase the success rate of each step, leading to a much higher overall reliability.

A model trained on one skill is smaller by design as opposed to by compromise. Most of the capacity in an answer-anything frontier model sits idle on any given request. This is incredibly inefficient and costly.

RT @anurahk: A model built for one domain already understands the context you are working in, so you are not describing it again on every request. That is most of what makes a domain-specific model easier to prompt than one built to answer anything.

A general model has no memory of your domain between requests. The full specification has to arrive in the prompt every time, and the model substitutes whatever it has seen most often. For design, that specification is large, and almost none of it is information about the world.

RT @anurahk: Open weights decide who can realistically build a model. Training one from nothing takes the resources of a frontier lab. By allowing companies to avoid the prohibitive costs of starting from scratch, companies can create state of the art models for specific domains. @cursor_ai post-trained @Kimi_Moonshot K2.5 into Composer 2.5. @perplexity_ai built its Sonar search model on Llama 3.3 (@AIatMeta) @harvey post-trained Kimi K3 into Tenet, its legal model. All of them are doing work a frontier lab was never going to do for them.