Databricks

“Just because the model looks better on the benchmarks doesn't mean it actually got better on your very specific use case.” Model launches make headlines. Our Chief AI Scientist @jefrankle gets into the research, RL, evals, and tradeoffs behind them with @readtechnically.

"You never quite know what you're going to get when you push the frontier." But what does pushing the frontier actually look like day-to-day? What do AI researchers...actually do? On Ep 2 of Technically Speaking, @jefrankle (Chief AI Scientist @databricks) & @meganreyno get into it: → How he runs his team: "If I don't have numbers, I don't believe you." → Why you should ship before you feel ready: "you need to ship the best of what you have at any given time" → Who he hires: "people who are never quite satisfied" 🎬 Watch the full episode: Presented by our friends at @Railway. Stay tuned for some of our favorite moments...

Coding agents can work in parallel. Their databases should too. Lakebase branches an entire Postgres database in under a second, giving each agent an isolated environment for migrations and testing without copying the parent database. How it works:

OBO authorization for Databricks Apps is now GA. Apps can use the signed-in user's existing Unity Catalog permissions, while API scopes limit what they can do on their behalf. Build personalized, governed apps without recreating access rules in code:

Our PMs use Genie One to run weekly adoption reviews, investigate usage spikes, and forecast what happens next. Genie Ontology gives it the context to know which metrics and sources to trust, so the analysis starts with our business logic instead of from scratch. A look at how we use it:

RT @matei_zaharia: Great writeup on what Databricks is and how our tech fits into the broader changing nature of work and companies!

Load billions of rows into Lakebase Postgres without competing with live application workloads. With LTAP, Spark handles bulk loads on separate compute, keeping the primary focused on serving queries.

RT @a16z: Ben Horowitz on Databricks CEO Ali Ghodsi: "You just don't get this good of a market opportunity with this good of an entrepreneur. That's the rarest of rare things." Despite surpassing a $7 billion run rate and growing 80% year over year, Databricks might be the most under the radar story in Silicon Valley. Three years ago, every interaction on Databricks involved a human. Today, the company is seeing more queries from agents than people. Ali believes AGI is already here. Yet inside most companies today, AI still isn't trusted to reliably update records in Salesforce. "AI does not have an intelligence problem," Ali explains. "It has a context problem." Even the smartest human would be all but useless without access to the company's data, permission to act, feedback, or the knowledge in other employees' heads. Ali's answer is what he calls a "system of record for agents." But the bigger question is how agents will change the firm itself. 85 years ago, James Burnham's The Managerial Revolution described how a small class of professional managers centralized the decision-making inside of companies. Today, thanks to a new wave of technology, the craft practitioners at the front lines are regaining responsibility again. The irony is that the decline of managerialism may look more like everyone becoming a manager. In this issue of Profiles of the Future, @bhorowitz and @mikemcg0 explain how seven Berkeley academics built Databricks, and why it will power the future of the firm:

What people do at work all day is changing. Agents can take on more of the work we already know how to do, while people closest to the problem gain more room to investigate and act without waiting on layers of hierarchy. a16z's @bhorowitz and @mikemcg0 explore why the system of record powering agents matters so much to that shift, and why Databricks is well-positioned to power it. Read the full article:

Now available on Databricks: GPT-6.1 Terra. On our OfficeQA Pro v2 benchmark, it shows a large quality improvement and sits on the quality-cost pareto frontier. Use it, or any of 70+ other models, on the data already in your Lakehouse to build governed AI apps and agents. Unity Gateway governs, monitors, and secures every call from a single control plane. Try GPT-6.1 Terra ↓

Finance teams spend too much time figuring out why numbers changed. Genie One helps them investigate variances, cash flow, close exceptions, and spend using their own business context and governed data.

What if agent memory didn’t need another sync pipeline? Join @rxin, Databricks Co-founder and Chief Architect, to unpack the LTAP architecture for agentic apps: • Transactions + analytics read the same copy of data • Agent memory, application state + historical data share the same permissions • Runs on standard Postgres with existing drivers, extensions + ORMs Register:

AI is getting more autonomous, and governance has to keep up. Teams need control over what AI can access and do, a handle on rising AI costs and the freedom to choose the best models, agents and tools. The Big Book of AI Governance shows how leading organizations manage all three at enterprise scale without slowing innovation. Download the eBook →

[DEMO] With the Databricks Excel Add-in, bring the right Databricks data into Excel with more flexibility and less repetitive work: → Generate cascading filter values to narrow data before importing → Reuse existing Databricks SQL queries directly in Excel → Use custom functions like DATABRICKS.SQL and DATABRICKS.Table to import data into your workbook Watch the full demo to see how they work:

A 2,500 GB database with a 20 GB hot working set doesn’t need 2,500 GB of RAM. That principle runs through Lakebase Postgres cost optimization: size around what your application actually needs. ✦ Sync only the working set your app queries ✦ Match Snapshot, Triggered or Continuous sync to freshness requirements ✦ Size compute so hot data fits in cache ✦ Set PITR and snapshots around actual recovery needs The full guide breaks down how to keep costs predictable while maintaining the performance, availability and developer experience applications need:

What if the next big leap in AI agents comes from better memory, not bigger models? @theDeepView explores MemCurrent. In our internal benchmark of 114 enterprise tasks, agent success rose from 22.3% without memory to 68.4% with episodic and derived memory.

Every new agent can become another integration, policy set and observability stack to maintain. As agents take actions across data, models, tools and business systems, that fragmentation gets expensive fast. The better pattern is shared infrastructure for choice, context and control: 1️⃣ Agent Bricks for building, governing and optimizing agent fleets 2️⃣ Omnigent for composing agents across harnesses 3️⃣ Unity Gateway for centralized access, cost controls and observability

With Unity Catalog managed tables, you can keep your data in cloud storage you own, at a location you control. ✓ Set managed storage locations at the metastore, catalog, or schema level ✓ Change where new tables land as requirements evolve ✓ Choose the destination when converting external tables to managed Read our guide to storing Unity Catalog managed tables: ​​

RT @matei_zaharia: Databricks AI Runtime (AIR) is now GA, offering an easy and super-fast way to launch GPU workloads. Unlike provisioning your own GPU servers, you get GPUs with your code in seconds, with easy access to all your data. This is especially relevant now that agents can automate much of AI training, and want to scale up/down experiments quickly. Read how companies like Robinhood, Superhuman, Factset and Merck have been using AIR to accelerate AI training:

Introducing MemCurrent, a persistent memory architecture designed to help production agents stop starting from zero. MemCurrent combines episodic memory with derived memory across three access patterns: • Plugins to log and load context automatically • Tools the agent calls on demand • An offline “dreaming” pipeline that consolidates memory in the background Agents carry forward user preferences, prior corrections, metric definitions and successful workflows instead of rediscovering them every session. On our internal benchmark built from real enterprise data, MemCurrent triples the agent's pass rate while using 31% fewer tokens and running 26% faster. It's built into Agent Bricks' Memory offering and already powers memory in Genie. Read the research:

Databricks and @nvidia are expanding our collaboration so enterprises can customize open models on their own data and take them all the way to production. With AI Runtime, developers can use NVIDIA NeMo, NGC containers and new Nemotron cookbooks to train and fine-tune models on serverless NVIDIA GPUs alongside governed enterprise data. Nemotron 3.5 lightning and Super VL are also ready to load, adapt and serve on Databricks from day one. Customize with governed enterprise data. Serve in production. All on Databricks.

DeepSeek V4.1 Flash 1017 is now available on Databricks as a new endpoint. Same architecture, updated weights, with substantial improvements across reasoning, coding, agentic tasks and visual benchmarks.

More than 1 million Genie Agents were created in 2026. Which ones should you build first? Across dozens of rollouts, the agents that get adopted fastest share a clear pattern: tightly scoped domains, high-frequency repetitive questions, and governed data with strong metadata. One team built a risk and portfolio Q&A agent that saw usage climb before it even reached production. Another had a working agent live in weeks because the demand and data were already there. We put together a simple rubric to help you score and rank candidate workflows before you build 👇

Open table formats are important. But they don’t answer the harder question: who actually controls the data, and can you move it without copying everything somewhere else? That’s the standard we use for openness in Unity Catalog. Your managed tables can live in customer-owned storage. The best table features work on those tables. And with REGISTER and UNREGISTER in Private Preview, tables can move between catalogs while the underlying data stays in place. For us, an open catalog should give you storage ownership, full capabilities on the tables you own, and a clean way out.

AI Runtime is now generally available, bringing serverless, on-demand GPUs directly into Databricks to power the next generation of AI development. As AI teams scale model training and agent-driven experimentation, they need GPU infrastructure that can keep pace. AI Runtime delivers seconds-level launch latency, reliable multinode training, and Unity Catalog governance on every workload. Hundreds of customers, including Robinhood, PicPay, Flo Health and FactSet already use it to ship deep learning models to production. Accelerate AI innovation, from experimentation to production →

DeepSeek 4.1 Flash is on Databricks the day it ships. On OfficeQA Pro V2, it lands on the Pareto frontier, making it a strong open-source option for enterprise document reasoning. You can point it at the data already in your Lakehouse and use it to build governed AI apps and agents from day one. ↳ Every call routes through Unity Gateway ↳ Access, spend, and performance stay under one view ↳ The same control layer applies across @deepseek_ai and every other model you run

New York is becoming one of Databricks’ biggest AI hubs. 🗽 We’re expanding at 5 Bryant Park to ~120,000 square feet across five floors, including a dedicated Engineering and Research floor. With teams building technologies behind Genie, Unity Gateway and CustomerLake, our momentum in NYC is only growing.

AI company Databricks doubling footprint at 5 Bryant Park