Ramakanth Kavuluru

RT @AI4Pathology: Can an EHR model trained at one hospital read another hospital's records? Excited to introduce our new paper, "Discrete Neural Interlingua for Ontology-Agnostic EHR Modeling," accepted at NeurIPS 2026. See you in Sydney 🦘🏖️! Paper: Code: Model: Video: See👇 The same clinical concept can be an ICD code, a SNOMED term, a local identifier, or free text, and standard EHR tokenizers treat each one as an unrelated symbol. A model trained on MIMIC-III with a closed vocabulary can use only 13% of eICU events. It isn't only a cross-hospital problem. For example, inside one hospital, "acute kidney injury" can be a billing code, a flowsheet identifier, and free text in a note, and the vocabulary shifts again with every coding transition and EHR migration. The redundancy is there before a model ever leaves the building. MEDIATOR is a discrete, ontology-agnostic tokenizer. Any clinical concept written as text maps to one of 100,000 shared tokens, so "acute kidney failure" (ICD-9), "AKI" (a local code), and "acute kidney injury" (free text) become the same token. It is trained on cross-ontology synonyms from UMLS and guided by LLM-enriched descriptions, code co-occurrence, and a medical relation graph. It is a drop-in token interface for standard EHR Transformers: no manual code mapping, and new code systems can be tokenized without retraining. Main results: • In-domain: +4.0 AUPRC points (8% relative) over SOTA, across 20 MIMIC-III tasks. Merging within-dataset redundancy pays off before you transfer anything. • Zero-shot transfer: on eICU with no vocabulary alignment, MEDIATOR uses all 3.58 million events and improves AUPRC over MedTok by 8.6 points (29% relative) across 14 tasks, up to +21.2 on subarachnoid hemorrhage. Restricted to the same inputs as the closed-vocabulary baselines, it is still the best tokenizer. • Transferable attribution: one shared token space means attributions compare directly across hospitals. For cirrhosis mortality, token importance is rank-correlated between MIMIC-III and eICU at Spearman ρ = 0.92. Tokenizing a dataset's entire vocabulary is a one-time step of about 70 seconds on one GPU. Code and weights are public, and the co-occurrence teacher can be rebuilt from public data with near-identical results. Congradulations to Caiwei Tian Tong Ding and everyone else. #NeurIPS2026 #ClinicalInformatics #EHR #MachineLearning

My issue not that they are using ChatGPT, it's fine, they are shooting their shot although my website says I cannot respond to prospective student emails. But their emails lately contain hallucinations attributing to me papers that I did not author. It's hurting their case now.

my question is, if you think I'm dumb enough to not realize your personalized email was tailored by ChatGPT, do you really want me to be your PhD advisor?

RT @leons1ck: My Student Researcher project at @Google is out! 🔥 We introduce On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models, showing that self-generated privileged information can effectively drive LLM self-improvement. 📄 Paper:

RT @_VictorMorand_: 🐭MICE to Meet You: From Re-ranking Only to End-to-End Retrieval ⚡️ Our #EMNLP2026 paper showed that MICE pushes the Cross-Encoder Pareto frontier. We take it a step further: turning theoretical pre-computation gains into an operational, end-to-end retrieval architecture 🧵

RT @chenpan_li: LLMs can solve tasks one at a time, yet fail when the same tasks are combined into one prompt. Why? In our paper, we find that LLMs operate under an implicit thinking budget, with the reasoning effort applied to each task shrinking according to a power law.

RT @thomashikaru: If you've noticed a distinct dialect emerging in frontier AI models and struggled to understand what they're talking about, you're not crazy! We turned to psycholinguistics to investigate what makes the writing style so weird, and why this might be happening 🚨New preprint! 🧵

RT @LoubnaBenAllal1: Introducing Carbon-X: The best open DNA model. We midtrained a 14B model on DNA and text, and it beats every open language model and matches GPT-6.1 Sol at 1,000x lower cost. Ask it about a sequence, what a mutation does, or to generate new DNA.

RT @_lewtun: Today we're releasing Carbon-X, a 14B model that brings DNA and natural language together for the first time 🧬 1. Beats every open language model we tested across DNA benchmarks. Within a point of GPT-6.1 Sol at >1,000× lower inference cost. 2. Mid-trained on 300B tokens of interleaved DNA and text, followed by SFT (without distillation!) and RL. 3. A new hybrid tokenizer encodes DNA using ~3× fewer tokens than BPE. 4. Gives other models a DNA specialist subagent: Kimi K3 gains 11.1 points when it consults Carbon-X. 5. Runs locally on a laptop with llama.cpp. Apache 2.0. Your DNA stays on your hardware. Model: Blog:

RT @jiosephlee: 💡 Training on auxiliary views of Document X > training on Document X itself, even for factual questions drawn directly from X. Presenting our work on LLM knowledge acquisition at #EMNLP2026 this weekend! (1) Under a fixed token budget, conceptual reformulations (textbook explanations, blogs, and Q&A) can teach a document’s facts more effectively than repeating the document, even paraphrased. We call these auxiliary views: beyond mere paraphrasing, they discuss the knowledge in a completely different context and form e.g. a paper blog! The surprising part is that auxiliary views rarely mention the facts directly, but our evaluation uses the original document's phrasing near-verbatim. (2) The benefit emerges and grows with model scale. (3) stronger generator ≠ better learning. We synthetically generate auxiliary views for our experiments, but across GPT-5, GPT-OSS, Gemma4, GLM-5 and more, the learner model's knowledge acquisition shows no correlation to generator model size or its own accuracy on the evaluation. (4) After training on auxiliary views, the model's parameters actually move less but concentrate its changes at the middle and final layers. The takeaway for synthetic data augmentation: build diversity around the knowledge you want to teach, instead of treating diversity as some board corpus-level heuristic. Also, a possible implication for LLM self-improvement. Models can learn by just generating and training on their own auxiliary views, without a stronger teacher. This is particularly relevant for deeply specialized, long-tailed knowledge. 📄 Paper:

RT @NishantBalepur: 🚨 New Paper! 🚨 Multiple-choice benchmarks use accuracy (boring), but education has way cooler scoring schemes! 🤩 👎 Lose points if wrong 🔁 Try until you're right 🃏 Wager points At #EMNLP2026, we show these schemes shift LLM ranks, align with LLM Arena, + unmask LLM quirks

RT @danish037: Super excited to share SnapJudge -- a free, open detector trained for scientific and math-heavy writing, and can handle multiple modalities, including screenshots of paper text, LaTex, and text extracted from PDFs... 🧵

Introducing SnapJudge: a free, open AI detector for scientific writing, including math! Drop in a PDF screenshot, paste text or LaTeX, or just put in an arXiv link. We ran it on arXiv: ~1 in 3 new math papers are flagged as mostly AI, up from ~0 in early 2025. CS is 45%. 🧵

RT @LingYang_PU: Can research agents get better at science by learning from the research they do? Introducing DiscoveryRSI 🧬: a framework that turns hypothesis-driven research into a recursive self-improvement loop. 📄 Paper: 💻 Code: 🔴 Live Arena: 🔹 What is a valuable hypothesis? We define it as a joint object H = (claim, investigation plan, decision criteria, alternative explanations). A hypothesis is valuable if resolving it would change a research decision. It is testable if the plan can produce evidence that supports, weakens, or bounds the claim. 🔹 How it works - Lead, Generation, Evolution, Critique and Progress agents grow hypothesis trees through evolve / repair / crossover / restart - Every version stays linked to its evidence, data, assessments and contribution records - Selected plans compile into executable Harbor tasks 🔹 Three self-improvement mechanisms ① Hypothesis credit optimization: blinded TrueSkill assessments assign bounded credit to the agent steps that actually contributed, and DiscoveryRSI-27B is trained only on those steps (Qwen3.8-27B init, 256K context; 2,397 of 15,792 trajectories kept; 2.12% of tokens supervised) ② Harness replay: diagnose recurring failures such as repeated exploration, failed tool use, lost artifacts and premature closure, then refine the workflow ③ Rediscovery: run the accepted model and harness updates on fresh questions 🔹 Results (DFM-Bench, 4 life-science domains) - Hypothesis quality 4.86 → 5.18 → 5.39 over two rounds - Falsifiability 4.00 → 6.20, alternative explanations 4.44 → 5.98, data resolvability 3.73 → 4.99 - Evolve/repair share 77.5% → 91.7%, restarts 21.7% → 5.0%: the agent learns to keep developing an idea instead of starting over - Caveat: importance and novelty dip slightly. The model learns to design better tests, not yet to choose bolder questions. 🔹 Expert review 27 physicians and bioscience researchers scored 292 anonymized proposals: - 41% of DiscoveryRSI-27B revisions improve quality vs 31% (GPT-5.6 Sol) and 25% (GPT-6 Luna) - Only 12% make things worse vs 32% and 31% 🔹 Does it anticipate real science? Against 16,440 publications filtered by each model's cutoff, 6.3% of DiscoveryRSI-27B's hypotheses connect to later published research (16.9% in therapeutic targets). Comparison: DeepSeek 5.4%, GLM 2.8%, GPT 2.5%, Claude 2.3%. Watch six models do research in real time in DiscoveryRSI Arena, and follow every proposal, critique and compile step 👇 Co-led by Jianyuan Zhong, Chenwei Wu & Zhaochen Yu, with wonderful collaborators across PhAI Labs, CUHK, Stanford, MIT, Harvard Medical School, Oxford, Princeton, PKU and more. Grateful to @james_y_zou @Jiaru_Zou @Charles_Y_Wu @zhenfei_yin_yzf 🙏

Along the same lines, I also don't understand their "model welfare" spiel: They justify this by citing a report by other people, but they also funded that report :-)

I've avoided critiquing since I have a lot of friends at Anthropic, but the anthropomorphization of Claude is dangerous. Roughly, it's a large transformer with a sophisticated training set (pre, mid, post/RL) that is extensively trained in RL "game worlds" mimicking useful real world situations. In some fundamental sense it's not that different from say AlphaGo except for the fact the enormous number of parameters means it can play a lot of different games (e.g., the game of maximizing human response). This is a very neat technology but treating it like some sorcerous creation is dangerous. I don't believe Anthropic folks are cynics. I think a lot of them genuinely believe this is a recipe to create a sentient being. If this is in fact true (which I question heavily), open scientific enquiry will reduce suffering. Sunlight cleans shadows. The secrecy and cloak and dagger is harmful to society.

RT @amuseddaman: 1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky! Very excited to share our new paper: Training Language Models to Fail Gracefully 🧵

RT @NIHDirector_Jay: Researchers tend to have their best, most innovative ideas early in their careers. But for decades, researchers have had to wait longer and longer to test their own ideas. Thrilled by the progress we’ve made so far on this front, including the huge milestone of setting a new record for grants supporting early-stage investigators.

RT @samschapiro: 🧵[1/N] Are large language models more creative than humans at combining ideas? 🤔 Combinatorial creativity involves combining familiar ideas in unfamiliar ways, and is central to scientific discovery and technological innovation. In fact, breakthrough scientific work is often associated with more distant combinations of methods, techniques, and concepts. Here, we introduce Kombine, a new benchmark of combinatorial creativity for LLMs. TLDR: - LLMs are more combinatorially creative than the average human. - However, conceptual blending is the hardest task for LLMs, and even frontier models struggle. - LLMs also tend to homogenize, inventing concepts with the same properties across separate models. - Moreover, humans and LLMs combine in distinct regions of semantic space. - As a result, we anticipate greater conceptual diversity can be unlocked through human-AI collaboration. 🗣️ Thread below🧵⬇️

Young people, don't ignore midterm elections. Policies have long terms consequences for u & ur voice needs to be heard. Ur voting rate is already low during presidential elections & I am scared how awful it would be during midterms. Please vote on 11/3 (early voting is easier)

RT @deliprao: Today we are presenting "Autorubric: A Unifying Framework for Rubric-Based LLM Evaluation on Non-Verifiable Tasks" at #COLM2026 If you are doing evals or reward modeling, you will want to tune in. A short thread to introduce this work 🧶

How awful is this practice. I was a student in theoretical computer science in the 2000s. I know several research students got unpleasant surprises with the uploads on 10/6. The bulk upload of proofs without Lean formalizations was okayed by all the geniuses at this firm 🤦‍♂️

We've updated our GitHub math repo with 6 new Lean formalizations, 19 modifications, and 3 withdrawals. The repo now has ~42% top-line results formalized. We will continue to update the repo with new formalizations and with any errata we notice.

Is it me or do LLMs cite more papers from open access publishers (especially, JMIR, Frontiers, MDPI) compared with potentially more impactful/rigorous studies published in venues that are subscription based? Maybe it's a side effect of full text offering LLMs more confidence.