Daniel Yamins✓

Streams of thought for natural reasoning in the physical world. *This is PSI-0.6 release* Congrats @khai_loong_aw @KlemenKotar at the PSI team!

Chain-of-thought transformed AI reasoning, but it is limited to words. What's the physical-world analogue? Introducing Ψ-0.6, one model that reasons in *streams of thought*: words, depth, motion, camera pose, and imagined scenes. And it learns when to use which! 🧵

RT @kutsch_d: 1/ Animals explore without rewards, tasks, or instructions. What computational objective drives that autonomy? We trained virtual flies with intrinsic-motivation algorithms to see which best reproduce real fly behavior and brain activity. Work with @rdkeller and @aran_nayebi !🧵

RT @mcxfrank: How do children learn the meanings of their first words? One idea is cross-situational learning: tracking which words go with which things across many situations. In a new preprint, we test this idea at scale on video from children's own perspective 🧵

RT @martin_schrimpf: Super excited to share that I'm co-founding Read/Write Labs for #NeuroAI applications to human brain health and human-aligned AI. We could not ask for a more wonderful team:

RT @PrecisionNeuro_: Introducing the Precision Brain Atlas: high-resolution neural recordings from 70 patients across 12 academic medical centers. A growing resource for studying human neural function- and a step toward a Whole-Brain Interface.

People are prophecying the death of math but actually in the last few months is the first time that I have even started thinking about hiring a math PhD as a postdoc, because the prospects of doing advanced math in my area are so much brighter now.

RT @JiahengHu1: We’ve all seen the cool “Astra for robotics” demos. But behind the scenes? Slow rollouts, hand-engineered APIs, and/or simulated behaviors that struggle on real robots. How do we make coding agent actually work in the physical world? 🦾 Introducing SimEX: Simulation-Integrated Robotics AutoResearch 🧵

RT @apurvaratan: Another update from our topography work from (high-schooler!) Navneet Prakash. Navneet trained path-integrating RNNs with TopoLoss and found that they recover a much more brain-like center-surround RF connectivity structure than baseline models (w/o any post-hoc sorting)!

RT @martin_schrimpf: ...that was supposed to read "model-brain alignment is now strong enough" 🤦 Posting early morning on the way to the airport took a toll

RT @martin_schrimpf: Questions in Systems Neuroscience are indeed changing, but to me the new ones are even more exciting! Some years ago, @JamesJDiCarlo showed an old clip of Hans-Lukas Teuber (the founder of @mitbrainandcog: saying that science goes through stages: explain, predict, control. #NeuroAI first moved us to predict, and I think it's super exciting that we're firmly moving into control: - prediction is pretty well established by now (e.g. although models are not perfect and can fail on simple stimuli (what is missing?) - the field is actually making progress on a theory for why this works (@dyamins & @aran_nayebi @tyrell_turing - model-synthesized stimuli can neuromodulate visual cortex and the human language network (@PouyaBashivan @KohitijKar Walker @sinzlab @GretaTuckute and more) - even for invasive techniques, topographic models predict and steer the behavioral effects of microstimulation; optogenetics and muscimol ( - and finally model-brain alignment is not strong enough that we can model atypical processing in dyslexia, autism, aphasia, and anhedonia (@melikahnd_1 @melikahnd_1 How processing breaks down, and how we can push it back toward healthy, are very much algorithmic questions about the brain with a clear real-world impact. And now we can actually test them, how cool is that!

I had a feeling recently that seems to be corroborated by chats with colleagues, so I decided to share it here. A lot of systems neuroscientists driven by algorithmic-level questions are going through what looks and feels like a research depression. 🧵 (1/n)

Yeah seriously @SakanaAILabs ImageNet or it didn’t happen… my guess is the true state of the art in bioplausible learning surpassed this years ago …

Hmm, these results are only on MNIST, which even trivial methods like Feedback Alignment work well on. Of course, everything works on MNIST! 😉 Which is why we focused on bioplausible learning that scaled w/ backprop on ImageNet 6 years ago...

RT @aran_nayebi: CMU is hiring an Assistant Professor in NeuroAI! Come join our awesome NeuroAI community 🧠🤖:

APPLY TO THIS:

RT @phillip_isola: With all the Astra demos, there might be a feeling in robotics of a bitter lesson taking hold? I think this moment is rather the opposite. Astra works *because* it leverages traditional human-engineered methods as its tools. (...for now)

RT @martin_schrimpf: Hiring postdocs @EPFL: please reach out and/or apply to the AI Center fellowship My #NeuroAI group is excited about testing and building models of brain and behavior, from sensory through language to cognition, and from healthy to dysfunction (treatment)!

RT @KriegeskorteLab: To learn how brains compute, we need to experimentally adjudicate among competing computational hypotheses. How do we do this in the age of complex neural network models? New review paper: “Making models disagree to learn how brains compute” with @TalGolanNeuro, @SchuettHeiko

My latest substack post describes detailed empirical results for RSA vs ridge regression across a wide range of visual system tests. I think we're genuinely finally building a better sense of how all these things are related. Substack: We're testing the consequences of RSA contravariance theory: TL;DR: - RSA and ridge regression are actually pretty well correlated, especially in high-quality fMRI data. - The theoretically predicted Ridge-RSA formula works decently! - BUT: strong behavioral tests expose issues with raw RSA. Ridge model-brain similarity is uniformly more predictive of behavioral match across MANY metrics. - And the same is true for low-level visual phenomena: Ridge-based model-brain similarity better predicts 20+ V1 physiology metrics. enjoy!

The Contravariance theory for RSA (and CKA) is here. Substack: Gory mathematical details: TL;DR: - Theiss @ScienceLukas @ermgrant @SaxeLab show RSA decomposes into a "task relevant core geometry" part and a "task irrelevant" nuisance component. The nuisances can dominate the RSA badly. - The nuisances can be removed by projecting onto the privileged axes. Once you do that, full contravariance applies to the task-relevant core geometry RSA. - In the hard-task condition, the three metrics (privileged axes, linear regression, and task-relevant RSA) all line up.

Yams

RT @s_y_chung: Come work with us! Kempner fellowship is a great opportunity to do exciting research on neuroAI, ML, and theory.

The application is now open for our #KempnerInstitute Research #Fellowship! Postdocs studying the foundations of #intelligence or applications of #AI are encouraged to apply. Learn more and apply by Oct. 1: #LLMs #NeuroAI #ML @ShamKakade6 @blsabatini