RoboPapers✓

We've hit 100 episodes! Here's a look back on our journey so far (website with some stats): Some highlights in the thread 🧵:

Using simulation to teach robots would unblock the massive scale of robotics data that seems to be missing right now. The issue is usually sim-to-real transfer; MolmoBOT aims to solve this with sufficient diversity of simulation data, so that no real data is necessary for robot deployment. It has a few parts: MolmoBot-Engine produces diverse pick-and-place demonstrations across many different simulated environments; MolmoBot-Data is a dataset of 1.7 million expert trajectories for item manipulation and pick-and-place tasks; and they train a number of VLA policies which can achieve strong performance on real-world pick-and-place tasks with zero real robot data. @rosemhendrix, @SnehalJauhri, Abhay Deshpande, and Maya Guru join us to give us the details. To learn more, watch RoboPapers episode 113 with @chris_j_paxton and @ruijie_sg!

Touch is crucial for humans when they want to perform precise tasks to a high degree of reliability, and yet robots almost universally use vision — they’re often completely blind to this whole modality. Two works from Tong Jian, Tao Yu, Saleh Nabi, and Matteo Grimaldi show us how important this sense is. In SlipSense, they look at how to detect objects slipping in grippers using a compact multimodal tactile sensor, which generalizes across unseen objects and to different robots. They then talk about how tactile sensing can be used to enable high real-world performance with a challenging cable-tracing task, where a cable must be fed through the hand with a repeated pinch-and-curl motion. To learn more, watch Episode 112 of RoboPapers today, with @micoolcho and @ruijie_sg!

Full episode dropping soon! Geeking out with @c_voelcker, @ArpitBahety, and @zhiyuan_zhou_ on OopsieData, a community effort to collect and learn from real-world robot failures. Co-hosted by @chris_j_paxton and @DJiafei

Full episode dropping soon! Geeking out with @c_voelcker, @ArpitBahety, and @zhiyuan_zhou_ on OopsieData, a community-driven effort to build the first large-scale dataset of real robot manipulation failures Co-hosted by @chris_j_paxton and @DJiafei

Full episode dropping soon! Geeking out with @rosemhendrix, @SnehalJauhri, Abhay Deshpande, and Maya Guru on MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation Co-hosted by @chris_j_paxton and @ruijie_sg

Full episode dropping soon! Geeking out with @rosemhendrix, @SnehalJauhri, Abhay Deshpande, and Maya Guru on MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation Co-hosted by @chris_j_paxton and @ruijie_sg

Full episode dropping soon! Geeking out with team with Tong Jian, Tao Yu, Saleh Nabi, Matteo Grimaldi on SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection & Touch2Trace: Tactile-Driven Imitation Learning for Dexterous Cable Tracing Co-hosted by @micoolcho @ruijie_sg

Full episode dropping soon! Geeking out with team with Tong Jian, Tao Yu, Saleh Nabi, Matteo Grimaldi on SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection & Touch2Trace: Tactile-Driven Imitation Learning for Dexterous Cable Tracing Co-hosted by @micoolcho @ruijie_sg

Reinforcement learning is often seen as key to achieving scalable, highly reliable real-world robot performance, yet it requires good reward functions that can supervise the wide range of tasks that need to be performed. The obvious thing to try would be a large vision-language model (VLM), but these models often fail under common real-world conditions like partial observability and occlusion. As a solution @Philip_MIT and @thomas_weng propose SOLE-R1, a video-language reasoning model trained to serve as a reward signal for online RL. Given real video, SOLE-R1 will reason over input video frames, generates reasoning traces, and predicts a reward signal. This allows online RL from random initialization on new manipulation tasks. To learn more, watch Episode 111 of RoboPapers, with @micoolcho and @DJiafei!

Full episode dropping soon! Geeking out with @Philip_MIT @thomas_weng on SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL Co-hosted by @micoolcho @DJiafei

Full episode dropping soon! Geeking out with @Philip_MIT @thomas_weng on SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL Co-hosted by @micoolcho @DJiafei

In many fields, automated research has become increasingly plausible with the improving capabilities of frontier language models, and the field robotics may be no exception. In ENPIRE, @_wenlixiao, @jiaxie_jason, and @TongheZhang01 build a harness framework that captures physical feedback and allows for automated robotics research, minimizing human effort while training policies that achieve a 99% success rate on challenging, dexterous tasks like organizing a pin box or fastening a zip tie. Learn more in Episode 110 of RoboPapers, with @micoolcho, @chris_j_paxton and @ruijie_sg!

Full episode dropping soon! Geeking out with @_wenlixiao @jiaxie_jason @jiaxie_jason on ENPIRE: Agentic Robot Policy Self-Improvement in the Real World Co-hosted by @micoolcho @chris_j_paxton @ruijie_sg

Full episode dropping soon! Geeking out with @_wenlixiao @jiaxie_jason @jiaxie_jason on ENPIRE: Agentic Robot Policy Self-Improvement in the Real World Co-hosted by @micoolcho @chris_j_paxton @ruijie_sg

What happens if you connect a hundred AI-powered robots to the internet and just let people give them any instructions they want? @Enigma_AI founder @j0nathanj joins us to talk about his experience deploying a fleet of robots on the internet, keeping them operational, and watching people truly test the generality and robustness of frontier robotics models. Watch Episode 109 of RoboPapers with @chris_j_paxton and @ruijie_sg today!

Full episode dropping soon! Geeking out with @j0nathanj on @Enigma_AI Co-hosted by @chris_j_paxton @ruijie_sg

Pretrained vision-language-action models for robots often fail when a task is performed in a slightly different environment. With Vision-Language Steering, you can get better generalization performance out of a frozen robot policy, by steering the sampling process of your pretrained diffusion or flow-matching policy to handle out-of-distribution changes like obstacle position or changing visual appearance. @liu_shuo42927 and @ishneet0710join us to talk about how to better generalize robot policies with no new training data. Watch Episode 108 of RoboPapers today, with @ruijie_sg and @DJiafei!

Full episode dropping soon! Geeking out with @j0nathanj on @Enigma_AI Co-hosted by @chris_j_paxton @ruijie_sg

Full episode dropping soon! Geeking out with @liu_shuo42927 @ishneet0710 on VLS: Steering Pretrained Robot Policies via Vision–Language Models Co-hosted by @ruijie_sg @DJiafei

Full episode dropping soon! Geeking out with @liu_shuo42927 @ishneet0710 on VLS: Steering Pretrained Robot Policies via Vision–Language Models Co-hosted by @ruijie_sg @DJiafei