English
Philip Schroeder
Report

Philip Schroeder✓

@philip_mit

MIT PhD student in Computer Science. @MIT @MIT_CSAIL @MITEECS robots, RL, embodied reasoning, video-language models

2.2K Following2.4K Followers133 Posts
  1. Philip Schroeder✓@philip_mit
    Excited to share SOLE-R1 is accepted to NeurIPS 2026! General-purpose reward models are fundamental to unlocking real-world RL. We show SOLE-R1's zero-shot reward prediction can serve as the SOLE signal for learning over 20 unseen manipulation tasks through online RL (i.e., randomly initialized policies, 0% starting success rate). This work provides a strong proof of concept: RL with zero-shot dense reward prediction can do more than just increase success/speed for existing tasks; it can enable robots to learn entirely new tasks from scratch, without any demonstrations or ground-truth rewards, starting from zero competence. Very proud of this project! Webpage: Thank you to my coauthors @thomas_weng, @_ericrosen, @BizaOndrej. #NeurIPS2026
    Download
    1161147.5K
  2. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @_ericrosen: The Boston Dynamics Spot legs have always intrigued me, specifically how the knees are actuated! There is a great patent on this, “US20180172121A1: Screw Actuator for a Legged Robot”. The abstract captures the design quite succinctly (included below). I got Claude to generate a rough diagram to see how the linear screw actuator can be used to control the lower leg link! I’m also including an image from the patent to compare to the real thing! “An example robot includes: a leg having an upper leg member and a lower leg member coupled to the upper leg member at a knee joint; a screw actuator disposed within the upper leg member, where the screw actuator has a screw shaft and a nut mounted coaxial to the screw shaft such that the screw shaft is rotatable within the nut; a motor mounted at an upper portion of the upper leg member and coupled to the screw shaft; a carrier coupled and mounted coaxial to the nut such that the nut is disposed at a proximal end of the carrier; and a linkage coupled to the carrier, where the linkage is coupled to the lower leg member at the knee joint.”
    0402
  3. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @MengxueBi: Great materials for learning about AI fundamentals that balance technical facts and digestability: 1. How LLM works (in 20min) This whole podcast (Fragmented -- AI developers podcast) is a goldmine for whoever wants to learn about essential concepts for AI models/agents without touching the hardcore science behind them. 1. How always-on agents work 2. DeepSeek R1 Explained to your grandma 4.. Direct Preference Optimization (DPO) paper explained
    0101
  4. Philip Schroeder✓@philip_mit
    fine-tuning our reward models on @dm_robots task. Still early, but cool to see our model show near-perfect progress prediction on this unseen trajectory with interweaving errors and recoveries. the task has a nice combination of factors that make it a good challenge for the reward model (and policy): - bimanual + multi-view: requires coordinating both arms and reasoning across three camera views - on-robot cameras only: no external cameras providing a global view of the scene - precision: demands precise insertion of the actuator into the hole where the visual difference between near-miss and success is very subtle - start state and goal state change every episode: different actuator to pick up, different hole to insert it into the task is right on the edge of hard, but not too hard to get to >99% success rate exciting to see @DominiqueCAPaul and @aurel_arnold make so much progress with this task in a very short period of time
    Download
    38503.5K
  5. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @thomas_weng: First time being on a podcast, and it's RoboPapers! 🤩 I had a great time discussing SOLE-R1 ( with @micoolcho, @DJiafei, and @philip_mit
    RoboPapers @RoboPapers
    Reinforcement learning is often seen as key to achieving scalable, highly reliable real-world robot performance, yet it requires good reward functions that can supervise the wide range of tasks that need to be performed. The obvious thing to try would be a large vision-language model (VLM), but these models often fail under common real-world conditions like partial observability and occlusion. As a solution @Philip_MIT and @thomas_weng propose SOLE-R1, a video-language reasoning model trained to serve as a reward signal for online RL. Given real video, SOLE-R1 will reason over input video frames, generates reasoning traces, and predicts a reward signal. This allows online RL from random initialization on new manipulation tasks. To learn more, watch Episode 111 of RoboPapers, with @micoolcho and @DJiafei!
    0201
  6. RoboPapers✓@RoboPapers
    Reinforcement learning is often seen as key to achieving scalable, highly reliable real-world robot performance, yet it requires good reward functions that can supervise the wide range of tasks that need to be performed. The obvious thing to try would be a large vision-language model (VLM), but these models often fail under common real-world conditions like partial observability and occlusion. As a solution @Philip_MIT and @thomas_weng propose SOLE-R1, a video-language reasoning model trained to serve as a reward signal for online RL. Given real video, SOLE-R1 will reason over input video frames, generates reasoning traces, and predicts a reward signal. This allows online RL from random initialization on new manipulation tasks. To learn more, watch Episode 111 of RoboPapers, with @micoolcho and @DJiafei!
    Download
    13495.1K
  7. Philip Schroeder✓@philip_mit
    the @RoboPapers episode on SOLE-R1 is out now! we break down our recipe for training a reward model that stays robust to reward hacking as the policy actively optimizes against its zero-shot reward prediction during online RL. check it out if you're interested in real-world reinforcement learning or general-purpose reward reasoning models!
    RoboPapers @RoboPapers
    Reinforcement learning is often seen as key to achieving scalable, highly reliable real-world robot performance, yet it requires good reward functions that can supervise the wide range of tasks that need to be performed. The obvious thing to try would be a large vision-language model (VLM), but these models often fail under common real-world conditions like partial observability and occlusion. As a solution @Philip_MIT and @thomas_weng propose SOLE-R1, a video-language reasoning model trained to serve as a reward signal for online RL. Given real video, SOLE-R1 will reason over input video frames, generates reasoning traces, and predicts a reward signal. This allows online RL from random initialization on new manipulation tasks. To learn more, watch Episode 111 of RoboPapers, with @micoolcho and @DJiafei!
    21241.7K
  11. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @_ericrosen: Designing reward functions for real-world RL is a serious challenge in learning for manipulation. Zero/few-shot reward models that “just work” is going to be a huge unlock for the entire field to go beyond behavior cloning!
    Philip Schroeder @philip_mit
    Excited to share SOLE-R1 is accepted to NeurIPS 2026! General-purpose reward models are fundamental to unlocking real-world RL. We show SOLE-R1's zero-shot reward prediction can serve as the SOLE signal for learning over 20 unseen manipulation tasks through online RL (i.e., randomly initialized policies, 0% starting success rate). This work provides a strong proof of concept: RL with zero-shot dense reward prediction can do more than just increase success/speed for existing tasks; it can enable robots to learn entirely new tasks from scratch, without any demonstrations or ground-truth rewards, starting from zero competence. Very proud of this project! Webpage: Thank you to my coauthors @thomas_weng, @_ericrosen, @BizaOndrej. #NeurIPS2026
    0201
  12. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @thomas_weng: Currently looking at new research directions and SOLE-R1 keeps coming up!
    Philip Schroeder @philip_mit
    Excited to share SOLE-R1 is accepted to NeurIPS 2026! General-purpose reward models are fundamental to unlocking real-world RL. We show SOLE-R1's zero-shot reward prediction can serve as the SOLE signal for learning over 20 unseen manipulation tasks through online RL (i.e., randomly initialized policies, 0% starting success rate). This work provides a strong proof of concept: RL with zero-shot dense reward prediction can do more than just increase success/speed for existing tasks; it can enable robots to learn entirely new tasks from scratch, without any demonstrations or ground-truth rewards, starting from zero competence. Very proud of this project! Webpage: Thank you to my coauthors @thomas_weng, @_ericrosen, @BizaOndrej. #NeurIPS2026
    0101
  13. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @DominiqueCAPaul: Working on first real-world RL token experiments today. > I’m only activating it for the critical part (insertion) as in paper > first rollouts with RLT work decent > After ~10 rollouts with updates the policy gets very jerky - maybe because I’m injecting noise for exploration or the residual policy isn’t regularised strongly enough. Noise injection should show up as jerkiness is first rollouts as well though? In the video you can see the jerkiness. Also, once I turn the RLT network off and unedited VLA takes over everything runs smooth and actuator is inserted cleanly.
    0601
  14. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @DominiqueCAPaul: Two months of ablating π0.5 finetunes on a real manufacturing task. The policy now succeeds 98% of the time. We're publishing all results, all the data, and every run, including the ones that went nowhere. What we learned: 1/ 5x more data took us from 63% to 76%. Pure scaling was the weakest lever we found. 2/ Diversity got more out of the same hours. 4h spread across five scenes beat 4h in the eval scene by 30pp. 3/ Quality did even better. One clean, slower hour on top of the 21h: 76% → 90%. That single hour added more than the 17 hours before it. 4/ So we tried quality on its own: 1h of clean data plus 240 rollouts with human interventions. 1.7h in total, 28% → 88%. That's 12 points above the 21h model with a twelfth of the data. 5/ Inference settings add more on top, with no retraining. They took the 21h model from 76% to 93%, and the 21h+1h model from 90% to 98%. Just as useful: what didn't matter. Learning rate, batch size, relative vs. absolute joints, image augmentation. All less important than we expected. Full write-up linked below.
    06301
  15. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @thomas_weng: Blog post out for our RL post-training method, Q2RL!
    RAI Institute @rai_inst
    Looking to improve your deployed robot with reinforcement learning in the real world? Presented at RSS 2026, Q-Estimation and Q-Gating from Behavior Cloning for Reinforcement Learning (Q2RL) improves robot policies in a few hours of real world interaction, avoiding catastrophic forgetting when fine-tuning a Behavior Cloning (BC) policy via Reinforcement Learning (RL) post-training. Q2RL estimates a Q-function for the BC policy, and uses it to guide online RL. Learn more about the method here:
    0401
  16. RAI Institute✓@rai_inst
    Looking to improve your deployed robot with reinforcement learning in the real world? Presented at RSS 2026, Q-Estimation and Q-Gating from Behavior Cloning for Reinforcement Learning (Q2RL) improves robot policies in a few hours of real world interaction, avoiding catastrophic forgetting when fine-tuning a Behavior Cloning (BC) policy via Reinforcement Learning (RL) post-training. Q2RL estimates a Q-function for the BC policy, and uses it to guide online RL. Learn more about the method here:
    17494.5K
  17. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @thomas_weng: Exciting to be organizing a #CoRL2026 workshop on real-world reinforcement learning! Post-training in the real world will be critical for deploying useful robots, but it's currently being slept on. The R2RL workshop brings together researchers working in this area to network and share ideas. If you're pioneering this research, consider joining us at the workshop! Workshop submissions accepted until Sep. 30.
    Jesse Zhang @Jesse_Y_Zhang
    We’re trying something new at the R2RL Workshop at CoRL 2026. 🔗 No senior-speakers, no passive audience. Key Q: What are the real bottlenecks in Sample-efficient, Real-World RL? 🧑‍🔬 No senior speakers. Students & scientists who actually deploy RL on robots. 🗳️ Crowdsourced problems. Audience proposes challenges, we all vote on what to focus on. 🧩Breakout sessions. Audience helps to propose and discuss solutions for top-voted problems. 🎤 Panel. The breakout discussions determine what we debate. 📝 Whitepaper: a community-constructed whitepaper summarizing the most urgent challenges for Real-World RL in the next year. Submit your papers! Attend to help us figure out what we should actually be solving next! Submission deadline: September 30
    0302
  18. Jesse Zhang✓@Jesse_Y_Zhang
    We’re trying something new at the R2RL Workshop at CoRL 2026. 🔗 No senior-speakers, no passive audience. Key Q: What are the real bottlenecks in Sample-efficient, Real-World RL? 🧑‍🔬 No senior speakers. Students & scientists who actually deploy RL on robots. 🗳️ Crowdsourced problems. Audience proposes challenges, we all vote on what to focus on. 🧩Breakout sessions. Audience helps to propose and discuss solutions for top-voted problems. 🎤 Panel. The breakout discussions determine what we debate. 📝 Whitepaper: a community-constructed whitepaper summarizing the most urgent challenges for Real-World RL in the next year. Submit your papers! Attend to help us figure out what we should actually be solving next! Submission deadline: September 30
    Download
    32417844.9K
  19. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @DozenDucc: 1. Robot data isn't the only way models can learn to control robots. Coding, math, computer-use all contain information about the physical world. I bet that generalist LLMs will work better than robot-specific models for this reason 2. The output format (eg. joint angles, code-as-policy, tool calls as in Gemini ER) is up to debate. But why not just let an LLM choose?
    Jay Chooi @chooi_jeq
    GPT-6 Astra scored 95% on a robot control task, up from Fable 5.1's 40%, with 6.2x fewer output tokens at 2.3x lower cost. 🧵
    0601
  20. Jay Chooi✓@chooi_jeq
    GPT-6 Astra scored 95% on a robot control task, up from Fable 5.1's 40%, with 6.2x fewer output tokens at 2.3x lower cost. 🧵
    Download
    1656335.7K2M
  23. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @_ericrosen: One thing I really like about the @pollenrobotics Microduck design are the legs! I’ve labeled my guesses about where the joint axes are on two images, with red/blue/green to group axis alignments. For each leg, starting from top: - The hip is 3 DoF: yaw (red), then roll (I.e., abduction, green), then pitch (flexion, blue). - The knee and ankle are each 1DoF (flexion, blue)). It’s fun to compare the 🦆design to many humanoid designs that instead have flexion higher up the kinematic chain!
    01301
  25. Philip Schroeder✓@philip_mit
    Philip Schroeder
    RT @unnatjain2010: "The generalization you want in your VLA has to be in the expert demos!" folks say. "Didn't we train VLM backbones on trillions of tokens already for it?", we ask. We set out to reclaim some of this (free) OOD generalization in VLAs. Just need to think beyond BC! 🥲 Two edits on top of BC. Same demos. No extra data.
    027019.6K
  26. Show more posts