Juan Carlos Niebles @jcnieblesNew blog post: "Are your Visual Programs Right for the Wrong Reasons?" 🤔 Dive into the motivation behind our upcoming @CVPR #CVPR2025 paper! 📰 Blog: ➡️ Project: 📄 Paper: w/ @artemispng & @zhou_honglu
Salesforce AI Research @SFResearch🎉Just Announced: "ViUniT: Visual Unit Tests for More Robust Visual Programming" has been accepted at #CVPR2025! Paper Link: Project Page: Researcher’s walk-through 👇 In collaboration with @UPenn, we introduce ViUniT, a framework that enhances the reliability of visual programs by automatically generating unit tests by leveraging #LLMs and #DiffusionModels. Our approach: 📊 Boosts model performance by 11.4% and outperforms gpt-4o-mini by 7.7%. 🔄 Reduces “right-for-wrong-reasons” errors by 40%. 💡 Introduces innovative applications like best program selection, answer refusal, and unsupervised reward design. Dive into the 🧵👇 (1/5) Method Overview ViUniT leverages the power of language models and diffusion models to create robust visual unit tests: 📝 Unit Test Generation: Language models generate image descriptions and expected answers for visual queries. 🎨 Image Creation: Diffusion models then synthesize corresponding images, ensuring logical coverage and diversity. ✅ Logical Verification: Visual programs are tested for logical correctness against these generated unit tests, catching errors beyond output accuracy.
Salesforce AI Research @SFResearch🎉Just Announced: "ViUniT: Visual Unit Tests for More Robust Visual Programming" has been accepted at #CVPR2025! Paper Link: Project Page: Researcher’s walk-through 👇 In collaboration with @UPenn, we introduce ViUniT, a framework that enhances the reliability of visual programs by automatically generating unit tests by leveraging #LLMs and #DiffusionModels. Our approach: 📊 Boosts model performance by 11.4% and outperforms gpt-4o-mini by 7.7%. 🔄 Reduces “right-for-wrong-reasons” errors by 40%. 💡 Introduces innovative applications like best program selection, answer refusal, and unsupervised reward design. Dive into the 🧵👇 (1/5) Method Overview ViUniT leverages the power of language models and diffusion models to create robust visual unit tests: 📝 Unit Test Generation: Language models generate image descriptions and expected answers for visual queries. 🎨 Image Creation: Diffusion models then synthesize corresponding images, ensuring logical coverage and diversity. ✅ Logical Verification: Visual programs are tested for logical correctness against these generated unit tests, catching errors beyond output accuracy.
