IEEE 2024 ICRA Workshop VLMNM Submissions

Leveraging YOLO-World and GPT-4V LMMs for Zero-Shot Person Detection and Action Recognition in Drone Imagery
Christian Limberg, Artur Gonçalves, Bastien Rigault, Helmut Prendinger
- Published: 05 Apr 2024, Last Modified: 28 Apr 2024
- VLMNM 2024
- Readers: Everyone
Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation
Abdelrhman Werby, Chenguang Huang, Martin Büchner, Abhinav Valada, Wolfram Burgard
- Published: 05 Apr 2024, Last Modified: 16 Apr 2024
- VLMNM 2024
- Readers: Everyone
What Matters in Employing Vision Language Models for Tokenizing Actions in Robot Control?
Nicolai Dorka, Chenguang Huang, Tim Welschehold, Wolfram Burgard
- Published: 05 Apr 2024, Last Modified: 29 Apr 2024
- VLMNM 2024
- Readers: Everyone
OVAL-Prompt: Open-Vocabulary Affordance Localization for Robot Manipulation through LLM Affordance-Grounding
Edmond Tong, Anthony Opipari, Stanley Robert Lewis, Zhen Zeng, Odest Chadwicke Jenkins
- Published: 05 Apr 2024, Last Modified: 22 Apr 2024
- VLMNM 2024
- Readers: Everyone
Explore until Confident: Efficient Exploration for Embodied Question Answering
Allen Z. Ren, Jaden Clark, Anushri Dixit, Masha Itkina, Anirudha Majumdar, Dorsa Sadigh
- Published: 05 Apr 2024, Last Modified: 26 Apr 2024
- VLMNM 2024
- Readers: Everyone
Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals
Moritz Reuss, Ömer Erdinç Yağmurlu, Fabian Wenzel, Rudolf Lioutikov
- Published: 05 Apr 2024, Last Modified: 14 Apr 2024
- VLMNM 2024
- Readers: Everyone
DeliGrasp: Inferring Object Mass, Friction, and Compliance with LLMs for Adaptive and Minimally Deforming Grasp Policies
William Xie, Jensen Lavering, Nikolaus Correll
- Published: 05 Apr 2024, Last Modified: 29 Apr 2024
- VLMNM 2024
- Readers: Everyone
AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
Michael Ahn, Debidatta Dwibedi, Chelsea Finn, Montserrat Gonzalez Arenas, Keerthana Gopalakrishnan, Karol Hausman, brian ichter, Alex Irpan, Nikhil J Joshi, Ryan Julian, Sean Kirmani, Isabel Leal, Tsang-Wei Edward Lee, Sergey Levine, Yao Lu, sharath maddineni, Kanishka Rao, Dorsa Sadigh, Pannag R Sanketi, Pierre Sermanet et al. (7 additional authors not shown)
- Published: 05 Apr 2024, Last Modified: 24 Apr 2024
- VLMNM 2024
- Readers: Everyone
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Soroush Nasiriany, Fei Xia, Wenhao Yu, Ted Xiao, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, Quan Vuong, Tingnan Zhang, Tsang-Wei Edward Lee, Kuang-Huei Lee, Peng Xu, Sean Kirmani, Yuke Zhu, Andy Zeng, Karol Hausman, Nicolas Heess et al. (3 additional authors not shown)
- Published: 05 Apr 2024, Last Modified: 30 Apr 2024
- VLMNM 2024
- Readers: Everyone
MOKA: Open-Vocabulary Robotic Manipulation through Mark-Based Visual Prompting
Fangchen Liu, Kuan Fang, Pieter Abbeel, Sergey Levine
- Published: 05 Apr 2024, Last Modified: 14 Apr 2024
- VLMNM 2024
- Readers: Everyone
Talk Through It: End User Directed Manipulation Learning
Carl Winge, Karthik Desingh, Adam Imdieke, Bahaa Aldeeb, Dongyeop Kang
- Published: 05 Apr 2024, Last Modified: 14 Apr 2024
- VLMNM 2024
- Readers: Everyone
Open Scene Graphs for Open-World Object-Goal Navigation
Joel Loo, Zhanxin Wu, David Hsu
- Published: 05 Apr 2024, Last Modified: 30 Apr 2024
- VLMNM 2024
- Readers: Everyone
Wearable Robot Control Method Based on Vision-Language Models
Hyeon-Seok Seong, Seif Farag, Sang-Wook Lee, Igor Gaponov, Jee-Hwan Ryu
- Published: 05 Apr 2024, Last Modified: 14 Apr 2024
- VLMNM 2024
- Readers: Everyone
Keypoint Action Tokens Enable In-Context Imitation Learning in Robotics
Norman Di Palo, Edward Johns
- Published: 05 Apr 2024, Last Modified: 28 Apr 2024
- VLMNM 2024
- Readers: Everyone
Open-loop VLM Robot Planning: An Investigation of Fine-tuning and Prompt Engineering Strategies
Shogo Akiyama, Rousslan Fernand Julien Dossa, Kai Arulkumaran, Shivakanth Sujit, Edward Johns
- Published: 05 Apr 2024, Last Modified: 19 Apr 2024
- VLMNM 2024
- Readers: Everyone
Visual Preference Inference: An Image Sequence-Based Preference Reasoning in Tabletop Object Manipulation
Joonhyung Lee, Sangbeom Park, Yongin Kwon, Jemin Lee, Minwook Ahn, Sungjoon Choi
- Published: 05 Apr 2024, Last Modified: 22 Apr 2024
- VLMNM 2024
- Readers: Everyone
Language Models as Zero-Shot Trajectory Generators
Teyun Kwon, Norman Di Palo, Edward Johns
- Published: 05 Apr 2024, Last Modified: 28 Apr 2024
- VLMNM 2024 Spotlight
- Readers: Everyone
Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulation
Daniel Honerkamp, Martin Büchner, Fabien Despinoy, Tim Welschehold, Abhinav Valada
- Published: 05 Apr 2024, Last Modified: 16 Apr 2024
- VLMNM 2024 Spotlight
- Readers: Everyone
OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics
Peiqi Liu, Yaswanth Orru, Jay Vakil, Chris Paxton, Nur Muhammad Mahi Shafiullah, Lerrel Pinto
- Published: 05 Apr 2024, Last Modified: 14 Apr 2024
- VLMNM 2024
- Readers: Everyone
Deploying and Evaluating LLMs to Program Service Mobile Robots
Zichao Hu, Francesca Lucchetti, Claire S Schlesinger, Yash Saxena, Anders Freeman, Sadanand Modak, Arjun Guha, Joydeep Biswas
- Published: 05 Apr 2024, Last Modified: 27 Apr 2024
- VLMNM 2024 Spotlight
- Readers: Everyone
Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control
Gunshi Gupta, Karmesh Yadav, Yarin Gal, Dhruv Batra, Zsolt Kira, Cong Lu, Tim G. J. Rudner
- Published: 05 Apr 2024, Last Modified: 01 May 2024
- VLMNM 2024
- Readers: Everyone
Synapse: Learning Preferential Concepts from Visual Demonstrations
Sadanand Modak, Noah Tobias Patton, Isil Dillig, Joydeep Biswas
- Published: 05 Apr 2024, Last Modified: 26 Apr 2024
- VLMNM 2024
- Readers: Everyone
Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-Language Models
Ivan Kapelyukh, Yifei Ren, Ignacio Alzugaray, Edward Johns
- Published: 05 Apr 2024, Last Modified: 26 Apr 2024
- VLMNM 2024
- Readers: Everyone
LAN-grasp: An Effective Approach to Semantic Object Grasping Using Large Language Models
Reihaneh Mirjalili, Michael Krawez, Simone Silenzi, Yannik Blei, Wolfram Burgard
- Published: 05 Apr 2024, Last Modified: 17 Apr 2024
- VLMNM 2024
- Readers: Everyone
Pointing Gesture Understanding via Visual Prompting and Visual Question Answering for Interactive Robot Navigation
Kosei Tanada, Shigemichi Matsuzaki, Kazuhito Tanaka, Shintaro Nakaoka, Yuki Kondo, Yuto Mori
- Published: 05 Apr 2024, Last Modified: 30 Apr 2024
- VLMNM 2024
- Readers: Everyone