Post-Doctoral Research Visit F - M Developing 3D-Vlms For Embodied Perception And Reasoning H/F INRIA

Post-Doctoral Research Visit F/M Developing 3D-VLMs for Embodied Perception and Reasoning
Le descriptif de l'offre ci-dessous est en Anglais
Type de contrat : CDD

Contrat renouvelable : Oui

Niveau de diplôme exigé : Thèse ou équivalent

Fonction : Post-Doctorant

Contexte et atouts du poste

The work will be conducted in the WILLOW team at Inria Paris research center, supported by the French National Research Agency (ANR) JCJC funding. Renowned for its exceptional work in computer vision and robotics, the WILLOW team has consistently produced high-quality research, resulting in publications in major journals and conferences.

As part of the team, you will have access to a well-established laboratory featuring multiple robotic arms, hands, quadrupeds, bipeds, and mobile manipulators.

Additionally, you can expect frequent visits and talks by esteemed researchers from top research laboratories around the world. Opportunities abound for collaboration with leading researchers both in Europe and globally.

Furthermore, you will join an international and welcoming team environment, where we regularly organize various events ranging from casual after-work gatherings to multi-day lab retreats.

Mission confiée

As populations age and labor shortages grow, autonomous robots are increasingly seen as a practical solution in fields ranging from manufacturing and agriculture to healthcare, customer service, and home assistance. In recent years, robotic hardware has made impressive strides. For example, dexterous end-effectors, high-precision sensors, and affordable mobile platforms are now widely available. Yet, the intelligence driving these robots still lags behind. Most deployed systems still operate on rigid, pre-programmed scripts. While this works well in controlled, predictable settings, performance quickly breaks down in unstructured environments where objects, scenes, and tasks vary widely. The inability to generalize across such variations remains one of the most significant bottlenecks in robotics.

To address this challenge, this project aims to develop advanced 3D vision-language models (3D-VLMs) that endow robots with high-level perception and reasoning skills. These models will need to understand complex 3D scenes, perform spatial and commonsense reasoning, long-horizon planning and precise spatial grounding. These capabilities should generalize across different robot embodiments. For example, when instructed to take the shoes out of the shoe box, the model should first detect that the box is closed, reason about how to open it, and predict the location and contact points of the lid. At any stage, a user could intervene, e.g., correcting place the shoe on the left, not the right.

To this end,the candidatewill investigate three main directions: 1) Learning generalizable 3D representations for robotics; 2) Designing new 3D-VLM architectures for reasoning and high-level planning; 3) Improving computational efficiency to make these models practical for real-world deployment.

Collaborators:

Shizhe Chen, Cordelia Schmid, Stephane Caron

Principales activités

- Read papers
- Propose methods
- Conduct experiments
- Analyze results
- Write papers
- Present work in conferences
- Co-supervise students (optional)

Compétences

The candidate must have an excellent track of records and a PhD Degree. The candidate must have the following qualifications:

- Strong background in computer vision, robotics, or related fields

- Excellent programming skills with deep learning using Python and Pytorch

- Strong proficiency in both written and spoken English

- Ability to work independently as well as collaboratively

- Publications in top-tier vision/robotics conferences and contributions to open-source vision/robotic projects are appreciated

Avantages

- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage