Phd Position F - M Phd Student For Weakly-Supervised Video Anomaly Detection H/F INRIA

PhD Position F/M PhD student for Weakly-supervised video anomaly detection
Le descriptif de l'offre ci-dessous est en Anglais
Type de contrat : CDD

Contrat renouvelable : Oui

Niveau de diplôme exigé : Bac +5 ou équivalent

Fonction : Doctorant

A propos du centre ou de la direction fonctionnelle

The Inria centre at Université Côte d'Azur includes 42 research teams and 9 support services. The centre's staff (about 500 people) is made up of scientists of dierent nationalities, engineers, technicians and administrative staff. The teams are mainly located on the university campuses of Sophia Antipolis and Nice as well as Montpellier, in close collaboration with research and higher education laboratories and establishments (Université Côte d'Azur, CNRS, INRAE, INSERM ...), but also with the regiona economic players.

With a presence in the fields of computational neuroscience and biology, data science and modeling, software engineering and certification, as well as collaborative robotics, the Inria Centre at Université Côte d'Azur is a major player in terms of scientific excellence through its results and collaborations at both European and international levels.

Contexte et atouts du poste

Inria, the French National Institute for Computer Science and Applied Mathematics, promotes scientific excellence for technology transfer and society. Graduates from the world's top universities, Inria's 2,700 employees rise to the challenges of digital sciences. With its open, agile model, Inria can explore original approaches with its partners in industry and academia and provide an efficient response to the multidisciplinary and application challenges of digital transformation. Inria is the source of many innovations that add value and create jobs.

Team

The STARS research team combines advanced theory with cutting-edge practice, focusing on cognitive vision systems.

Team website:

Scientific context

STARS group works on automatic video monitoring and human behavior understanding for health applications. The Deep Learning platform developed in STARS, detects mobile objects, tracks their trajectory, and recognizes related behaviors predefined by experts. This platform contains several techniques for detecting people and for recognizing human postures/gestures using conventional cameras. However, there are scientific challenges in people tracking when dealing with real-world scenes: cluttered scenes, handling wrong and incomplete person segmentation, handling static and dynamic occlusions, low contrasted objects, moving contextual objects (e.g., chairs), similar appearance of clothes among different people ...

This Project aims to detect critical situations in the CCTV video stream. Weakly-supervised video anomaly detection (wVAD) has recently gained popularity thanks to its ability to provide frame-level binary labels (i.e., 0: Normal, 1: Anomaly) using only video-level labels during training. Despite decent progress on simple anomaly detection (such as an explosion), recently proliferated methods still suffer from complex real-world anomalies (such as shoplifting). This is mainly due to two reasons: (I) undermining the anomaly diversity during training: previous methods assemble diverse categories of anomalies under a unified label, thereby ignoring the category-specific key attribution. (II) Lack of precise temporal information (i.e., weak-supervision): limits the ability of the methods to capture complex abnormal attributes that can viably blend with normal events. Towards addressing this, we plan to first decompose the anomaly diversity into multiple experts for encoding category-specific representations and then to entangle pertinent cues of each expert by exploiting the semantic intercorrelation between them. Further, existing anomaly detection methods primarily focus on immediate detection, lacking the capability to anticipate anomalies well in advance. This shortcoming is particularly critical in systems where early warning can prevent anomalies. By leveraging the strengths of auto-regressive models, which predict future values based on historical data, we aim to extend the predictive horizon, allowing for timely and informed decision-making.

.

Mission confiée

We will leverage state-of-the-art VLMs to bridge the gap between visual data and linguistic interpretations. By interacting with the VLMs and LLMs, users can query, interpret, and refine the detection process, fostering a more dynamic and adaptable anomaly detection system. To further enhance interpretability, we will integrate Large Language Models (LLMs) with Chain-of-Thought (CoT) reasoning and Retrieval-Augmented Generation (RAG) techniques. CoT enables LLMs to break down complex reasoning tasks into intermediate steps, mirroring human cognitive processes. Combined with RAG, which retrieves relevant external knowledge for grounding responses, this approach significantly reduces hallucination while improving anomaly explainability.

Principales activités

The Inria STARS team is seeking a PhD student with a strong background in computer vision, deep learning, and machine learning.

The candidate is expected to conduct research related to the development of computer vision algorithms for video understanding.

Main activities:

- Analyze the requirements of end-users and study the limitations of existing solutions.
- Proposea new algorithm for detectingvideo anomalies (wVAD)
- Evaluate and optimize the proposed algorithm on the targeted video datasets
- Oral presentation and writing reports
- Submit a scientific paper to a conference

Compétences

Candidates must hold a Master's degree or equivalent in Computer Science or a closely related discipline by the start date.

The candidate must be grounded in computer vision basics and have solid mathematical and programming skills.

With theoretical knowledge in Computer Vision, OpenCV, Mathematics, Deep Learning (PyTorch, TensorFlow), and technical background in C++ and Python programming, and Linux.

The candidate must be committed to scientific research and substantial publications.

In order to protect its scientific and technological assets, Inria is a restricted-access establishment. Consequently, it follows special regulations for welcoming any person who wishes to work with the institute. The final acceptance of each candidate thus depends on applying this security and defense procedure.

Avantages

- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
- Social, cultural and sports events and activities
- Access to vocational training
- Contribution to mutual insurance (subject to conditions)

Rémunération

Duration: 36 months
Location: Sophia Antipolis, France
Gross Salary:2 300€ per month