Phd Position F - M Phd Position F - M Privacy Risk Assessment And Enforcement In Agentic ai Systems H/F

INRIA

PhD Position F/M PhD Position F/M Privacy Risk Assessment and Enforcement in Agentic AI Systems
Le descriptif de l'offre ci-dessous est en Anglais
Type de contrat : CDD

Niveau de diplôme exigé : Bac +5 ou équivalent

Fonction : Doctorant

A propos du centre ou de la direction fonctionnelle

The Inria research centre in Lyon is the 9th Inria research centre, formally created in January 2022. It brings together approximately 300people in 19 research teams and research support services.

Its staff are distributed at this stage on 2 campuses: in Villeurbanne La Doua (Centre / INSA Lyon / UCBL) on the one hand, and Lyon Gerland (ENS de Lyon) on the other.

The Lyon centre is active in the fields of software, distributed and high-performance computing, embedded systems, quantum computing and privacy in the digital world, but also in digital health and computational biology.

Contexte et atouts du poste

The PhD will be hosted by Inria, within the PRIVATICS team in Lyon, with close collaboration involving CNRS/LIRIS Lyon and Inria Lille (MAGNET Team). The project is funded by INESIA's initiative on the evaluation of AI. The scientific team brings complementary expertise in privacy, security, trustworthy AI, distributed systems, privacy attacks, anonymisation, differential privacy and confidential computing.

The thesis will be co-supervised by:

- Mohamed Maouche, Researcher at Inria
- Raouf Kerkouche, Researcher at Inria
- Sonia Ben Mokhtar, Researcher at CNRS

Scientific context

Recent advances in Large Language Models (LLMs) have enabled the development of agentic artificial intelligence systems. Unlike conventional conversational systems, AI agents can generate action plans, invoke external software tools and APIs, communicate with other agents, and perform actions on behalf of users. These capabilities create opportunities in areas such as personal assistance, healthcare, software development, mobility, supply-chain optimisation and resource management.

At the same time, agentic AI introduces significant privacy and security challenges. Agents may accumulate and process sensitive personal information, use it to interact with external services, and transmit data to tools or other agents that are not fully trusted. Their autonomous and loosely defined interactions can lead to accidental disclosure, malicious extraction of private information, misuse of tools, prompt-injection attacks, or privacy leakage caused by compromised agents and software vulnerabilities.

Mission confiée

Assignments :

This PhD project will investigate how to evaluate and enforce privacy in agentic AI systems. The central objective is to understand how sensitive information is collected, transformed, transmitted and potentially exposed throughout an agentic workflow, and to design mechanisms that enable agents to interact and act on behalf of users while reducing privacy risks.

The research will address privacy risks at both the client side, where users express their intent through text, voice, images, preferences or contextual data, and the server side, where agentic systems generate action plans, exchange data between agents, and invoke external tools and APIs.

Research Objective: The thesis will develop and evaluate a privacy risk-assessment and privacy-enforcement framework for agentic AI. Specifically, the project will focus on:

- Privacy risk assessment of user data: analysing the sensitivity and uniqueness of information used to express user intent, and estimating the consequences of data leakage, including re-identification risks.
- Sensitive data-flow analysis: tracking how personal and confidential information propagates through agentic workflows, including communication between agents and interactions with external services.
- Action-plan risk analysis: evaluating the sensitivity of generated action plans and assessing how much external tools, APIs and communication partners can be trusted.
- Black-box auditing and privacy attacks: assessing external tools and agentic components through black-box analysis and existing privacy attacks to identify potential information leakage.
- Privacy-enhancing technologies: developing or adapting mechanisms such as data minimisation, anonymisation, encryption, controlled information sharing, differential privacy, privacy proxies and confidential computing.
- Secure agent execution: integrating privacy protections on both the client and server sides so that sensitive data is protected before transmission and action plans can be executed with controlled access to external tools and APIs.
- Empirical evaluation: validating the proposed methods on realistic agentic AI workflows and measuring the trade-offs between privacy, utility, security and system performance.

The expected outcome is a robust agentic framework that supports safe multi-agent interactions while protecting client data, together with practical guidelines and tools for developers and organisations deploying agentic AI systems.

Principales activités

- Conduct original research on privacy and security for LLM-based and multi-agent AI systems.
- Design a toolbox for privacy-risk assessment on both client-side inputs and server-side agent workflows.
- Implement auditing methods for sensitive data flows, external tools and APIs, and agent-to-agent interactions.
- Develop and evaluate privacy-enhancing mechanisms tailored to agentic AI systems.
- Build experimental prototypes and benchmark privacy, utility and system-performance trade-offs.
- Collaborate with researchers from Inria Lyon, Inria Lille and CNRS/LIRIS Lyon.
- Disseminate research findings through peer-reviewed publications and presentations at leading venues in machine learning, privacy, security and distributed systems.

Compétences

We are looking for a candidate with:

- Good programming skills, preferably in Python, and strong analytical abilities.
- A solid background in machine learning, computer security, privacy, distributed systems, or a closely related area.
- Knowledge of Large Language Models, agentic AI, privacy-enhancing technologies, or information-flow analysis is a plus.
- Interest in experimental research, system implementation and rigorous evaluation.
- Good written and spoken English.

Avantages

- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage under conditions

Rémunération

2300 euros gross salary /month