Title
Advancing Contextual Appropriateness in Personal AI Agents: A Role-Based Framework
Abstract
As personal AI agents become increasingly general purpose, they must juggle a wide range of social roles - from companions to therapists to career advisers - within a single human-AI relationship. Current alignment strategies often default to a single assistant persona, resulting in models prone to harmful, context- and user-specific behaviors. Here we propose a role-based framework of appropriate behaviour in personal AI agents. According to this framework, we should design personal agents to be capable of picking the right role at the right time, adhering to the normative boundaries of that role, and shifting to a different role when interaction objectives evolve. However, general-purpose agency introduces three fundamental challenges to this approach: role misidentification, role brittleness, and role spillover. We conclude with recommendations for empirically evaluating role-appropriateness to shape a safer future for human-AI cooperation.
About Arianna
Arianna Manzini is a Senior Research Scientist at Google DeepMind, where she works on ethical and societal considerations around advanced AI agents. Prior to Google DeepMind, Arianna was a Research Associate at the University of Bristol, where she worked on a project on trustworthy autonomous systems. Arianna holds a PhD on ethical concerns around emerging technologies in genomics from the University of Oxford.