Project SMART (Social Mindreading AI in Real-Time)
Summary
Autistic people face significant challenges in social interaction, leading to social isolation, anxiety, educational underachievement, unemployment, and poor mental health.
Research, pioneered by the Autism Research Centre (ARC) in Cambridge (www.autismresearchcentre.com), has shown that these adverse outcomes are attributable in part to difficulties in Theory of Mind (ToM), or the ability to attribute mental states such as beliefs, thoughts, intentions and emotions to others, to make sense of and predict other people’s behaviour.
ToM is also called cognitive empathy, mindreading, or mentalizing. Here we use the term ToM. This research was summarised in Baron-Cohen’s (1995) monograph Mindblindness (MIT Press). To help autistic people learn to recognize other people’s mental states the ARC has developed two interventions: Mindreading , an electronic encyclopedia of human emotions, and The Transporters, an animation for autistic children (https://resources.autismcentreofexcellence.org/courses/).
Such interventions have shown that practice leads to improvement for autistic people but are limited by their limited real-world applicability.
Project aims
This PhD project proposes the development of a novel social interaction platform leveraging generative artificial intelligence (AI) and wearable technology.
This research project brings together the ARC with an industry partner Emteq (www.emteqlabs.com) to create a wearable technology for social interaction that will allow recognition of mental states in real time whilst viewing real faces from people of diverse ethnicities, in diverse contexts, and across individual characteristics (such as age and gender).
The project has 5 specific aims:
- To develop a dataset of diverse social interaction stimuli for assessing and teaching facial expression recognition and mental state attribution;
- To train AI models to interpret these social cues;
- To integrate these models into a wearable device such as Google Glasses, designed to assist autistic people in real-time social interactions;
- To evaluate the device’s impact on users’ social skills and well-being;
- To conduct community engagement to establish the acceptability and feasibility of the wearable device for autistic people, taking into account its beneficial uses in autism such as the Brain Power software (https://brain-power.com), and prior concerns about such technology.
The experimental approach combines state-of-the-art technologies such as AI-generated digital characters, computer vision, and large language models with wearable smart glasses equipped with sensors for real-time feedback.
The project will include data collection with both autistic and neurotypical participants, algorithm development for personalised social teaching, and the deployment and testing of the wearable device in real-world scenarios.
The anticipated impact is a transformative social interaction tool for autistic individuals, analogous to a hearing aid for those with hearing impairments.
This innovation has the potential to significantly enhance social inclusion and quality of life for autistic people, contributing to the development in assistive technologies for social cognition.
Contact details
Professor Simon Baron-Cohen - sb205@cam.ac.uk
Opportunities
This project is open to applicants who want to do a:
- PhD