Lucknow, May 25: Indian Institute of Management Lucknow Researchers have conducted a new study exploring the growing use of Artificial Intelligence (AI) companion applications (smartphone applications) and the hidden psychological and social risks associated with them
The rise of AI companion applications has transformed the way people communicate, seek emotional support, and build connections. These applications have taken on roles traditionally associated with human interactions. While these applications/platforms are gaining popularity for providing comfort and companionship, limited research has been conducted to assess the harmful effects they can create on user behaviour.
To address this gap, Prof. Pradeep Kumar, Professor, Information Technology and Systems, IIM Lucknow along with his research scholar, Ms. Chitra Gautam conducted a study titled, “Are AI Companions Real Companions? A BERT-Based Study of Replika Reviews”. The findings of this research have been published in the prestigious Journal of Consumer Behaviour.
To conduct this study, the research team analysed more than 157,000 user reviews from the Google Play Store using advanced AI-driven language analysis tools. As an additional step, the research team combined large-scale Natural Language Processing (NLP) with consumer satisfaction modelling to examine both user experiences and behavioural outcomes in the context of AI companion apps.
Further, by introducing relationship-related stressors alongside technological and cognitive concerns, the research team extended its framework to explore how negative experiences with AI companions can translate into measurable behavioural and psychological outcomes.
The research team identified six major categories of negative user experiences:
- Technical and app-related problems
- Financial frustrations linked to subscriptions and paywalls
- Inappropriate or disturbing AI behaviour
- Privacy, ethical, and security concerns
- Problems affecting real-life relationships and social interactions
- Behavioural and psychological harms, including addiction and emotional dependence
Speaking about the key findings of the study, Prof. Pradeep Kumar said, “We found that many users showed signs of excessive emotional reliance on AI companions. Users increasingly turned to AI instead of human relationships for emotional comfort, which in some cases contributed to social withdrawal and reduced real-world interaction. We also found that some users became dependent on AI apps for emotional regulation and mental health support.”
Researchers warn that such overreliance could become risky if these technologies are used without safeguards.
The research team also found that even users who reported harmful experiences, such as addictive behaviour or unhealthy emotional attachment, still rated the app highly satisfying, creating a “technology paradox”. This finding reveals that traditional customer satisfaction scores may fail to capture deeper psychological risks associated with emotionally driven AI products.
Speaking about concerns around ethical AI design, Ms. Chitra Gautam said, “Based on the findings of our study, it is essential for AI companion platforms to integrate stronger safeguards against harmful behaviour and reduce features that encourage emotional dependency.”
As AI companions, which can create a direct psychological impact on users, are becoming popular worldwide, the findings of this study call for regulators and policymakers to recognise AI companion apps as a separate category from task-oriented AI systems.