Role: Lead Author, Hardware & Software
Duration: 4 weeks (2020)
Accolades: Finalist, HRI Student Design Competition

Brief

 
Childhood adversities can profoundly shape adult life, leading to psychological and behavioral challenges. Such experiences during the foundational years are connected with heightened risks of conditions like depression and anxiety. Stories, however, play a transformative role in a child's cognitive growth, nurturing empathy, imagination, and divergent thinking. In this work, we delve into harnessing the power of storytelling through an interactive toy—a "transitional object"—to assist children in navigating and comprehending stressors during their developmental years.

Juno is more than a plaything; it's an empathetic ally in a child's world that helps children express and understand their emotions. Using advanced computer vision and machine learning, it reads both verbal and non-verbal cues to gauge a child's emotional state. When it detects emotional challenges, Juno draws from its story database to share relevant tales that offer comfort and guidance. Parents stay connected through an app that monitors their child's emotional well-being, providing insights and alerts for timely intervention. Through its engaging design and patient listening, Juno creates a safe space where children feel understood, while helping prevent minor emotional challenges from growing into larger issues.

The child has a particularly troubling day. He accidentally spills juice on his father’s file. Since no one saw him do this, he decides to not tell anyone, concerned about the consequences. He walks into his room, thinking about what transpired.

As soon as he walks in, Juno’s eyes light up. As the child moves around the room, Juno’s eyes follow, inviting the child to initiate conversation. The child approaches Juno, and when in close proximity Juno greets him

Employing computer vision and natural language processing to identify both verbal and non- verbal cues, Juno recognizes something is awry. He asks the child if everything is fine. The child explains the events that ensued, and his eventual mistake. He confesses to fearing the consequences. Juno intently listens and processes this data.

At bedtime, when the child asks Juno to read him a story, Juno, based on the events of the past day, reads him the story of “George Washington and the cherry tree” that emphasizes on the positive consequences of honesty.

The child feels motivated to be honest and forthcoming, and the following day admits to his mistake and professes his remorse to his father

Prototyping

Detecting Emotions
Creating Juno involved a deep dive into the world of emotional recognition. We aimed for a seamless blend of technology and empathy, ensuring that Juno didn't just detect but truly understood a child's emotions.

1. Visual Understanding: We utilized advanced computer vision to capture facial expressions and body language. This enabled Juno to discern the subtle shifts in a child's mood, from joy to distress.

2. Voice Analysis: Beyond just listening, Juno processes speech using natural language processing. This allows it to pick up on the sentiment behind words, recognizing the underlying emotions from a child's tone and choice of words.

3. Integrative Approach: By combining facial recognition with tone analysis, Juno offers a holistic understanding of the child's emotional state. We harnessed machine learning and integrated IBM Watson's capabilities to enhance this interaction, ensuring Juno's responses are timely and empathetic.

At its core, Juno is powered by a curated technology stack: Node JS, Python, IBM Watson, and FaceApiJS. This ensemble ensures that Juno stands out, not just as a toy, but as an emotionally attuned companion.

Form Explorations