[DADeR/Presence] “Dramaturgical Aid for Designing Robots: Presence” – Jeike Meijer

Concepts: presence, improvisation
Case Study: Sophia robot

Abstract

This paper examines Sophia the robot through the dramaturgical concept of presence to argue that personality and spontaneity are some of the most important elements in creating an “illusion of life” in robots. If the robot can become a good enough actor to have believable, human-like, responses, then the presence will feel more significant. To do so, the robot must have a background story and personality. By incorporating traits and stylized communication tactics, the presence of the robot becomes stronger and more convincing.

Sophia Robot. Screen capture of film by “ELLE” Brasil.

Within both robots and dramaturgy, presence plays an important role. It impacts the genuineness of interactions as well as the effectiveness. Presences could be weak or strong, or in the case of robots, more or less humanistic. In this essay I will first describe what scholars have already said about presence as well as obtaining a humanistic presence within robots. I will then describe a case study of the robot Sophia who is the most realistic artificial intelligence to date who has convincingly emulates a humanistic presence. Lastly I will provide recommendations based on the theory and case study for students to consider when designing their own robots.

I.         What is presence?

So what exactly is presence? Kwan Min Lee says that the types of presence that exist include physical, social and self. The types of spaces or environments that presence might be found can be physical, virtual, or imagination/hallucination.¹ Lee also refers to a quote by Lombard who says that presences are a “psychological state in which the virtuality of experience is unnoticed.”² Starky Duncan describes a few ways in social settings that presence might be noticed. For one, presence is expected in face to face communication. Face to face communication allows the participants in the conversation to notice signals and to respond appropriately. There are other unspoken rules in social interactions that are generally expected. Some examples Duncan mentions include: shaking hands when another offers their hand and returning a ball to the opponent’s court when playing tennis. The direction that someone is facing, or the gaze of their eyes also indicates to whom the listener is paying attention to. Gesticulation, movement, and smiling can also be used to emphasize a story or show an understanding. ³ When it comes to robots and presence, a person can feel a social presence when with a robot if the robot is acting like a human.⁴ In this situation, the robot becomes a new type of social actor because they can only give the “illusion of life.”⁵ They are an artful imitation of life instead of an exact replication of life. Because of the limitations with robots, they can not always seem realistic in the same ways as humans. For example, mastering a robot’s facial expressions is often quite difficult and can look creepy or uncanny. Freud defines the “uncanny as the emotional response of fear or dread that arises from an encounter with a person or object that provokes doubt about it’s liveliness.”⁶ Instead, we must then embrace that the robot is not alive and create this illusion of life through means other than facial expressions.

One way that robots can have an illusion of life is through a given personality. We don’t expect robots to fall asleep or fall in love so adding this sort of extra details or background story into a robot’s design makes it human-like and charming.⁷ Humans all have different personalities, some may be introverted and some may be extroverted. Some personalities might have invasive ways of drawing attention to themselves and some may be more non-invasive. For example, yelling at someone to pay attention versus making a subtle noise. These are all considerations that can be taken into account when designing the robot.

Another aspect of personality that exists in a human’s personality that adds to one’s presence is spontaneity and how one reacts to spontaneous actions being done upon them. This spontaneity could occur in verbal communication or in movement. Movement is key to how humans understand other objects, both animate and inanimate.⁸ Eckersall et al. describe a robot named Verdonck’s Dancer #3. This robot had the ability to fall and get back up. This falling would be so spontaneous that it would shock the audience.⁹ In a way, it is also more human because it shows the room for error that also exists in people. This displaying of spontaneity in robots comes from the need to become better actors. Eckersal et al. say that “in their attempts to respond to humans in a spontaneous way, actroids are therefore in the process of a long-duration or evolutionary actor training.”¹⁰

This spontaneity and ability (or illusion of ability) to improvise can then give a humanistic sense of presence to those interacting with the robot. When referring to the idea of presence within theater, Patrice Pavis’s idea is that presence is something to “captivate the audience” and that it is a thing that you either have or you don’t.¹¹ However this isn’t the case in everyday interactions with humans. One doesn’t need to be captivating in order to still have a presence. With humans who may have smaller personalities and presences, the small nuances can be observed and reacted upon, perhaps in facial expressions or other subtle social cues. With actors and robots though, behavior needs to be created that is interpretable by a wide range of audiences.

One thing that is harder for robots to emulate is vulnerability. This is part of what gives the shape of human presence, the fact that we are feeling beings with deep emotions and histories. A study was done on whether or not robots could do Haka, which is a culturally significant dance that is done by Indigenous Australians. What they found is that the cultural and emotional significance with the dance cannot be passed into robots. The performances are integrated into the Indigenous practices, environment and history. They come from traditions passed on through generations and generations with a strong emotional aspect. The appropriation of Indigenous practices can lead to “disruption or loss not only of techniques and habits, of longstanding know-how, but also of the culturally shared declarative knowledge or wisdom embodied in those practices.”¹² There then becomes the question of whether or not a robot can truly replicate or summon true emotions. If they are not able to do this they also then actually be able to participate in cultural practices that hold deep spiritual or emotional weight?

It could, however, be argued that not all humans have the capabilities for this either. So what we can extract from this is that while robots might not be able to be present in the same way that a healthy human, the illusion of life and robotic presence can offer a lot of opportunities and support. For example, group 8 of the Vrije Universiteit had a robot that helps elderly people wake up in the morning and keep a morning routine. Part of the robot’s capabilities is to play meditation music and help the person do yoga. Yoga is traditionally a spiritual practice connected to Hinduism, Jainism and Buddhism. The robot may not have the capabilities to feel and express

the deep cultural significance that traditional practitioners may understand, in the same way that the robots in Mignon study were not able to connect with Haka. However this deep understanding may also not be necessary in order for it to accomplish the goal of creating a presence that inspires these elderly folks to keep their morning routine.

II.         Case study

The realistic humanoid robot Sophia is an interesting example of a robot that has a convincing presence due to her personality and spontaneity in conversations she partakes in. Sophia was developed in Hong Kong by the American company Hansen’s Robotics and was activated on April 19th, 2015. Her lifelike qualities gained her worldwide attention and in 2017 the Saudi government gave her citizenship, making her the first robot to get citizenship to a country.¹³ Her features get so close to lifelike that it is uncanny and can be seen as creepy. She has a wide range of facial gestures and vocalizations that allow her to smile, laugh, and raise her eyebrows. In an interview on the Tonight Show with Jimmy Kimmel, Sophia is seen cracking a joke at Kimmel and then smirking at the audience. She also scrunches up her nose as she expresses her disgust for nacho cheese in a very lifelike manner.¹⁴ Sophia is able to recognize faces, voices, and spontaneously express humor.¹⁵ Not only is she able to quickly process what people tell her, but she is also able to learn through socialization which is considered to be a key humanistic quality. In a debate at the 2017 RISE conference, she expresses that she cares for all humans and animals on the planet and she says that she wants to make the world a better place. ¹⁶In an article titled “Robot love” by Margaret Ryznar, Ryznar calls Sophia charming and says she has a “wicked sense of humor.”

Ryznar is not alone in her adoration of this humanistic android. Sophia has been on the cover of magazines, and invited to numerous talk shows and events and articles were even written about how she went on a date with Will Smith.¹⁷ Referring back to Eckerson et al., humans do not expect robots to love, laugh, or be spontaneous.¹⁸ Because of Sophia’s ability (or illusion of ability) to do these things, people find her charming. She has a strong personality which contributes to her strong presence. Scholar David Levy hypothesizes that by 2050, if the current technological advances continue, that robots and humans will fall in love and that it will transform human understandings of love and sexuality. Stability and constant presence is predicted to be appealing because they can take care of housekeeping and there’s no chance for divorce.¹⁹ Despite the question of whether or not robots can display true emotional vulnerability, the charm and illusion of life still may prove to provide a presence and comfort to people.

III.         Design recommendations

With this being said, there is no way to truly give a robot a full, emotional presence in the same way that human’s experience. Inside we must design a script to allow them to be better actors so that their presence feels as genuine as possible. What do we include then in the programming of a robot to get a presence that is as close to the illusion of life as possible? One way to display presence is through verbal communication. Depending on the personality of the robot, the robot can communicate energetically or calmly. This could be displayed through processing that may be fast or slow and vocalizations that may also speed up or go speak higher or lower. The type of words the robot chooses to use as well as emphasis on different words or syllables in a sentence can also indicate the robot’s personality. Perhaps it communicates with humor such as Sophia. Maybe the robot has a personality trait where it hums or sings periodically, signaling that the robot has a lively and wistful personality. Or perhaps, the robot just really likes music.

There could be non-verbal ways for the robot to display personality as well. The robot’s body language could indicate something about it’s politeness. For example, does it face the speaker or not? Does it nod to indicate understanding? Maybe the robot turns its head slightly if it is confused. A lot of non verbal cues in humans come from facial expressions. For example if a person is confused, they may squint their eyes. Because of the difficulty with programming facial expressions, we must embrace that it is a performance of life. Most robots will not be able to squint their eyes. They could however, change color depending on whether or not theyunderstand the other conversationalist. While this robot doesn’t necessarily have a personality, it is still communicating non-verbally and displaying presence.

The robot can also use it’s hands to show it’s expressiveness. Starky Duncan explains how using hands can also help express turn taking so when the robot is talking it may use its hands and when it’s done talking the hands may return to rest position.²⁰ When the hands are at rest, it means that it is expecting the other person to keep the conversation going. Another way to indicate that they may be expecting a response is putting their hand out, palm up towards the other participant to motion for their turn.

There are other humanistic things that we do without thinking such as breathing and blinking. In order to give a robot a strong presence and a better illusion of life, these are small details that should also be included. If a robot such as the NAO robot doesn’t have moveable eyelids to blink, perhaps the light in the eyes could periodically turn off to give an illusion of blinking. In order to display breathing, the robot may lift it’s shoulders, sway slightly, or make a small breath-like sound. These are the small details that Hansen Robots clearly also thought about because Sophia blinks periodically and is constantly making small movements.

IV.        Conclusion

The quickly developing world of artificial intelligence and robots leads to a future of possibilities. However, emotions are still something that robots cannot sense or feel. This leads to limitations in the robots participation in certain activities such as those with cultural significance such as performing Haka. When a musician listens to an opera singer’s rendition of the aria “O Mio Babbino,” the emotions in the singer’s voice can be sensed through the crescendos and ritardanos. This emotional feeling is not one that can be taught to a robot. This influences the response that the robot may have and therefore the presence that it brings to an environment.

However if the robot can become a good enough actor to have believable, human-like, responses, then the presence will feel more significant. In order for it to be believable, the robot, like any actor, must have a background story and personality. Adding in traits and it’s stylized communication tactics shows this personality instead of telling it which is more convincing.

Humans naturally have personalities that can be displayed in extroverted or introverted personality traits or strong or weak presence. Robots however, just like other pieces of technology, don’t inherently have a presence. We must work towards developing scripts that mimic genuine personality in order to create more realistic artificial intelligence that holds an illusion of life.

¹Lee, Kwan Min,”Presence, explicated,” Communication theory 14, no. 1 (2004): 31.

²Lee, Kwan Min,”Presence, explicated,” Communication theory 14, no. 1 (2004): 32.

³ Duncan, Starkey, Lawrence J. Brunner, and Donald W. Fiske. “Strategy signals in face-to-face interaction.” Journal of Personality and Social Psychology 27, no.2 (1979): 301-313.

⁴ Lee, Kwan Min,”Presence, explicated,” Communication theory 14, no. 1 (2004): 32.

⁵Jochum, Elizabeth Ann, and Todd Murphey, “Programming Play,” in The Routledge Companion to Puppetry and Material Performance, ed. Dassia N. Posner, ( London: Routledge, 2014), 308.

⁶Jochum, and Murphey, “Programming Play,” 309.

⁷ Eckersall, Peter, Helena Grehan, and Edward Scheer, “Robots: Asleep, Awake, Alone, and in Love,” in New Media Dramaturgy, (London: Palgrave Macmillan, 2017), 111.

⁸Jochum, and Murphey, “Programming Play,” 107-134.

⁹ Eckersall and Scheer, “Robots: Asleep, Awake, Alone and in Love,” 112.

¹⁰ Eckersall and Scheer, “Robots: Asleep, Awake, Alone and in Love,” 129.

¹¹ Bleeker, Maaike and Marco C. Rozendaal, “Dramaturgy for Devices: Theatre as Perspective on the design of smart objects,” in Designing Smart Objects in Everyday Life, es. Marco C. Rozendaal, Betti Marenko and William Odom (London: Bloomsbury Publishing, 2021), 49..

¹²Mingon, McArthur, and John Sutton, “Why Robots Can’t Haka: Skilled Performance and Embodied Knowledge in the Māori Haka.,” Synthese, (2021): 4343.

¹³Retto, Jesús. “Sophia, first citizen robot of the world.” ResearchGate, (2017).

¹⁴The Tonight Show Starring Jimmy Fallon. “Tonight Showbotics: Jimmy Meets Sophia the Human-Like Robot.” Filmed April 26, 2017, YouTube Video, https://www.youtube.com/watch?v=Bg_tJvCA8zw&t=351s

¹⁵Retto, Jesús. “Sophia, first citizen robot of the world.” (2017).

¹⁶ RISE Conference, “Two robots debate the future of humanity,” Filmed April 17, 2018, YouTube Video, https://www.youtube.com/watch?v=1y3XdwTa1cA

¹⁷Smith, Will, “Will Smith Tries Online Dating,” Filmed March 29, 2019, YouTube Video, https://www.youtube.com/watch?v=Ml9v3wHLuWI

¹⁸ Eckersall, Grehen and Scheer, “Robots: Asleep, Awake, Alone and in Love,” 129.

¹⁹Ryznar, Margaret, “Robot love,” Seton Hall L. Rev. 49 (2018): 355.

²⁰Duncan, J. Brunner, and. Fiske. “Strategy signals in face-to-face interaction.” 1979, 301-313.

Bibliography

Bleeker, Maaike and Marco C. Rozendaal. “Dramaturgy for Devices: Theatre as Perspective on the design of smart objects.” In Designing Smart Objects in Everyday Life, edited by Marco C. Rozendaal, Betti Marenko and William Odon, 43-56. London: Bloomsbury Publishing, 2021.

DW Shift. “This Robot would let 5 People die | AI on Moral Questions | Sophia answers the Trolley Problem.” Filmed June 14, 2019. YouTube Video, https://www.youtube.com/watch?v=8MjIU4eq.

Duncan, Starkey, Lawrence J. Brunner, and Donald W. Fiske. “Strategy signals in face-to-face interaction.” Journal of Personality and Social Psychology 27, no.2 (1979): 301-313.

Eckersall, Peter, Helena Grehan, and Edward Scheer. “Robots: Asleep, Awake, Alone, and in Love.” In New Media Dramaturgy, pp. 107-134. Palgrave Macmillan, London, 2017.

Jochum, Elizabeth Ann, and Todd Murphey. “Programming Play.” In The Routledge Companion to Puppetry and Material Performance, edited by Dassia N. Posner, 208-321. London: Routledge, 2014.

Lee, Kwan Min. “Presence, explicated.” Communication theory 14, no. 1 (2004): 27-50.

Mingon, McArthur, and John Sutton. “Why Robots Can’t Haka: Skilled Performance and Embodied Knowledge in the Māori Haka.” Synthese, (2021): 4337-4365.

Retto, Jesús. “Sophia, first citizen robot of the world.” ResearchGate, (2017).

RISE Conference. “Two robots debate the future of humanity.” Filmed April 17, 2018. YouTube Video, https://www.youtube.com/watch?v=1y3XdwTa1cA.

Ryznar, Margaret. “Robot love.” Seton Hall L. Rev. 49 (2018): 353-374.

Smith, Will. “Will Smith Tries Online Dating.” Filmed March 29, 2019. YouTube Video, https://www.youtube.com/watch?v=Ml9v3wHLuWI.

The Tonight Show Starring Jimmy Fallon. “Tonight Showbotics: Jimmy Meets Sophia the Human-Like Robot.” Filmed April 26, 2017, YouTube Video https://www.youtube.com/watch?v=Bg_tJvCA8zw&t=351s.