Thursday, April 17, 2014

Can music convey a personality?

I recently tried out Beau Sievers' super-cool Bouncing Ball software. It generates both – you guessed it – an animated bouncing ball, and also music to accompany its movements. By changing around 5 parameters, you can try and get it to match an emotion or mood.

How does it work? Well, you choose the music clip's speed, irregularity, consonance (i.e. major or minor), pitch range, and probability of making upwards or downward sequences. There is no set score – the software chooses the next note and timing based on probabilities.

Example: "Cheerful" parameter settings

Now, I'm wondering if these 5 parameters can convey not just emotions, but personality types as well. I'm hoping to contribute to Julien Richard's NaoBrain project.

Here are a few attempts. Let me know what you think!

Cheerful: happy, bright, optimistic and positive.
Cute: like a toddler, infantile, a bit unsure
Kind: a caregiver, like Mary Poppins!
Machine: your typical machine
Sarcastic: dry and ironic, like House
Serious: formal, butler-like

Tuesday, February 18, 2014

Why Designing an Algorithm is like Designing a Chocolate Chip Cookie Recipe

What is an algorithm? In a world where understanding code and technology is becoming essential, I'd like to offer this –perhaps obvious–analogy.

Algorithms are like Chocolate Chip Cookie Recipes.

An algorithm is a set of steps to solve a problem, while trying to minimize or maximize something. A recipe is a set of steps to solve the problem of "I'm hungry!" while fitting your constraints.

An algorithm has input, procedures, and output. A recipe has ingredients, recipe steps, and a finished dish.

There can be many different algorithms to do the same thing. There can be tons of sorting algorithms, just like there can be a billion Chocolate Chip Cookie recipes on

Algorithms try to optimize for different scenarios. They try to minimize space and cost, or maximize speed, or elegance.

Recipes also try to optimize for different things. You can maximize speed, because you want to eat cookies NOW. You can try minimize space, and make it all in one bowl. You can try and make it resource-non-intensive, because you're lazy. You can try to make it really beautiful, because you enjoy the look as much as the taste. Or you can maximize deliciousness while minimizing calories. A multiparameter optimization!

You can optimize all these things, but it’s up to you, the algorithm designer–the cook–to decide which metric is important to YOU.

Designing an algorithm also requires iterative testing. You can try combining different methods, and must check the result and tweak parameters until you think it works well. Sometimes you just throw it all out if it's a mess and start over. When making a recipe, you can combine steps from different recipes, lick the spoon, and tweak ingredient amounts and until you deem it worthy. You, the cook, decide when it's good and ready to serve up to others.

Deciding which problem to solve is also pretty important. You're a chef, you have all these skills–do you use them to cater wedding parties? To open a restaurant? To make designer cupcakes? Which problem in the world do you solve with your skills?

Algorithms are like recipes. We even have cookbooks :) Have I missed anything? 

Whelp, I'm off. Let me know if this has been a totally obvious post. All I know is, these pictures are making me hungry...

Edit: Some more contributions from a friends:

Experience in breaking things can help guide you in your design. Copying an algorithm line-by-line can sometimes get the job done, just like following a recipe word-for-word can give you cookies with no effort. But you don't necessarily understand what you did. What really teaches you, though, is messing up. What? If I replace margarine with butter, then my cookie edges burn? What? My algorithm doesn't work when I use X instead of Y? So experience and trial and error matter. I'd like to suggest that this is how many coders that grow up learning to code, before taking any computer science classes.

However, a theoretical background matters a lot, too. How do you become a master chef? You learn the theory. You know all of your ingredients, their properties, and methods. You know that egg whites contain proteins that break down when salt is added. You know that the point of whipping is not to mix vigorously, but to add air into the mixture, so a flick-lift method is better than a spinning whip. With a theoretical background, you can guess what the outcome will be even before you do it. A good background in computer science and algorithms will give you this.

Tuesday, January 21, 2014

Empathy towards machines: When communicating less is more

Recently I spoke with a professor regarding empathy and machines. Her mom had a close relationship with her 20-year-old washing machine. If the washing machine acted up, her mom would ask it -- do you need more water? Are there too many clothes? How about I take one piece out and we'll see if you'll start.

This is just what parents do when they interact with their baby. They constantly have to put their mind in the infants shoes. Are you hungry? Did you wet your diaper? Are you just tired? And they try things till it helps.

I think this kind of perspective-taking, active empathy connects us. And maybe this explains why we loved the mysterious R2-D2 so much. We had to put ourselves in its shoes to understand his beeps, his movements. And by doing this curiosity exercise, we cultivated empathy.

As another example, the Fish-Bird robots communicated with people only with simple printouts and movements. People interacted with it for ages. What do you think? Could less be more?

Wednesday, December 18, 2013

What does "innate" really mean?

A cool story by biologist Dr. Stuart Firestein, from the book This Will Make You Smarter:
"Instinct refers to a set of behaviors whose actual cause we don't know or simply don't understand or have access to; and therefore we call them instinctual, inborn, innate. Often this is the end of the exploration of these behaviors, they are the nature part of the nature-nurture argument [...] and therefore can't be broken down or reduced any further. But experience has shown that this is rarely the truth. In one of the great examples of this, it was for quite some time thought that when chickens hatched and they immediately began pecking the ground for food, this behavior must have been instinctive.
In the 1920s a Chinese researcher named Zing-Yang Kuo made a remarkable set of observations on the developing chick egg that overturned this idea — and many similar ones. Using a technique of elegant simplicity he found that rubbing heated Vaseline on a chicken egg caused it to become transparent enough to see the embryo inside without disturbing it. In this way he was able to make detailed observations of the development of the embryo from fertilization to hatching. One of his observations was that, in order for the growing embryo to fit properly in the egg, the neck is bent over the chest of the body in such a way that the head rests upon the chest just where the developing heart is encased. As the heart begins beating the head of the chicken is moved in an up-and-down manner that precisely mimics the movement that will be used later for pecking the ground. 
Thus the "innate" pecking behavior that the chicken appears to know miraculously upon birth has, in fact, been practiced for more than a week within the egg."
The book's contents are all available online.

Books on Robots and Emotions

Are you interested in reading up on robots and emotions?

Here are some of the books I've come across while writing my thesis. I've put an "easy-to-read" rating on each one. The more stars, the more textbook-like it is, with research and modeling -- but it might be harder to dive into if you're a beginner.

The books with one ★ are easiest to read to get a quick overview of the field.
Books with ★★ are more comprehensive, and still readable.
Books with ★★★ are a compendium of papers from different researchers. Your mileage may vary.

In order of recommendation from me:

How to build an Emotional Robot
★★ Designing Sociable Robots, Cynthia Breazeal

Emotions for Human-Computer Interaction
★ Affective Computing, Rosalind Picard
★★★ Blueprint for Affective Computing, Klaus Scherer et al.

Artificial Intelligence based on Emotions
★★★Who Needs Emotions? The Brain Meets Robot, Jean-Marc Fellous and Michael Arbib

What are Emotions? Why do we need them?
★★ Descartes Error, Antonio Damasio

Emotion Design for Interfaces
★ Design for Emotion, Trevor Van Gorp
Emotional Design: Why we love or hate everyday things, Donald Norman (Last chapter is on robots)

Emotions, Personality - Faking It
★ The Media Equation, Byron Reeves and Clifford Nass (Only partly mentions emotions)

Social Implications of Emotional Robots
★★ Alone Together, Sherry Turkle [Warning: This book is very critical of robots with emotions.]

...and just a really well-edited book that can give you a fresh perspective on emotions. I love this book:

Emotion in Music
★★★Handbook of Music and Emotion, Patrik Juslin and John Sloboda

Wednesday, December 11, 2013

The Development of Primary Emotions for Robots (Intro)

Robot and Frank, 2012
"When dealing with people, remember you are not dealing with creatures of logic, but creatures of emotion.” – Dale Carnegie 
The most emotional moments of our lives are the most memorable. Our best friends help us lead lives that are happy and bright, and are endearingly empathetic when we’re down. Emotions colour our world, our interactions, our words, from humming in the morning over breakfast, to smiles before sleeping at night. Positive emotions help us to be more creative, be more optimistic, and even work harder. Negative emotions help us focus, narrow our field of view to attack a problem, or change course when one direction isn’t working out.

Robots show promise in helping us in these emotion-governed lives. Just as the Internet and mobile technology has made us more connected, new robotic technologies are opening a door towards supporting an aging society. In Japan, almost 25% of the population is over 65 years old, and they seek a life of retirement with independence in the community, physical activities, and an active social life.

To meet the rising demand for healthcare workers and more, the Japanese government has estimated that the service robot market will reach over 4900 billion yen by 2035, exceeding the demand for robot manufacturing by almost twofold. It is hoped, for example, that robots can help the bedridden become mobile, and the dependent become independent.

Yet robots have to overcome the challenge of navigating our world, because it is not always black and white. 

Imagine a healthcare robot overseeing an elderly patient named Linda at the hospital – the robot is set to close the room by 9pm. Soaked by the rain, the patient’s daughter, Mary, knocks on the hospital room door. She has driven 50 kilometers from the airport, but a thunderstorm has delayed her arrival. Mary yearns to hold her mother’s hand – it has been 3 years since their last meeting. Linda is delighted to see her daughter through the hospital room window, but it is now 9:01pm. Crestfallen, the mom and daughter eyes meet, as the healthcare robot locks the door with a loud thud. 

The rules are rules. 
"The heart is a strange beast and not ruled by logic.” – Maria V. Snyder
Nurse Noakes (Cloud Atlas, 2012) runs the nursing home with an iron fist. 
Robots do not share our capacity for emotion. In science fiction, Star Trek’s android lieutenant Data was described as human-like in many ways, except that he lacked emotions: “human behavior flows from three main sources: desire, emotion, and knowledge,” Plato once said, and Data had goals and knowledge, but no emotion. In many futuristic movies, this emotional shortfall drives robots to take over the world. Like history’s worst dictators, the robots’ calculating brilliance, logic, and lack of empathy bind together in cruel combination.

It is easy to see why, in a 2012 survey, 60% of EU citizens stated that robots should be banned in the care of children, the elderly, or the disabled. Large majorities would also agree to ban robots from ‘human’ areas such as education (34%), healthcare (27%) and leisure (20%) [their quotes]. Of course, in certain environments like factories, bomb-detection or remote operating tables, the precision and predictability of robots is a necessity. Yet a new breed of “service robots” are advancing to our doorstep quickly, with the potential to change the lives of children and the elderly, able-bodied and disabled, students and more. For robots to be accepted in our daily lives as helpers, we must release robots from their pure, programmed logic and make them more emotional, more empathetic, to interact with humans on their own terms.

How do we start to build such a robot? One guiding principle could be to look to human development for inspiration. Just as each human has linguistic abilities (whether through voice or sign-language), each human is equipped with the capacities of emotion expression and understanding. And whether they were raised in Japan, the USA, China or France, each person is unique based on their upbringing and environment. They may express happiness loudly or quietly, they may fear snakes or love snakes. They may be more or less sympathetic. They may openly declare displeasure or only show it through one eyebrow. Their abilities may fall on a spectrum of what we consider underdeveloped emotional intelligence, or autism. Clearly, there is no one-size-fits-all definition, and likewise, a robot’s emotions should be adaptive, too. Sometimes this zealous focus on pliable, human-like models may appear to be a detriment to the short-term accuracy of the systems we engineer. But with the goal of autonomous, ever-learning robots, our hope is that in the long-term, we will be building the foundation of a powerful artificial emotional intelligence. 

Monday, December 09, 2013

Emotions as a basis for theory of mind

Artificial intelligence researchers may have something to learn from the latest trends in autism therapy: theory of mind springs from emotional understanding.

From an article on "FloorTime" therapy, based on the Developmental, Individual-Differences, Relationship-based (DIR) model:
"Psychologists and researchers in autism have coined the term "theory of mind" to describe the ability to understand how other people reason as they do. Greenspan and his associates asked themselves, Why do many autistic people lack theory of mind? And why can't autistic children make the leap into abstraction? From a traditional developmental point of view, there was no reason to assume that autistic children would have trouble conceptualizing abstractions. The pioneering Swiss psychologist Jean Piaget had persuasively argued long ago that abstractions are grasped when a child operates on his environment (he pulls a string, and a bell rings: causality). But Greenspan was convinced that some mechanism must be missing in an autistic baby's mind. What was it? The answer was staring him right in the face. Or, rather, the answer was in all those young faces that simply couldn't look him in the eye. Greenspan and his colleagues made a leap: these children, they suddenly realized, wouldn't understand abstractions until they understood their own emotions
Already celebrated for his work in developmental psychiatry, Greenspan had, by observing the dysfunction of autistic children, come to a turning point in his understanding of human cognitive development. He understood that everything a child does and thinks as he is developing he does largely because of his emotions. Children apply to the physical world what they have already learned emotionally; they are not, as Piaget thought, introduced to abstractions by the physical world. "The first lesson in causality," Greenspan says, "is not in pulling the string to ring the bell. The first lesson in causality happens months earlier—pulling your mother's heartstrings with a smile in order to receive one back." Furthermore, he says, the earliest concepts of math are nothing but reasoning driven by emotion. "For instance, when a child is learning concepts of quantity, he doesn't understand conceptually, he understands emotionally, in terms of his affective universe. What is 'a lot' to a toddler? It's more than you expect. What is 'a little'? It's less than you want.""
Perhaps, too, on the path to Kurzweil's singularity and truly intelligent robots, emotions are at the starting line.

Edit: Here's a video supplement called From Emotion to Comprehension that talks about true understanding of language.

"For a kid that's going through the normal stages of emotional development, apple's not just the name of a fruit that's round, shiny and red -- it's the name of something that's juicy and makes a yummy crunchy sound when you bite into it.

But for a kid who's not been taken through these stages of emotional development, apple's just a label."