Get all the news you need about Smart Buildings with the Memoori newsletter

Which research categories are you interested in?

“Imagine your fridge working with you to eat healthier, or your wearable and TV team up to get you off the couch! Your bathroom mirror senses that you’re a bit stressed and interacts with your lighting to adjust it while turning on the right mood-enhancing music,” it states on the website for emotional recognition firm Affectiva.

Affectiva believes that in our world of hyper-connected smart technology and appliances, our devices have lots of cognitive intelligence but no emotional intelligence. And by using a variety of sensory technology, such as wearables and video surveillance, we can take the internet of things (IoT) to the next level – sensing human emotion to predict and react to human behaviour.

A smart supermarket might not only sense the number of people queuing but also identify the level of frustration in the queue when making the decision to open more tills. A smart hospital may not only measure a patient’s vital signs but also his emotional state to prompt a nurse to visit or to preempt an emergency situation. An office could even use emotion recognition to increase the productivity of its workers.

Emotion is not logical however, therefore not very predictable. There’s something not quite right about combing words like “emotional data” or “emotional analytics”. Emotion seems too complex for quantification and digital interpretation.

On the other hand, the forth technological revolution includes superior sensory systems, learning ability and big data analytics that promise to elevate the power of data to understand even the most complex things.

Data driven emotion recognition has been a feature of marketing companies for many years, but even those within the marketing industry are still embarrassed to accept it. “Then we take all that data we’ve gathered and we run it through a tool which analyses it to pull out the ‘emotionality’. Yes, it does kill me a little inside that this is now a word and I hate myself a little bit that it’s one I’m using more and more in business meetings,” said Lydia Daly, senior vice-president of social media and branded content strategy at full-service marketing agency Viacom Velocity.

Corporate marketing machines such as Procter & Gamble, Unilever, Coca-Cola, Diageo or Mondelez may be reluctant to talk about it, but they are all using emotional analytics. The fact that they are doing so suggests that it could be a new and exciting frontier for big data.

“It is not something we can ignore,” Daly says. “It’s the evolution of measuring sentiment. At the moment, I’d say take it with a grain of salt, because it should be seen as something that is evolving and marketers should be exploring.”

It seems likely that advances made in the competitive, fast-paced world of marketing could lead to more sophisticated emotion recognition systems that could be used in other sectors, such as smart buildings and cities. Consider a vehicle for example, wouldn’t it be beneficial if a car could sense if the driver is feeling angry or distracted, then warn them of the dangers of driving in such a state, or even prevent them from driving.

Back in the smart building that is trying to optimize comfort and productivity. An emotion recognition system may sense that an executive is feeling angry, realize that is counter-productive, then try to alleviate their anger by adjusting the temperature and light in their office, or by playing soothing music.

Would this work, or might it lead to the executive realizing what the system is trying to do and this increases their anger? Could the emotion recognition system inadvertently be causing the emotion it is trying to reduce? I think we’d all agree that technology has the ability to generate anger in humans.

We may then quickly realize that by recognizing emotion and trying to change it “for the better,” we are in fact practicing emotional control, on the scale of a building or city that’s population control. It conjures thoughts of George Orwell’s Big Brother, as is often the case with this generation of technology.

Police, for example, may use emotion recognition to scan faces in a crowd to preempt when a peaceful protest may turn violent. Their preemptive measures, however, such as increasing police presence, would more likely be the reason for a change in the crowd’s mentality. We would not be able to separate the two.

The stereotypical scientist or technology purist would say that with enough data we could predict the future. Logic also suggests that if we knew everything and how it works, we could in fact know exactly what will happen, be it the weather or human behavior. But can we ever really know everything and does the mere fact of us trying to recognize and preempt human emotion, change the emotion itself.

While the emergence of emotion recognition technology may be well underway, it will probably be limited to applications in which being wrong is not so frustrating for the user. Identifying emotion during the purchasing process in order to change the process to increase sales, rather than preventing someone from driving because they show signs of anger.

However, when systems develop to the point of relative certainty they could become commonplace in our society. While that may seem a long way off, we must also consider the rate of change that AI brings about on its road to human-like robots – AI with a heart.

Get all the news you need about Smart Buildings with the Memoori newsletter

Which research categories are you interested in?

By signing up you agree to our privacy policy