Robots can be designed to clean the floor, as long as everything they might encounter in the room has a programmed response. Suddenly introduce an open bottle of juice and the robot is likely to knock it over then smear juice all over the floor, completely unaware that it is making the situation worse. Robots can be designed to answer questions from humans, as long as every question has a programmed response - ask something new and the robot will have very little to say. Robots can even be designed to dance, but try to dance with it and we are likely to get hurt as the robot cannot read and react to our unpredictable human dance moves. 

This is the stage robotics is at today, but recent developments in artificial intelligence (AI) may now open the door to robots that can whirl you round the dancefloor, charm you with natural conversation, and even clean up the unpredictable mess afterwards. Artificial Intelligence is the key to unlocking the true potential of robots in real-world environments, such as our commercial buildings, where numerous benefits await clear use-cases and capable technologies. 

“For a very long time, perhaps 15 years, the potential for AI to help transform buildings performance to make them more adaptive, efficient, and less costly to manage and maintain has been discussed and explored in research papers,” explains our new AI & Machine Learning in Smart Commercial Buildings report. “What was missing from the picture until recently though, were credible use-cases and case-studies of commercially available solutions that offered tangible value to building owners and occupiers via AI or machine learning methods.”

This all may change with the new release of Droidlet, an open-sourced AI platform from the tech giant Facebook. Droidlet is a modular, heterogeneous embodied agent architecture, and a platform for building embodied agents, that sits at the intersection of natural language processing, computer vision, and robotics. It simplifies the integration of a wide range of state-of-the-art machine learning algorithms in embodied systems and robotics to facilitate the rapid prototyping required to bring intelligent robots into real-world spaces, such as buildings.

“There is much more work to do — both in AI and in hardware engineering — before we will have robots that are even close to what we imagine in books, movies, and TV shows. But with droidlet, robotics researchers can now take advantage of the significant recent progress across the field of AI and build machines that can effectively respond to complex spoken commands like “pick up the blue tube next to the fuzzy chair that Bob is sitting in,” says Anurag Pratik et al, in their 2021 research paper. “We look forward to seeing how the research community uses droidlet to advance this important field.”

Rather than seeing robots as a single entity, Droidlet sees a collection of components, some heuristic and some learned. By creating an open-sourced platform for everyone from corporations to hobbyists to experiment with, they can drive collective progress much quicker, as each finds new problems and solutions to real world navigation. Using this representation of the world around them, robots may soon be able to leverage pre-trained neural semantic parsers that convert natural language to programs and, therefore, respond to natural language commands like “move the bottle of orange juice before you clean the floor”.

“Significant advances have been made in the ability of natural language processing (NLP) algorithms to interpret the meaning of not only what we say, but the nuance and context of what a question might mean, in short, they are becoming better at understanding and resolving user intent,” explains our in-depth AI report on commercial buildings. “For solutions with a learning capability each interaction and additional piece of data collected by digital assistants and chatbots can improve their ability to understand a user's intent, making interactions more frictionless over time.”

The ability to understand natural language, comprehend new objects, and react appropriately to unknown elements of the world around them is the next step in our path towards advanced robotics. In our future buildings, such robots will go beyond cleaning and dancing to bring about a wide range of new applications in physical security, environmental control, wayfinding, customer service, and much more. However, for us to understand that next step, we must work together through an open-source approach that feeds off the collective intelligence of a broad range of researchers. Facebook’s Droidlet platform maybe that approach.

“Building intelligent machines that work in the real world is a fundamental scientific goal in AI. Facebook AI is helping the community by releasing not only droidlet and Habitat, but also other, independent research projects such as DD-PPO, our advanced point-goal navigation algorithm; SoundSpaces, our audio-visual platform for embodied AI; and our simple PyRobot framework,” explains Pratik et al. “The path is long to building robots with capabilities that approach those of people, but we believe that by sharing our research work with the AI community, all of us will get there faster.”