Toyota Research Institute robots leave home • businessroundups.org

“I think that I probably as guilty as anyone else,” admits Max Bajracharya, senior vice president of robotics at the Toyota Research Institute (TRI). “It’s like our GPUs are better now. Oh, we have machine learning and now you know we can do this. Oh, okay, maybe that was harder than we thought.

Ambition is of course an important aspect of this work. But there is also a great, unavoidable tradition of re-learning from mistakes. The smartest people in the room can tell you a thousand times why a specific problem hasn’t been solved, but it’s still easy to convince yourself that this time – with the right people and the right tools – it’s just going to be different .

In the case of TRI’s in-house robot team, the impossible task is the house. The lack of success in the category is not due to a lack of trying. Generations of roboticists agree that there are plenty of problems waiting to be automated, but so far the successes have been limited. Outside of the robot vacuum, there has been little breakthrough.

TRI’s robotics team has long made the home a primary focus. This is due in no small part to the choice of elderly care as the ‘northern star’, for the same reason that Japanese companies are so far ahead of the rest of the world in this category. Japan has the world’s highest percentage of citizens over the age of 65 – behind only Monaco, a microstate in Western Europe with a population of less than 40,000.

In a world where our health and well-being are so closely linked to our ability to work, it is an issue bordering on crisis. It’s the kind of thing that assistant professors get from Yale The New York Times headlines for suggesting mass suicide. That is clearly the most sensational of all ‘solutions’, but it is still a problem in search of a meaningful solution. As such, many Japanese roboticists have turned to robotics and automation to address issues such as home care, food preparation, and even loneliness.

Early professionally produced videos showed robotics in the home, performing complex tasks such as cooking and cleaning a wide variety of surfaces. When TRI opened the doors to its labs in South Bay this week to select the press to show off a range of its various projects, it was notably lacking the home element. Bajracharya showed some robots. The first was a custom turnkey arm that moved boxes from a stack to nearby conveyors, in a demo designed for truck unloading – one of the more difficult tasks to automate in an industrial warehouse environment.

The second was a wheel robot that goes shopping. Unlike the warehouse example, which had standard parts with a custom gripper, this system was largely designed in-house out of necessity. The robot is sent out to pick different products on the shelf based on barcodes and general location. The system can extend to the top shelf to find items before determining the best method for grabbing the wide range of different objects and dropping them into the basket. The system is an outgrowth of the team’s pivot away from house-specific robots.

To the side of both robots is a mock kitchen, with a portal system configured up to the top of the walls. A quasi-humanoid robot hangs down, motionless and lifeless. It’s not acknowledged during the demos, but the system will be familiar to anyone who’s watched the team’s early concept videos.

“The house is so hard,” says Bajracharya. “We choose challenging tasks because they are difficult. The problem with the house isn’t that it was too hard. It was that it was too difficult to measure the progress we were making. We have tried many things. We tried to mess it up procedurally. We would put flour and rice on the tables and we would try to clean them up. We would put things all over the house to make the robot tidy. We were betting on Airbnbs to see how well we did, but the problem is we couldn’t get the same house every time. But if we did, we’d fit too much into that house.

The move to the grocery store was an attempt to address a more structured environment while addressing a pressing issue for the older community. In testing the product, the team moved from Airbnbs to a local mom-and-pop grocery store.

“To be perfectly honest, the challenge issue doesn’t really matter,” Bajracharya explains. “The DARPA Robotics Challenges, they were just made-up tasks that were difficult. The same goes for our challenge tasks. We love the house because it is representative of where we ultimately want to help people in the home. But that doesn’t have to be the house. The grocery market is a very good representation because it has an enormous diversity.”

In this case, some of the lessons presented in this setting translate to Toyota’s broader needs.

What constitutes progress for such a team is a difficult question to answer. However, it’s certainly a topic that’s held in high esteem as major corporations have begun to cut roles in longtail research projects that have yet to deliver tangible monetization results. When I put the question to Gill Pratt yesterday, the TRI boss told me:

Toyota is a company that has worked very hard not to let employment follow the business cycle. The automotive industry is one that has peaks and valleys all the time. You may know that it’s Toyota’s history to try not to fire people when times are tough, but instead go through some stuff. One is shared sacrifice, where people take the cause. The second is to use that time to invest in maintenance, planning and training to help people get an education.

Toyota is known in the industry for its “no layoffs” policy. It’s certainly an admirable goal, especially as companies like Google and Amazon are in the midst of tens of thousands of layoffs. But when goals are more abstract, as is the case with TRI and fellow research departments, how does a company measure relevant milestones?

“We made progress at home, but not as quickly and not as clearly as when we go to the supermarket,” explains the director. “When we move to the grocery store, it really becomes very clear how well you’re doing and what the real problems are in your system. And then you can really focus on solving those problems. As we toured both Toyota’s logistics and manufacturing facilities, we saw all of these opportunities where they’re essentially the grocery shopping challenge, except a little bit different. Now, instead of the parts being groceries, the parts are all the parts in a distribution center.”

As is the nature of research projects, Bajracharya adds, the beneficial results are sometimes unexpected: “The projects are still looking at how we ultimately empower people at home. But over time, as we pick out these challenge tasks and drip out things that apply to these other areas, we use these short-term milestones to show the progress in the research that we’re making.

The path to realizing such breakthroughs can also be vague at times.

“I think we understand the landscape a little bit now,” Bajracharya. “Maybe I was naive in the beginning thinking that we just have to find that person to whom we are going to transfer the technology to a third party or someone inside Toyota. But I think what we’ve learned is that whatever it is – be it a business unit, or a company, or like a startup or a unit within Toyota – they don’t seem to exist.”

Rolling out startups — similar to what Alphabet has done with its X labs — is certainly on the table, even if it’s not likely to be the main path to production. What form that path will ultimately take, however, remains unclear. Although robotics as a category is much more viable today than when TRI was founded in 2017.

“In the last five years, I feel we’ve made enough progress on that very challenging problem that we’re now starting to see it changing in these real-world applications,” says Bajracharya. “We have deliberately shifted. We’re still 80% on the state of the art with research, but we’ve now allocated maybe 20% of our resources to figuring out if that research might be as good as we think it is and if it can be applied to real-world applications. We could fail. We may realize we thought we’d made some interesting breakthroughs, but it’s nowhere near reliable or fast enough. But we put 20% of our effort into trying.”