Buckeyes excel in Amazon Alexa Prize TaskBot Challenge

A team of engineering graduate students from The Ohio State University emerged as one of three top performers in the inaugural Alexa Prize TaskBot Challenge, Amazon’s global university competition to advance conversational artificial intelligence (AI).

Launched in March 2021, the TaskBot Challenge involved ten university teams developing bots that could help customers complete cooking or do-it-yourself home improvement tasks that required multiple steps and decisions — and adapt those instructions based on the resources and tools available to the customer. If, for example, a customer ran out of an ingredient halfway through a recipe or didn’t have a specific tool for a DIY project, the taskbot had to adjust the plan and suggest possible solutions.

Advised by Computer Science and Engineering Associate Professor Huan Sun, the team of Buckeye engineers, earned third place, the best performance among competitors from the U.S. The University of Glasgow and Portugal’s NOVA School of Science and Technology finished first and second, respectively. The top performers earned prize money to be divided among the team members: $500,000 for first place, $100,000 for second and $50,000 for third.

Customers interacted with the taskbots by saying, “Alexa, let’s work together.” That prompt initiated an interaction with one of 10 taskbots. After the interaction ended, customers were asked to rate — on a scale from 1 to 5 — how helpful that taskbot was with the task.

The Ohio State task-oriented collaborative planning dialogue system — or TacoBot for short — was designed with a user-centered principle and aspires to deliver a collaborative and accessible dialogue experience. Towards that end, it is equipped with accurate language understanding, flexible dialogue management and engaging response generation.

The TaskBot Challenge is the first conversational AI challenge to incorporate multimodal customer experiences. In addition to receiving verbal instructions, customers were also presented with images or diagrams to guide them through the task.

Success required the teams to address many difficult AI challenges, from knowledge representation and inference, and commonsense and causal reasoning, to language understanding and generation, requiring fusion of multiple AI techniques.

In May 2021, the Ohio State team was selected as one of 10 entrants to participate in the challenge, receiving a research grant of $250,000 to support 12 months’ work. They also received four Amazon Alexa devices, free access to Amazon Web Services, and support from members of the Amazon Alexa team. This past April, they qualified as one of five teams to compete in the finals.

Ohio State team members said they learned about the importance of user experience in developing real-life dialogue systems.

“User engagement is a key aspect to consider in developing taskbots,” Professor Sun said. “Instead of only focusing on the task, users seem to have an interest in having a chit-chat with the bot from time to time. How to naturally transition between chit-chat and task-oriented turns requires prolonged efforts.”

“We believe that taskbots can potentially make technologies more accessible,” said student co-lead Ron Chen. “Instead of browsing through many web pages, users can have a multi-modal conversation with TaskBots to find instructions for their ongoing tasks and answers to their questions.”

All 10 teams participating in the competition have published research papers detailing their work.

“This challenge was motivated by our north star that Alexa will keep inventing next-generation conversational AI experiences that address our customers’ changing needs,” said Yoelle Maarek, vice president of research and science, Alexa Shopping. “We were delighted with the high level of engagement from the academic community, and by the advances the teams made against the research directions established for the challenge.”

Another group of Ohio State computer science and engineering students is competing in the Alexa Prize SimBot Challenge, in which teams will develop next-generation virtual assistants for completing real-world tasks by continuously learning and gaining commonsense reasoning ability. The SimBot finals will be scheduled between January 30 and March 27, 2023.