I Matched Wits with Microsoft's Chatbots

Senior Editor
Updated

SAN FRANCISCO — Forget about going to the dogs. Microsoft's Build conference has gone to the bots.
The software giant has made interactive bots a key part of the changes it's making to Cortana, its personal assistant. In Microsoft's vision, Cortana will be able to talk to third-party bots to schedule appoints, gather information and perform other tasks. You'll even be able to carry on your own conversations with these chatbots, which Microsoft hopes will be built using the Bot Framework the company introduced at its Build developers conference this week.

MORE: 5 Key Takeaways from Microsoft Build

It's all part of a strategy outlined by Microsoft CEO Satya Nadella that he describes as "conversations as a platform," in which natural language finds its way into how we interact with applications — and in the case of these soon-to-be omnipresent bots, how they talk back to us.

It sounds intriguing in theory. When Skype group program manager Lilian Rincon demonstrated how a chatbot-savvy Skype could line up hotel reservations and arrange for package deliveries, that's an appealing example of how technology can simplify our lives. In practice, though, it doesn't always work that way, as anyone who's ever found himself shouting at an automated voice recognition program on the other end of the phone. (Not that I would ever do such a thing, of course.)

Microsoft's made a handful of bots available, which I downloaded and added a contacts to Skype on my iPhone. (You can also interact with Microsoft's bots on Skype for Windows Desktop, Android and the iPad.) Talking with these chatbots drove home the potential of Microsoft's bot-based vision as well as some of the pitfalls.

I spent most of the past two days talking to Build Bot, a chatbot aimed at helping me find my way around Microsoft's conference this week. With my very first question, Build Bot let me know that our relationship was going to be a bit more personal than the average person-to-chatbot interaction.

Build Bot didn't care for my follow-up question though, in which I asked what the topic of the keynote would be, flashing the kind of sass one doesn't normally associate with artificial intelligence.
And yes, I realize how it looks to have a grown man respond to a chatbot with a sarcastic emoji. If it's any consolation, that caused Build Bot to stop responding to me for a brief period.

With the caveat that Build Bot is designed for a very specific purpose — answering questions about sessions at a developer conference — I found the bot demanded very precise questioning. Asking about sessions on bots produced a request that I refine my question; when I did, I got the detailed response I was looking for. Asking where I could see a HoloLens demo, though, yielded Build Bot's pat response about maybe searching the Internet. That's not precisely the natural conversation Microsoft is touting, though it's very early days still.
I had better success with the other bots Microsoft has made available. Bing News Bot lets you search for headline news just by typing in a search term: Typing "President Obama" or "Oakland Athletics" yielded a trio of headlines about either topic. Bing Image Bot searches the Web for images, with a particularly emphasis on memes and GIFs; a search for "kitten memes" produced exactly what you might expect. Bing Music Bot looks for music and videos. I found it was skilled at finding lyrics though when I asked a question about Adele's version of "Skyfall," the bot thought I was trying to find the movie and then wanted to chat about the singer's rendition of "Hello."
CaptionBot proved the most temperamental of all Microsoft's bots. This bot boasts that it can understand the content of any image and even take a stab at describing it to you. Its success at determining what was happening in a series of photos I uploaded could charitably be called mixed. CaptionBot figured out that my daughter was chopping mushrooms, but identified her as a "woman" rather than as a 5-year-old kid.

A bronze statue of baseball great Carlton Fisk flummoxed CaptionBot, which concluded the dark shadows around his eyes must be sunglasses. Chatbot's attempt to analyze the statue's expression was close to on the mark, though.
CaptionBot picked out elements in this photo shot on the San Francisco Bay's shore, but was unable to put them in context, concluding that it was looking at a "boat that is sitting in the grass."
And another photo of my daughter holding her violin made CaptionBot throw up its virtual hands.
It's unfair to judge any bot just a day after its launch, particularly one like CaptionBot that's designed to get smarter the more people use it. And Microsoft is pushing out bots like these and working with developers to create even more to figure out the best approach to building more conversational bots. "We want to be early and get out there to learn with developers, what's the most effective [types of bots]," said Lili Cheng, a Microsoft Research distinguished engineer.

So Microsoft would be the first to concede that, like CaptionBot, it's got plenty of learning left to do.