How we developed the first artificial intelligence tools for Chrome

From an engineering standpoint, implementing LLM technology in Chrome was a challenge. "It's a new skill set," says Adriana. "We needed to not only learn how the technology works, but how to turn it into a product that people can use. Traditional browser functions work the same way every time you run them. If a function has the same input, it will produce the same output." When Adriana and her team write code for a new Chrome feature, they also write tests to verify that it works as expected. "If it passes the tests, you can be sure it works," she says.

Things are not so simple with features that use generative AI. Large language models recognize and generate text or images, and they need to be able to adapt to many types of user input. "We take a base model and train it to do what we want it to do for our use cases, and then evaluate how it performs in different user scenarios," says Adriana. It takes a deeper human evaluation to determine if it works. "It's not just a binary option of 'works' or 'doesn't work,'" Adriana says. "We look at it and think, 'Is the tone right? Is this length normal? Is this the level of specificity we're looking for?' It's a whole different process."

One training scenario that Adriana found particularly interesting was how the AI tab organizer uses emoji. "I really like how people use emoji to label tab groups," she says. "Seeing the emoji makes it easier to understand the theme of that tab group when you're scanning your tabs." The Chrome team wanted the new auto tab organizer to have emoji options for users, but also didn't want it to pick inappropriate options. For example, if you're planning to celebrate a celebration of life, Adriana explains, they didn't want Chrome to show you a skull and crossbones. So with the help of Google's emoji team, they decided to determine which tab group categories are safe for widespread use. "Travel, animals, places, nature - these are all great uses for emoji, so we know that the auto tab organizer has a good chance of doing well using only these categories," she says.

The Chrome team also wanted to make sure that people could use the new artificial intelligence features without having to understand the technology behind them. So they designed Help me write so that it gathers context from the web page you're on and uses it later. "It can understand that you want to write a restaurant review and adjust to that, rather than helping you fill out a form or selling you something," says Adriana. Similarly, when creating the AI themes tool, they initially thought users would be able to write their own visual theme prompts. "We realized that it's actually quite difficult to come up with a tooltip for that," Adriana says. Instead, they settled on a drop-down approach where you choose a theme - like Aurora Borealis or Rainbow - and then can use other drop-down prompts to add styling details and choose a color scheme. "We want people to be able to customize it, but at the same time give narrower options that will give good results," says Adriana.