The Right Way to Embed an LLM in a Group Chat

Summary

In a typical group chat, adding an AI assistant may be occasionally useful at best, and downright annoying at worst. If a user has a question for an AI related to the discussion, it isn’t much more effort to switch to a different app like ChatGPT and, if needed, copy the answer back to the group chat. Having AI embedded into the chat might save a step or two, but at the cost of potentially derailing the existing conversation.AI gets a bit more useful when you give it access to embedded tools. If your chat supports polls, for example, you could ask the AI to look up potential restaurants for a weekend hangout and create a poll for the group to vote on. If you relied on an external app like ChatGPT to do this research, you’d have to manually set up the poll and copy in the restaurant context, so you’ve just saved yourself a lot of time.Where AI-assisted group chat becomes incredibly useful is in the context of a purpose-built chat embedded in a larger application. Our use case is a travel planning app, TripJam, where travel groups collaborate on their itinerary, to-dos, budget, and more.TJ searches Google for restaurant ideas and adds them as poll options for the group to vote on.What’s the Problem?In an app like ChatGPT, the basic intent of each message is clear: the user is addressing the AI and expects a response. In the context of a group chat, however, an AI assistant’s job is much more difficult: the average message is more likely than not directed at another user, so the AI’s goal should be to respond only when needed and otherwise be unobtrusive. In an app like TripJam, the assistant is only useful to the extent that it can perform the action asked of it. This action may be as simple as summarizing a potential vacation location, or a complex multi-step process such a looking up top restaurants in an area, moving the map to display them next to the other itinerary locations, and adding them to the itinerary itself. Interaction with the assistant should feel lik...