Practical approach for streaming UI from LLMs

Summary

LLMs have enabled us to solve a new class of problems with more flexibility than ever, but as they are language models, they are inherently text powered, which has led to AI-based UI being incredibly text heavy. As someone who has been creating experiences with web technology my entire life, I’m not satisfied with so much UI being replaced with text. At Vetted, we have been building a shopping research assistant, and shopping is an inherently visual and UI heavy space. Products need to display images and the UI needs to present structured data to allow users to navigate between products and compare them. Over the years we have been experimenting with new ways to incorporate rich UI components into our LLM responses. We’ve done this by supplementing our text payloads with structured product data that can be rendered as product cards and research components, but we weren’t happy with these elements being split out from the core of the text answer. So, I set out to come up with a way to allow the LLM output to incorporate UI components into our markdown output. MDX Enhanced Dynamic Markdown Check out react-markdown-with-mdx. It’s an HOC (Higher-Order Component) wrapper around react-markdown that enhances its markdown processing to support JSX component tags. You can register white-listed React components with it, ensuring only a safe subset of JSX component tags can be rendered. The library comes with an optional validation helper function, mdxComponent which takes your React components with a zod validator for validating the JSX attributes. This enables you to prompt your LLM calls to generate JSX tags that can be consumed safely with a clean and easy integration. Here’s what it looks like in action in our prototype UI at Vetted: The code looks something like this: import ReactMarkdown from "react-markdown" import { withMdx, mdxComponent, type MdxComponents} from "react-markdown-with-mdx" const MdxReactMarkdown = withMdx(ReactMarkdown) interface MdxMarkdownRendererPro...