Netflix Revamps Tudum's CQRS Architecture with Raw Hollow In-Memory Object Store

Summary

Netflix replaced a CQRS implementation using Kafka and Cassandra with a new solution leveraging RAW Hollow, an in-memory object store developed internally. Revamped architecture of Tudum offers much faster content preview during the editorial process and faster page rendering for visitors. Netflix launched Tudum, its official fan website, in late 2021, to provide a destination for Netflix users interested in additional content associated with the company’s shows. The architecture of the website was initially based on the Command Query Responsibility Segregation (CQRS) pattern to optimize read performance for serving content. The write part of the platform was built around the 3rd-party CMS product, and had a dedicated ingestion service for handling content update events, delivered via a webhook. The ingestion service was responsible for converting CMS data into read-optimized page content by applying templates, as well as data validations and transformations. Read-optimized data would then be published to a dedicated Kafka topic. Initial Tudum data architecture (Source: Netflix Engineering Blog) On the query side, the page data service was responsible for consuming messages for the Kafka topic and storing the data in the Cassandra query database. The service utilized a near cache to improve performance, providing stored page data to the page construction service and other internal services. The initial architecture benefited from the decoupling of read and write paths, allowing for independent scaling. However, due to the caching refresh cycle, CMS updates were taking many seconds to show up on the website. The issue made it problematic for content editors to preview their modifications and got progressively worse as the amount of content grew, resulting in delays lasting tens of seconds. Eugene Yemelyanau, technology evangelist, and Jake Grice, staff engineer at Netflix, describe the cause for delays in retrieving content for displaying due to caching: Regardless o...