Show HN: Building WebSocket in Apache Iggy with Io_uring and Completion Based IO

Summary

Introduction​ In our 0.5.0 release blog post, we announced that work was underway on a complete rewrite of Apache Iggy's core architecture using io_uring with a thread-per-core, shared nothing design. This architectural redesign aims to further improve performance, reduce tail latecies and lower resource usage by leveraging io_uring's completion based I/O model. As part of this rewrite, we migrated from Tokio to compio, a completion-based async runtime that allows us to better utilize io_uring capabilities. However, it also presents different challenges when integrating with the wider Rust ecosystem. We came across one such challenge when we needed to add WebSocket support to Iggy server. WebSockets are useful for browser clients and streaming dashboards. The Rust ecosystem has excellent WebSocket libraries like tungstenite and tokio-tungstenite but they are made when poll-based IO was the dominanat IO paradigm. They expect shared buffers and readiness-based I/O, fundamentally incompatible with compio's completion-based model that requires owned buffers. Here we describe our journey of building compio-ws, a WebSocket implementation for the compio async runtime, and the engineering challenges we faced bridging two fundamentally different I/O models and it finally lead to us contributing to compio. Understanding the architectural divide (poll vs completion)​ Let's see why poll-based libraries can't easily work with completion based runtimes by examining the traits of compio, tungstenite. pub trait Read { fn read(&mut self, buf: &mut [u8]) -> Result<usize>; }pub trait AsyncRead { async fn read<B: IoBufMut>(&mut self, buf: B) -> BufResult<usize, B>; } The Read trait assumes you borrow a buffer. Compio's AsyncRead needs to take ownership of the buffer, submit it to the kernel, and return it later. pub struct WebSocket<Stream> { socket: Stream, }impl<Stream: Read + Write> WebSocket<Stream> { pub fn read(&mut self) -> Result<Message> { }} There's no way to make this work d...