An Intro to DeepSeek's Distributed File System

Summary

Series What is 3FS? 3FS (Fire-Flyer File SystemGeez, what a tongue twister) is a distributed filesystem released by DeepSeek during their open source release week. This blog post will dive into what distributed file systems are and how 3FS operates, starting with some background. What is a distributed filesystem? Distributed filesystems trick applications into thinking they’re talking to a regular local filesystem. This abstraction is incredibly powerful: a file that’s actually fragmented across 10 different machines appears as a simple file path like /3fs/stage/notes.txt Using the distributed filesystem is no different than local filesystem In the image above, I create the same folder and file on a local and distributed filesystem by running mkdir and cat. The commands are exactly the same. With a distributed filesystem, all of those details are abstracted away from the user, who can simply work with the files without worrying about how many machines, network calls or disks are involved behind the scene. Why use a distributed filesystem? Distributed filesystems provide two main advantages over local storage – they can serve massive amounts of data (up to petabytes) and provide high throughput that exceed the capabilities of a single machine. They offer fault tolerance (the system keeps running if one machine goes down) and redundancy (if data gets corrupted on one node, other nodes have original copies). Distributed filesystems are used in many practical applications: A deep dive into 3FS So, how does 3FS work? At its core, 3FS consists of four primary node types: Components involved in 3FS The components serve distinct purposes: Meta – manage the metadata: file locations, properties, paths, etc. Mgmtd – management server controls the cluster configuration: where are other nodes, which nodes are alive, and replication factor Think of it as a router that knows every node’s address and can help nodes find each otherA similar analogy is the centralized server used in ...