Object Storage: A Flat Namespace With a Hash Behind It

An object store’s core is smaller than it looks. A key such as photos/2026/cat.jpg is not a path; it is an opaque string containing slashes. The store applies a placement function to it — in practice a hash, so Hash Partitioning and the Modulo Trap and The Ring: Keeping the Mapping Stable When Membership Changes are exactly the mechanism — and gets a placement group, which the cluster map turns into an ordered list of storage nodes. The bytes are then written to those nodes as whole replicas or as erasure-coded fragments.

The consequence of hashing is that there is no directory tree anywhere. photos/2026/ is not a thing that exists; it is a *prefix*, and listing by prefix is served by a separate index over keys, not by walking a structure. That is why an object store has no atomic rename: renaming would mean copying every object to a new key, because the key determines the placement. A "rename a folder" operation on a million objects is a million copies and a million deletes, and every tool that offers it is hiding that fact.

Two properties fall out of this and explain most of the ergonomics. First, objects are immutable: you replace an object wholly, you do not edit its middle. This removes the entire class of concurrent-write problems that Distributed File Systems: Chunks, a Metadata Service, and Where the Copies Go had to define semantics for — there is nothing to interleave. Second, there is no coordinator on the data path at all; the client hashes, or a stateless front end hashes, and the request goes straight to the nodes holding the data. The metadata master that bounded a chunked file system to one machine’s memory simply does not exist, which is why an object store holds trillions of objects without noticing.

The most common object-storage bug is treating the store as one system when it is two: an object store keyed by exact key, and a listing index keyed by prefix. They have different consistency properties, and code that writes an object then lists the prefix to find it is depending on the weaker of the two.

For the object itself, the major providers now offer read-after-write consistency: a successful PUT is immediately visible to a subsequent GET of the same key, including overwrites and deletes. This was not always true — for years, overwrite and delete were eventually consistent, and an enormous amount of production code was written against the weaker model. If you are reading an old design document, or running a self-hosted store, check rather than assume.

For listings, be much more careful. Even where a provider documents strong consistency for listings, a great deal of tooling sits on top of a *cached* or *derived* index — an inventory report, a manifest, a metadata database updated by an event notification. Each of those hops is Eventual Consistency: If Updates Stop, Replicas Converge with its own lag, and the notification delivery is Where You Put the Acknowledgement Decides Everything with its own duplicates. The pattern that survives is: do not discover your own writes by listing. Write the key you are going to read, and read that key.

The eleven-nines durability figure attached to these services is not marketing about disks being good. It is arithmetic over erasure coding across failure domains. An object is split into k data fragments plus m parity fragments and any k of the k+m suffice to reconstruct it, with the fragments placed in separate racks or zones. You then continuously scrub and repair, so the system spends almost all of its time at full redundancy and only briefly at reduced redundancy after a failure. The number falls out of how quickly repair restores the margin, not out of how rarely disks die.

Two honest observations follow. First, the number describes the store losing your bytes, and that is essentially never your incident. Your incident is a lifecycle rule that expired objects you still needed, a bucket policy that let something delete them, or a job that overwrote a good object with a bad one — none of which durability arithmetic covers. Versioning and a separate retention boundary do; see the Cloud domain for how those are configured.

Second, erasure coding trades a small-read penalty for a large space saving: reconstructing an object touches k nodes rather than one, so a small object read has a wider fan-out and inherits Fan Out to 100 and the Component’s Tail Becomes the System’s Median. This is why stores commonly replicate small objects and erasure-code large ones — the same trade that appears every time this domain meets a fan-out.

A flat, immutable, hash-placed namespace removes a lot of problems, and it removes some capabilities you will miss on a specific Tuesday. There is no atomic rename and therefore no atomic "publish this directory" — the standard workaround is to write objects under a temporary prefix and then write one small manifest object whose PUT is the atomic publish point. There is no append and no partial update; changing one byte means rewriting the object, which is why log-structured formats and multi-part uploads exist. There is no cross-object atomicity, which is the same Cross-Partition Operations: Paying for What the Split Took Away problem in different clothing: two objects that must change together cannot.

And there is a per-request cost profile that surprises people who think of it as a disk. Every GET is a network request with a request charge, a first-byte latency of tens of milliseconds, and — if the caller is in another region or leaving the provider — an egress bill. A workload that would be one sequential file read on a local disk becomes ten thousand billed requests. That is not a performance footnote; it changes which algorithms are affordable, which is a large part of why Move the Computation to the Data behaves differently once storage is behind an object API.