Filespathfiledirectorymetadatapermissions

Files, Paths and Names

A filename is a directory entry that points at a file object; the object has metadata, permissions and blocks, and it can outlive — or have several — names.

ConceptualUnix-styleWindows
Interview question
Progress

The problem

Your deploy script deletes app.log while the server still has it open. On Linux the server keeps writing and the disk keeps filling, but ls shows nothing. Where did the bytes go — and what, exactly, did rm remove?

A name is not a file

Unix-style

The thing you call a file is two things. The file object is metadata (type, size, owner, permission bits, timestamps) plus a map from byte offsets to blocks on storage. The name is an entry in a directory: a string paired with a reference to that object. On Unix-style systems the reference is an inode number (Inodes); on NTFS it is a record number in the Master File Table. Either way the directory entry is the pointer, not the thing.

This split explains behaviour that looks paradoxical if you think of a file as "the bytes at this path". A hard link is a second directory entry pointing at the same object: ln a b gives the object two names, stat shows a link count of 2, and editing through either name edits the one object. A symbolic link is a different object whose content is a path — it can dangle, cross file systems, and point at a directory.

A directory is itself a file object whose content is a list of (name, reference) pairs. Renaming moves an entry between directories; it does not touch the object or its blocks, which is why mv within one file system is instant for a 100 GB file and why mv across file systems is a copy followed by a delete.

  • One object, many names: hard links. One name, one path-shaped object: a symlink.
  • The object never stores its own name. find -inum is how you go backwards.
  • Windows/NTFS supports hard links too (mklink /H), but most Windows tooling assumes one name per file.

What happens when you delete an open file

Unix-style

On Unix, rm calls unlink(2): it removes the directory entry and decrements the object’s link count. The object’s storage is reclaimed only when the link count reaches zero and no process holds it open. A server that opened app.log still has a descriptor (File Descriptors) referring to the object, so the object lives on with no name. It keeps growing; du cannot see it; df can. The classic symptom is a disk at 100% with nothing large in any directory.

lsof +L1 (or lsof | grep deleted) lists exactly these nameless files, and /proc/<pid>/fd/<n> still lets you read the content. The fix is to make the process close or reopen the file (SIGHUP for most log-rotation-aware daemons, or copytruncate in logrotate) — deleting the name again does nothing. The same mechanism is used deliberately: open then unlink gives a temporary file that vanishes automatically when the process exits, which is what tmpfile(3) and O_TMPFILE do.

Windows differs, and the difference matters if you ship cross-platform. By default a file opened without FILE_SHARE_DELETE cannot be deleted or renamed while open — you get ERROR_SHARING_VIOLATION, the "file is in use" dialog. Even with the share flag, the name persists in a deleted-pending state until the last handle closes, so creating a new file with the same name may fail. Replace-while-running (atomic rename over a running binary or a config being read) is a Unix idiom that does not port unchanged.

A deleted-but-open log on Linux
$ df -h /var/log
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p2   98G   97G  1.1G  99% /
$ du -sh /var/log
412M    /var/log
$ lsof +L1 | head -3
COMMAND   PID USER   FD   TYPE DEVICE   SIZE/OFF NLINK  NODE NAME
server  41822  app    4w   REG  259,2 84213487616     0 918273 /var/log/app.log (deleted)

Absolute, relative, and the current working directory

A path is a list of names separated by /. An absolute path starts at the root directory; a relative path starts at the process’s current working directory, which is per-process state kept by the kernel — not a shell concept, not a global. chdir(2) changes it for the calling process only; a child inherits it at fork and keeps its own copy afterwards. That is why cd must be a shell builtin: a /bin/cd binary would change its own cwd and exit.

The cwd is a reference to a directory object, not a stored string. If the directory is renamed under you, getcwd returns the new path; if it is deleted, getcwd fails with ENOENT and relative opens fail — the familiar "shell: getcwd: cannot access parent directories" after a deploy replaced the directory you were sitting in. Daemons chdir("/") at startup precisely so they never pin a mount point or a deleted directory.

Relative paths are a portability and security trap in code: a service started by systemd, by cron, and by hand has three different cwds. Resolve paths against a known anchor (the executable’s directory, an env var, or openat with a directory descriptor) rather than trusting where the process happened to start.

  • . is the directory itself, .. its parent; at the root, .. is the root again.
  • Per-process: cwd, root directory (chroot), umask. All three are inherited across fork and survive exec.
  • On Windows there is a cwd per drive letter in the C runtime and a single process-wide one in the kernel; UNC paths and \? prefixes have their own rules. Label your path code accordingly.

Path resolution, one component at a time

Unix-style

Opening /var/log/app.log is not one lookup. The kernel starts at the root (or the cwd for a relative path) and for each component in turn: reads the current directory, finds the entry with that name, checks that the process may search the directory (the execute bit on a directory means "traverse", not "run"), follows a symbolic link if the entry is one (restarting resolution from the link’s content; Linux caps the number of links at 40 and returns ELOOP), crosses a mount point if one is stacked there, and moves on. Only the last component is checked for read or write permission.

Each step is a lookup in the dentry cache on Linux, a hash from (parent directory, name) to the resolved object, so a hot path costs tens of nanoseconds per component with no disk access. A cold path — the first ls after boot, or a container with a 12-layer overlay file system — walks directories on storage and can take milliseconds. Deep paths and long PATH variables cost real time on cold caches: a shell searching 20 directories for node does 20 failed lookups per command.

Permission bits are checked against the process’s effective user and group: three triplets rwx for owner, group, other, plus setuid, setgid and sticky bits. chmod 644 is rw-r--r--. Removing a file requires write permission on the directory, not on the file — a read-only file in a writable directory is deletable, which surprises everyone once. Windows uses ACLs on each object instead of mode bits; the traversal-permission concept exists but is checked differently.

Resolving /var/log/app.log for open(…, O_WRONLY)
  1. Root directory `/`process root (chroot) or the global root
    ↓
  2. Entry `var` → directorysearch permission (x) on `/` checked; dentry cache hit
    ↓
  3. Entry `log` → directorysearch permission on `/var`; mount point? then switch to the mounted file system’s root
    ↓
  4. Entry `app.log` → file objectwrite permission on the object itself; symlink? restart from its target
    ↓
  5. Open file description + descriptoroffset 0 (or end with O_APPEND), flags recorded, fd number returned

Key points

  • A filename is a directory entry pointing at a file object. The object has the metadata and the blocks; it does not know its own name.
  • Hard links are extra names for one object; symlinks are objects whose content is a path.
  • Unix unlink removes a name. Storage is freed when the link count is zero and no descriptor is open — so deleted-but-open files silently eat disk.
  • Windows refuses to delete or rename an open file unless it was opened with FILE_SHARE_DELETE; the idioms do not port unchanged.
  • The current working directory is per-process kernel state, inherited at fork, referenced by object not by string.
  • Resolution walks the path component by component, checking search permission on every directory and only read/write on the final object.

Why does this exist?

Mechanisms are answers to constraints. Open each question before reading the answer.

▸Why separate the name from the object at all?

So that renaming is a metadata edit, not a copy; so one object can have several names; and so an open file stays valid while its name changes — a process reads a consistent object even as the directory around it is rewritten.

▸Why is the execute bit on a directory called "search"?

Because looking up a name inside a directory is a distinct operation from listing it (read). --x on a directory lets you reach files you already know the names of without letting you enumerate them — the mechanism behind shared drop-boxes and /home protection.

▸Why does every process carry its own cwd?

Because two processes started from different places must resolve ./config.yml differently, and because the OS cannot know which of a process’s threads "means" a relative path. Making it process state is the smallest scope that still allows cd to work in a shell.

How it fails

What the failure looks like from inside real software.

  • Disk full, nothing large in any directory: a rotated or deleted log is still open by a running process (lsof +L1).
  • Cron job or systemd unit fails with "no such file" while the same command works in your shell: relative path resolved against a different cwd.
  • "getcwd: cannot access parent directories" after a deploy replaced the directory a shell was sitting in — the old directory object is deleted, the shell still holds it.
  • Cross-platform tool that atomically replaces a file with rename works on Linux and throws EPERM/sharing violation on Windows because the target is open.
  • A backup or sync tool copies the same 20 GB object three times because it treats three hard links as three files (or, inversely, restores them as three independent copies).
  • Permission denied on open although the file is rw-rw-rw-: a directory in the path lacks the search bit for that user.
File Descriptors →