A configuration daemon opens /srv/app/config, validates its contents, and keeps the returned descriptor. An administrator writes a replacement file and atomically renames it over config. New calls to open("/srv/app/config", ...) reach the replacement; the daemon’s existing descriptor still reaches the old object. If the old name was its last directory entry, the old object has no pathname at all, yet the daemon can continue reading it.
The apparent contradiction disappears when a pathname and an open file are modeled as different references. A pathname is a request to traverse a mutable namespace. A descriptor is a process-local handle to an already opened description. Treating both as “the file” causes stale-read bugs, authority races, and incorrect durability claims.
Five objects hidden behind one integer
Assume a local POSIX-like filesystem, one mount namespace, no I/O error, and no concurrent namespace mutation until a trace explicitly introduces one. The model contains five distinct objects:
| Object | Identity | Mutable state relevant here | Typical lifetime |
|---|---|---|---|
| pathname string | bytes supplied by a caller | components may be interpreted relative to a directory | one operation |
| directory entry | pair (name, object-id) inside a directory | name and target mapping | until unlink or rename |
| filesystem object | inode-like identity | type, size, link count, permissions, block map | while linked or referenced |
| open file description | kernel record for an open instance | current offset and file-status flags | while any descriptor refers to it |
| file descriptor | small integer in one process | descriptor flags such as close-on-exec | until close or table replacement |
A symbolic link is itself a filesystem object whose payload is another pathname. A hard link is another directory entry for an existing object; it is not a pointer to another name. A process may have several descriptors for one open file description, and several independently opened descriptions may refer to one object.
The diagram separates two graphs. Directory entries form the namespace graph. Descriptors and open descriptions form the open-reference graph. Unlinking an edge in the first graph does not revoke edges in the second.
Retrieval checkpoint: what does dup copy?
Without consulting the answer, record which state is shared after b = dup(a): pathname, descriptor number, descriptor flags, current offset, file-status flags, and inode identity.
Reveal the state partitionExpandCollapse
a and b are different descriptor numbers in one table. They refer to the same open file description, so they share its current offset and file-status flags; they consequently reach the same inode. Descriptor flags are per descriptor and are not thereby shared. No pathname is stored as the descriptor’s operative identity. If the process calls open a second time instead of dup, the new descriptor normally receives a distinct open file description and therefore a distinct offset.
A response that says “everything is copied” has collapsed three layers: descriptor entry, open description, and inode. A response that gives each descriptor an independent offset has confused dup with a second open.
File descriptors, open descriptions, and namespace invariants
Evaluate each statement regarding POSIX file descriptors, directory entries, and file lifecycles:
Classify each option as correct or incorrect, or leave uncertain options unanswered (“I don't know”) and verify at any time.
Worked trace: shared and independent offsets
File records contains sixteen bytes abcdefghijklmnop. Reads below are successful and no signal interrupts them.
int a = open("records", O_RDONLY); // open description X, offset 0
int b = dup(a); // also X
int c = open("records", O_RDONLY); // open description Y, offset 0
Execute read(a, p, 4), read(b, q, 3), read(c, r, 2), then lseek(a, 1, SEEK_SET) and read(b, s, 3).
| Event | bytes returned | offset of X after event | offset of Y after event |
|---|---|---|---|
| initial state | — | 0 | 0 |
read(a, ..., 4) | abcd | 4 | 0 |
read(b, ..., 3) | efg | 7 | 0 |
read(c, ..., 2) | ab | 7 | 2 |
lseek(a, 1, SEEK_SET) | — | 1 | 2 |
read(b, ..., 3) | bcd | 4 | 2 |
The descriptor used for an operation does not determine the offset object. a and b both mutate X. c reaches the same inode through Y but cannot observe X’s offset changes.
Application checkpoint: inherited descriptors
A parent opens an output file, calls fork, and both processes write fixed-size records through the inherited descriptor without seeking. State the property that descriptor sharing supplies and the property it does not supply.
Reveal the diagnostic answerExpandCollapse
The inherited descriptors refer to one open file description, so they share one current offset. POSIX specifies relevant descriptor operations atomically with respect to the effects on regular files, preventing both writes from independently using the same old offset under the specified interface semantics. This does not impose an application-level order between parent and child records, combine several writes into one transaction, make buffered library I/O safe after fork, or make either record durable. If record order is part of correctness, the processes need an explicit protocol.
Path resolution is a stateful walk
For an absolute pathname, resolution begins at the process’s root directory. For a relative pathname, it begins at the current working directory or at a directory descriptor supplied to an *at interface. For each non-final component, the kernel must:
- verify that the current object is a directory and that search is authorized;
- look up the component in that directory;
- follow a symbolic link when the operation’s rules require it, splicing the link payload into the unresolved suffix; and
- continue from the resulting directory, crossing a mount point if the namespace contains one.
. selects the current directory. .. selects its parent subject to root and mount semantics. The final component is operation-specific: open normally follows a final symbolic link, lstat examines the link itself, and rename renames a final link rather than its target. A create operation resolves the parent and leaves the absent final name for insertion.
Worked trace: a symbolic-link pivot
Assume this initial namespace and a process whose current directory is /work:
/work/input -> symlink payload "../safe/report"
/safe/report -> inode 80
/attacker/report -> inode 99
The process first calls stat("input", ...), validates inode 80’s owner, and later calls open("input", O_RDONLY). Between the calls, an attacker who can modify /work replaces input with a symlink to ../attacker/report. The validation and use are two independent pathname walks:
| Time | operation | resolved object |
|---|---|---|
stat("input") | inode 80 | |
| attacker replaces directory entry | namespace now targets inode 99 | |
open("input") | inode 99 |
No corruption of the kernel’s lookup algorithm occurred. The application incorrectly treated a pathname result as a stable capability. Opening first and validating with fstat binds validation and subsequent I/O to the same open object. When policy requires confinement beneath a trusted directory on Linux, a trusted directory descriptor plus openat2 resolution constraints can also reject symlink, mount, or escape behavior; those flags are Linux interfaces, not portable POSIX guarantees.
Transfer checkpoint: close the authority race
A privileged image service must open a caller-supplied relative name beneath an already opened directory tenant_dir. The caller can concurrently rename entries and create symbolic links. Specify a safe ordering and state what remains unproved.
Reveal one defensible protocolExpandCollapse
Keep tenant_dir as an open directory descriptor acquired from trusted configuration. Resolve relative to that descriptor, not the process-wide current directory. On Linux, use openat2 with a policy-appropriate combination such as RESOLVE_BENEATH or RESOLVE_IN_ROOT, plus restrictions on symbolic links or mount crossings when the service’s contract forbids them. Open with the least required access, then inspect the returned descriptor with fstat and perform I/O through that descriptor.
The protocol binds later I/O to the object actually opened. It does not prove that file contents are benign, that the trusted directory itself was acquired securely, that every filesystem implements identical remote semantics, or that a kernel vulnerability cannot violate the boundary. Listing flags without deriving them from the service’s path policy is not a complete answer.
Links, removal, and replacement
For a regular file, the link count records directory entries that name the object. Open descriptions contribute references but not hard links. Storage may be reclaimed only after both the namespace link count and the relevant in-memory references reach zero.
Worked trace: unlink while open
Initial state: /tmp/job.out -> inode 17, nlink(17)=1, and process A has descriptor 5 open on inode 17.
| Step | namespace | link count | open references | result |
|---|---|---|---|---|
| 0 | job.out -> 17 | 1 | 1 | named and open |
1: unlink("/tmp/job.out") | no entry | 0 | 1 | new lookup fails; fd 5 still works |
| 2: A writes through fd 5 | no entry | 0 | 1 | inode 17 grows but remains unnamed |
| 3: A closes fd 5 | no entry | 0 | 0 | object becomes reclaimable |
Creating a new /tmp/job.out after step 1 creates or links some other object. It does not reattach the old descriptor by name.
rename(old, new) changes namespace edges. Within the scope where the operation is supported, another thread must not observe new absent during replacement: it refers to either the prior target or the renamed object. If new existed and its last link is removed, open references to that displaced object still keep it alive. Cross-filesystem rename may fail with EXDEV because one atomic metadata transaction cannot generally span unrelated implementations.
Boundary. Namespace atomicity is not storage durability. A successful
renameestablishes an allowed concurrent-observation result. It does not by itself prove which names or file contents survive power loss.
Application checkpoint: enumerate legal observations
old names inode A and new names inode B. Process R repeatedly resolves new while process W executes successful rename(old, new). No other namespace changes occur. Which inode identities may R obtain? What happens to an already open descriptor for B?
Reveal the observation setExpandCollapse
Each completed resolution of new may obtain B before the rename linearization point or A after it. The specified replacement does not expose an interval in which new is missing. A descriptor opened on B before replacement continues to refer to B; B’s content is reclaimed only after its last link and open reference are gone. An answer claiming that R’s existing descriptor switches from B to A has made descriptors resolve names repeatedly. An answer claiming that the result necessarily survives a crash has imported a durability guarantee that the premise does not provide.
Deriving a crash-aware replacement protocol
Consider a local filesystem whose documentation states that fsync(file) makes the file’s required data and metadata persistent and fsync(directory) persists directory-entry updates. Under those explicit assumptions, an application replacing config can seek either of two recovery states: the complete old object or the complete new object.
1. create a temporary file in the target directory without reusing an attacker-controlled name
2. write the complete new representation
3. fsync the temporary file and check the result
4. rename the temporary name over config and check the result
5. fsync the containing directory and check the result
Step 3 prevents a durable new name from exposing unwritten or stale content under the assumed interface. Step 4 provides atomic namespace replacement to concurrent readers. Step 5 requests persistence of the namespace update. Every failure path must preserve the old file or report that recovery is uncertain; ignoring an fsync error destroys the protocol’s evidence.
This recipe is not universal folklore. Filesystem, device-cache, network, and platform semantics can narrow or alter it. It does not make several files into one atomic transaction. It also does not update long-lived reader descriptors; readers that require fresh configuration must reopen or use an explicit version-notification protocol.
Deep Practice
Solve these problems to test and consolidate your understanding. Complete worked solutions and diagnostic hints are available for each problem.
Exercise 6.1.1: Tracing Shared versus Independent File Offsets Across dup and open
A file log.txt contains 20 bytes 0123456789abcdefghij. A process executes:
int fd1 = open("log.txt", O_RDONLY);
int fd2 = dup(fd1);
int fd3 = open("log.txt", O_RDONLY);
char buf[5];
read(fd1, buf, 4); // Op A
read(fd2, buf, 4); // Op B
read(fd3, buf, 4); // Op C- State the file offset for
fd1,fd2, andfd3after Op A. - State the byte string read into
bufby Op B and Op C. - Draw the relationship between descriptor table entries, open file descriptions, and the in-memory inode.
HintExpandCollapse
dup shares the open file description and offset; separate open calls create new open file descriptions.
Worked solutionExpandCollapse
- Offsets after Op A:
fd1andfd2point to the identical open file description (OFD 1). Whenread(fd1, buf, 4)consumes 4 bytes, OFD 1’s offset advances from 0 to 4. Therefore, bothfd1andfd2now observe offset 4.fd3refers to independent OFD 2, whose offset remains 0. - Read results:
- Op B (
read(fd2, buf, 4)): Reads 4 bytes from OFD 1 starting at offset 4, returning"4567". OFD 1’s offset advances to 8. - Op C (
read(fd3, buf, 4)): Reads 4 bytes from OFD 2 starting at its initial offset 0, returning"0123". OFD 2’s offset advances to 4.
- Op B (
- Architecture graph:
fd1(slot in proc table) Open File Description 1 (offset = 8) Inode (log.txt).fd2(slot in proc table) Open File Description 1 (offset = 8).fd3(slot in proc table) Open File Description 2 (offset = 4) Inode (log.txt).
Exercise 6.1.2: State Transitions for Unlinking Open Files with Active Descriptors
Process opens /var/run/worker.pid with descriptor 3. While runs, Process invokes unlink("/var/run/worker.pid").
- State the inode link count (
i_nlink) and in-core reference count (i_count) before and after theunlink. - Can continue to read and write through descriptor 3 after the
unlinksucceeds? - When are the file’s data blocks and inode metadata physically deallocated by the filesystem?
HintExpandCollapse
The directory link count drops to 0, but the kernel in-core reference count remains 1 while open.
Worked solutionExpandCollapse
- Reference counts:
- Before
unlink: On-diski_nlink = 1(directory entry exists); in-corei_count = 1‘s open file description holds an active reference). - After
unlink: The directory entry is removed from the directory block, so on-diski_nlink = 0. However, in-corei_countremains 1 because ‘s descriptor 3 is still active.
- Before
- Access persistence:
Yes. POSIX dictates that descriptor operations operate through the open file description, not pathname resolution. can continue reading, writing, and seeking through descriptor 3 without error. Any new attempt by other processes to
open("/var/run/worker.pid")will returnENOENT(unless a new file of that name is created). - Deallocation trigger:
When executes
close(3)(or when terminates), the kernel decrementsi_countfrom 1 to 0. Findingi_count == 0andi_nlink == 0, the VFS/filesystem driver deallocates all mapped data blocks to the free bitmap and marks the inode free on disk.
Exercise 6.1.3: Linearization Points and Concurrent Reader Observations Across rename
Process prepares an updated database file by creating temporary file db.tmp and executing rename("db.tmp", "db.live"). Concurrently, reader processes repeatedly call open("db.live", O_RDONLY).
- Prove that no reader process can ever observe a “file not found” (
ENOENT) error during the rename operation. - If reader opened
db.liveprior to the rename linearization point, what content does observe after the rename commits? - Explain why
renamealone does not guarantee that the new database file survives a power failure.
HintExpandCollapse
POSIX rename is atomic in namespace; readers either get the old inode or new inode; durability needs fsync.
Worked solutionExpandCollapse
- Namespace atomicity invariant:
POSIX specifies that
rename(old, new)is atomic with respect to namespace lookups. The filesystem directory implementation updates or replaces the directory entry in a single atomic directory transaction. At any point in time, a pathname resolution traversing"db.live"either resolves to the old inode (before the directory commit) or the new inode (after the commit). There is no intermediate window where the directory entry is empty or absent. - Reader observation: ‘s descriptor is bound to the open file description of the old inode Modifying the directory pointer does not change existing open descriptors. continues reading the original database contents until it closes its descriptor.
- Durability boundary:
renameguarantees directory namespace atomicity in RAM/dcache. It does not force dirty directory blocks or file data from volatile OS page cache to disk platters/flash. If power fails immediately afterrename, storage may restore to an older state, write back out-of-order blocks, or expose a 0-byte file unless preceded byfsync(fd_tmp)and followed byfsync(fd_dir).
Exercise 6.1.4: Designing a Crash-Resilient and Race-Free Atomic Publication Protocol
Specify the exact, crash-resilient 5-step sequence required to replace a configuration file /etc/daemon.conf using POSIX system calls, ensuring that crash recovery either yields the complete old file or complete new file without corrupt half-writes.
HintExpandCollapse
Create temp in same filesystem -> write -> fsync file -> rename -> fsync directory.
Worked solutionExpandCollapse
The atomic, crash-resilient publication protocol proceeds as follows:
- Create temporary file in same directory:
Call
int fd = open("/etc/daemon.conf.tmp.PID", O_WRONLY | O_CREAT | O_EXCL, 0644). The temporary file must reside in the same filesystem/mount as/etcso that subsequentrenameis a local directory update rather than a cross-deviceEXDEVcopy. - Write complete contents:
Execute a write loop ensuring all buffer bytes are written, handling partial writes and interrupted system calls (
EINTR). - Sync file data and metadata:
Call
fsync(fd)and check the return code. This guarantees all file data blocks and the inode are durable on non-volatile media before publication. Closefd. - Atomic rename:
Call
rename("/etc/daemon.conf.tmp.PID", "/etc/daemon.conf")and check the return code. This atomically swings the directory pointer from old to new. - Sync containing directory:
Open the directory
int dfd = open("/etc", O_RDONLY | O_DIRECTORY), executefsync(dfd), and closedfd. This commits the updated directory entry to durable storage.
Exercise 6.1.5: Symbolic Link Traversal Limits and Detection of Cyclic References
Explain how an operating system path resolution routine traverses symbolic links. Describe how a cyclic reference (e.g., ln -s /a/b /a/c and ln -s /a/c /a/b) is detected and handled to prevent infinite kernel loops.
HintExpandCollapse
Track symlink traversal count against SYMLOOP_MAX (typically 32 to 40) and return ELOOP.
Worked solutionExpandCollapse
- Traversal mechanism:
When
nameior path walk encounters a directory entry pointing to an inode of typeS_IFLNK, the kernel reads the symlink’s payload target string into a buffer. If the target begins with'/', traversal restarts from the root directory; otherwise, it resolves relative to the current directory. The remaining components of the original pathname are appended to the link path, and resolution continues. - Cycle detection and loop bounds:
Filesystems do not detect cycles at creation time (since creating a dangling or cyclic link is permissible). Instead, the path resolver maintains an integer counter tracking the number of nested symbolic links traversed during a single resolution call.
POSIX defines
SYMLOOP_MAX(typically 32 in modern Linux / BSD). If the counter exceedsSYMLOOP_MAX, the kernel halts traversal and returns-ELOOP(Too many levels of symbolic links), protecting the kernel stack from overflow and terminating the infinite loop.
References & Further Reading
- The Open Group, POSIX.1-2024 Base Definitions, Chapter 4: General Concepts — pathname resolution and filesystem-operation semantics.
- The Open Group, POSIX.1-2024
open()andopenat()— descriptor creation and final-component behavior. - The Open Group, POSIX.1-2024
rename()andrenameat()— atomic namespace replacement and boundary conditions. - Linux man-pages,
openat2(2)— Linux-specific resolution constraints for race-resistant path handling. - Linux kernel documentation, Overview of the Virtual File System — dentries, inodes, open file objects, and operation dispatch.
- MIT PDOS, xv6 RISC-V book and immutable xv6-rev5 source tree — the compact descriptor, inode, directory, and pathname implementation used for teaching-kernel examples.