Search Knowledge

© 2026 LIBREUNI PROJECT

Page 19 of 29

Filesystem Interface and Names

Path resolution, directory entries, open file descriptions, links, atomic namespace operations, and race-resistant file access.

Lesson briefPersistence · 5 h · 4 outcomes

Required knowledge

  • C pointers, structs, arrays, and system-call error handling
  • Processes, file descriptors, and virtual address spaces
  • Kernel/user privilege boundaries

Learning outcomes

  • Distinguish pathnames, directory entries, filesystem objects, open file descriptions, and process-local descriptors.
  • Trace path resolution, descriptor duplication, unlink, link, and rename under precisely stated concurrency assumptions.
  • Design race-resistant and crash-aware file update protocols without confusing atomic visibility with durability.
  • Diagnose namespace bugs involving symbolic links, shared offsets, stale names, and open-but-unlinked objects.

Assessment target

Produce and defend traces for unfamiliar pathname and descriptor schedules, then specify a race-resistant update protocol with explicit atomicity, authority, and durability assumptions.

Scope boundary

  • Network filesystem cache coherence, distributed naming, mandatory access control, and implementation-specific extended attributes require separate treatment.
  • Crash recovery is bounded here to interface guarantees; on-disk recovery mechanisms are developed independently.

A configuration daemon opens /srv/app/config, validates its contents, and keeps the returned descriptor. An administrator writes a replacement file and atomically renames it over config. New calls to open("/srv/app/config", ...) reach the replacement; the daemon’s existing descriptor still reaches the old object. If the old name was its last directory entry, the old object has no pathname at all, yet the daemon can continue reading it.

The apparent contradiction disappears when a pathname and an open file are modeled as different references. A pathname is a request to traverse a mutable namespace. A descriptor is a process-local handle to an already opened description. Treating both as “the file” causes stale-read bugs, authority races, and incorrect durability claims.

Five objects hidden behind one integer

Assume a local POSIX-like filesystem, one mount namespace, no I/O error, and no concurrent namespace mutation until a trace explicitly introduces one. The model contains five distinct objects:

ObjectIdentityMutable state relevant hereTypical lifetime
pathname stringbytes supplied by a callercomponents may be interpreted relative to a directoryone operation
directory entrypair (name, object-id) inside a directoryname and target mappinguntil unlink or rename
filesystem objectinode-like identitytype, size, link count, permissions, block mapwhile linked or referenced
open file descriptionkernel record for an open instancecurrent offset and file-status flagswhile any descriptor refers to it
file descriptorsmall integer in one processdescriptor flags such as close-on-execuntil close or table replacement

A symbolic link is itself a filesystem object whose payload is another pathname. A hard link is another directory entry for an existing object; it is not a pointer to another name. A process may have several descriptors for one open file description, and several independently opened descriptions may refer to one object.

Code
left to right direction
rectangle "process P\nfd 3" as P3
rectangle "process P\nfd 7" as P7
rectangle "process Q\nfd 4" as Q4
rectangle "open description A\noffset = 8192\nstatus = O_RDWR" as OFA
rectangle "open description B\noffset = 0\nstatus = O_RDONLY" as OFB
rectangle "inode 41\nregular file\nlink count = 2" as I41
rectangle "directory /srv/app\nconfig -> 41\nconfig.backup -> 41" as DIR
P3 --> OFA : indexes
P7 --> OFA : dup shares
Q4 --> OFB : independent open
OFA --> I41 : holds reference
OFB --> I41 : holds reference
DIR --> I41 : two names
process Pfd 3process Pfd 7process Qfd 4open description Aoffset = 8192status = O_RDWRopen description Boffset = 0status = O_RDONLYinode 41regular filelink count = 2directory /srv/appconfig -> 41config.backup -> 41indexesdup sharesindependent openholds referenceholds referencetwo names

The diagram separates two graphs. Directory entries form the namespace graph. Descriptors and open descriptions form the open-reference graph. Unlinking an edge in the first graph does not revoke edges in the second.

Retrieval checkpoint: what does dup copy?

Without consulting the answer, record which state is shared after b = dup(a): pathname, descriptor number, descriptor flags, current offset, file-status flags, and inode identity.

Reveal the state partitionExpand

a and b are different descriptor numbers in one table. They refer to the same open file description, so they share its current offset and file-status flags; they consequently reach the same inode. Descriptor flags are per descriptor and are not thereby shared. No pathname is stored as the descriptor’s operative identity. If the process calls open a second time instead of dup, the new descriptor normally receives a distinct open file description and therefore a distinct offset.

A response that says “everything is copied” has collapsed three layers: descriptor entry, open description, and inode. A response that gives each descriptor an independent offset has confused dup with a second open.

File descriptors, open descriptions, and namespace invariants

0 / 4 answered

Evaluate each statement regarding POSIX file descriptors, directory entries, and file lifecycles:

Classify each option as correct or incorrect, or leave uncertain options unanswered (“I don't know”) and verify at any time.

1
Unlinking a file removes its directory entry while existing open file descriptors continue reading its data.
2
Two separate open calls to the same file pathname in one process always share a single common file offset.
3
Duplicating a descriptor with dup creates a new descriptor that shares the exact file offset and status flags.
4
The POSIX rename system call provides immediate disk durability guarantees without requiring explicit fsync calls.

Worked trace: shared and independent offsets

File records contains sixteen bytes abcdefghijklmnop. Reads below are successful and no signal interrupts them.

int a = open("records", O_RDONLY);  // open description X, offset 0
int b = dup(a);                     // also X
int c = open("records", O_RDONLY);  // open description Y, offset 0

Execute read(a, p, 4), read(b, q, 3), read(c, r, 2), then lseek(a, 1, SEEK_SET) and read(b, s, 3).

Eventbytes returnedoffset of X after eventoffset of Y after event
initial state—00
read(a, ..., 4)abcd40
read(b, ..., 3)efg70
read(c, ..., 2)ab72
lseek(a, 1, SEEK_SET)—12
read(b, ..., 3)bcd42

The descriptor used for an operation does not determine the offset object. a and b both mutate X. c reaches the same inode through Y but cannot observe X’s offset changes.

Application checkpoint: inherited descriptors

A parent opens an output file, calls fork, and both processes write fixed-size records through the inherited descriptor without seeking. State the property that descriptor sharing supplies and the property it does not supply.

Reveal the diagnostic answerExpand

The inherited descriptors refer to one open file description, so they share one current offset. POSIX specifies relevant descriptor operations atomically with respect to the effects on regular files, preventing both writes from independently using the same old offset under the specified interface semantics. This does not impose an application-level order between parent and child records, combine several writes into one transaction, make buffered library I/O safe after fork, or make either record durable. If record order is part of correctness, the processes need an explicit protocol.

Path resolution is a stateful walk

For an absolute pathname, resolution begins at the process’s root directory. For a relative pathname, it begins at the current working directory or at a directory descriptor supplied to an *at interface. For each non-final component, the kernel must:

  1. verify that the current object is a directory and that search is authorized;
  2. look up the component in that directory;
  3. follow a symbolic link when the operation’s rules require it, splicing the link payload into the unresolved suffix; and
  4. continue from the resulting directory, crossing a mount point if the namespace contains one.

. selects the current directory. .. selects its parent subject to root and mount semantics. The final component is operation-specific: open normally follows a final symbolic link, lstat examines the link itself, and rename renames a final link rather than its target. A create operation resolves the parent and leaves the absent final name for insertion.

Code
start
:choose root, cwd, or directory fd;
while (unresolved component?) is (yes)
:select next component;
if (current object is searchable directory?) then (yes)
  :lookup name under current directory;
else (no)
  :fail with ENOTDIR or EACCES;
  stop
endif
if (component is followed symbolic link?) then (yes)
  :splice link payload before suffix;
  if (resolution limit exceeded?) then (yes)
    :fail with ELOOP;
    stop
  endif
endif
endwhile (no)
:apply final-component rule for operation;
stop
choose root, cwd, or directory fdselect next componentcurrent object is searchabledirectory?yesnolookup name under currentdirectoryfail with ENOTDIR or EACCESsplice link payload before suffixfail with ELOOPyesresolution limit exceeded?yescomponent is followed symboliclink?yesunresolved component?noapply final-component rule foroperation

Assume this initial namespace and a process whose current directory is /work:

/work/input        -> symlink payload "../safe/report"
/safe/report       -> inode 80
/attacker/report   -> inode 99

The process first calls stat("input", ...), validates inode 80’s owner, and later calls open("input", O_RDONLY). Between the calls, an attacker who can modify /work replaces input with a symlink to ../attacker/report. The validation and use are two independent pathname walks:

Timeoperationresolved object
t0t_0stat("input")inode 80
t1t_1attacker replaces directory entrynamespace now targets inode 99
t2t_2open("input")inode 99

No corruption of the kernel’s lookup algorithm occurred. The application incorrectly treated a pathname result as a stable capability. Opening first and validating with fstat binds validation and subsequent I/O to the same open object. When policy requires confinement beneath a trusted directory on Linux, a trusted directory descriptor plus openat2 resolution constraints can also reject symlink, mount, or escape behavior; those flags are Linux interfaces, not portable POSIX guarantees.

Transfer checkpoint: close the authority race

A privileged image service must open a caller-supplied relative name beneath an already opened directory tenant_dir. The caller can concurrently rename entries and create symbolic links. Specify a safe ordering and state what remains unproved.

Reveal one defensible protocolExpand

Keep tenant_dir as an open directory descriptor acquired from trusted configuration. Resolve relative to that descriptor, not the process-wide current directory. On Linux, use openat2 with a policy-appropriate combination such as RESOLVE_BENEATH or RESOLVE_IN_ROOT, plus restrictions on symbolic links or mount crossings when the service’s contract forbids them. Open with the least required access, then inspect the returned descriptor with fstat and perform I/O through that descriptor.

The protocol binds later I/O to the object actually opened. It does not prove that file contents are benign, that the trusted directory itself was acquired securely, that every filesystem implements identical remote semantics, or that a kernel vulnerability cannot violate the boundary. Listing flags without deriving them from the service’s path policy is not a complete answer.

For a regular file, the link count records directory entries that name the object. Open descriptions contribute references but not hard links. Storage may be reclaimed only after both the namespace link count and the relevant in-memory references reach zero.

Initial state: /tmp/job.out -> inode 17, nlink(17)=1, and process A has descriptor 5 open on inode 17.

Stepnamespacelink countopen referencesresult
0job.out -> 1711named and open
1: unlink("/tmp/job.out")no entry01new lookup fails; fd 5 still works
2: A writes through fd 5no entry01inode 17 grows but remains unnamed
3: A closes fd 5no entry00object becomes reclaimable

Creating a new /tmp/job.out after step 1 creates or links some other object. It does not reattach the old descriptor by name.

rename(old, new) changes namespace edges. Within the scope where the operation is supported, another thread must not observe new absent during replacement: it refers to either the prior target or the renamed object. If new existed and its last link is removed, open references to that displaced object still keep it alive. Cross-filesystem rename may fail with EXDEV because one atomic metadata transaction cannot generally span unrelated implementations.

Boundary. Namespace atomicity is not storage durability. A successful rename establishes an allowed concurrent-observation result. It does not by itself prove which names or file contents survive power loss.

old names inode A and new names inode B. Process R repeatedly resolves new while process W executes successful rename(old, new). No other namespace changes occur. Which inode identities may R obtain? What happens to an already open descriptor for B?

Reveal the observation setExpand

Each completed resolution of new may obtain B before the rename linearization point or A after it. The specified replacement does not expose an interval in which new is missing. A descriptor opened on B before replacement continues to refer to B; B’s content is reclaimed only after its last link and open reference are gone. An answer claiming that R’s existing descriptor switches from B to A has made descriptors resolve names repeatedly. An answer claiming that the result necessarily survives a crash has imported a durability guarantee that the premise does not provide.

Deriving a crash-aware replacement protocol

Consider a local filesystem whose documentation states that fsync(file) makes the file’s required data and metadata persistent and fsync(directory) persists directory-entry updates. Under those explicit assumptions, an application replacing config can seek either of two recovery states: the complete old object or the complete new object.

1. create a temporary file in the target directory without reusing an attacker-controlled name
2. write the complete new representation
3. fsync the temporary file and check the result
4. rename the temporary name over config and check the result
5. fsync the containing directory and check the result

Step 3 prevents a durable new name from exposing unwritten or stale content under the assumed interface. Step 4 provides atomic namespace replacement to concurrent readers. Step 5 requests persistence of the namespace update. Every failure path must preserve the old file or report that recovery is uncertain; ignoring an fsync error destroys the protocol’s evidence.

This recipe is not universal folklore. Filesystem, device-cache, network, and platform semantics can narrow or alter it. It does not make several files into one atomic transaction. It also does not update long-lived reader descriptors; readers that require fresh configuration must reopen or use an explicit version-notification protocol.

Section Exercises

Deep Practice

Solve these problems to test and consolidate your understanding. Complete worked solutions and diagnostic hints are available for each problem.

Exercise 6.1.1: Tracing Shared versus Independent File Offsets Across dup and open

Verificationapplication

A file log.txt contains 20 bytes 0123456789abcdefghij. A process executes:

int fd1 = open("log.txt", O_RDONLY);
int fd2 = dup(fd1);
int fd3 = open("log.txt", O_RDONLY);
char buf[5];
read(fd1, buf, 4); // Op A
read(fd2, buf, 4); // Op B
read(fd3, buf, 4); // Op C
  1. State the file offset for fd1, fd2, and fd3 after Op A.
  2. State the byte string read into buf by Op B and Op C.
  3. Draw the relationship between descriptor table entries, open file descriptions, and the in-memory inode.
HintExpand

dup shares the open file description and offset; separate open calls create new open file descriptions.

Worked solutionExpand
  1. Offsets after Op A: fd1 and fd2 point to the identical open file description (OFD 1). When read(fd1, buf, 4) consumes 4 bytes, OFD 1’s offset advances from 0 to 4. Therefore, both fd1 and fd2 now observe offset 4. fd3 refers to independent OFD 2, whose offset remains 0.
  2. Read results:
    • Op B (read(fd2, buf, 4)): Reads 4 bytes from OFD 1 starting at offset 4, returning "4567". OFD 1’s offset advances to 8.
    • Op C (read(fd3, buf, 4)): Reads 4 bytes from OFD 2 starting at its initial offset 0, returning "0123". OFD 2’s offset advances to 4.
  3. Architecture graph:
    • fd1 (slot in proc table) →\rightarrow Open File Description 1 (offset = 8) →\rightarrow Inode (log.txt).
    • fd2 (slot in proc table) →\rightarrow Open File Description 1 (offset = 8).
    • fd3 (slot in proc table) →\rightarrow Open File Description 2 (offset = 4) →\rightarrow Inode (log.txt).

Exercise 6.1.3: Linearization Points and Concurrent Reader Observations Across rename

Verificationtransfer

Process WW prepares an updated database file by creating temporary file db.tmp and executing rename("db.tmp", "db.live"). Concurrently, reader processes R1,R2,…R_1, R_2, \dots repeatedly call open("db.live", O_RDONLY).

  1. Prove that no reader process can ever observe a “file not found” (ENOENT) error during the rename operation.
  2. If reader R1R_1 opened db.live prior to the rename linearization point, what content does R1R_1 observe after the rename commits?
  3. Explain why rename alone does not guarantee that the new database file survives a power failure.
HintExpand

POSIX rename is atomic in namespace; readers either get the old inode or new inode; durability needs fsync.

Worked solutionExpand
  1. Namespace atomicity invariant: POSIX specifies that rename(old, new) is atomic with respect to namespace lookups. The filesystem directory implementation updates or replaces the directory entry in a single atomic directory transaction. At any point in time, a pathname resolution traversing "db.live" either resolves to the old inode IoldI_{\text{old}} (before the directory commit) or the new inode InewI_{\text{new}} (after the commit). There is no intermediate window where the directory entry is empty or absent.
  2. Reader observation: R1R_1‘s descriptor is bound to the open file description of the old inode IoldI_{\text{old}} Modifying the directory pointer does not change existing open descriptors. R1R_1 continues reading the original database contents until it closes its descriptor.
  3. Durability boundary: rename guarantees directory namespace atomicity in RAM/dcache. It does not force dirty directory blocks or file data from volatile OS page cache to disk platters/flash. If power fails immediately after rename, storage may restore to an older state, write back out-of-order blocks, or expose a 0-byte file unless preceded by fsync(fd_tmp) and followed by fsync(fd_dir).

Exercise 6.1.4: Designing a Crash-Resilient and Race-Free Atomic Publication Protocol

Constructionsynthesis

Specify the exact, crash-resilient 5-step sequence required to replace a configuration file /etc/daemon.conf using POSIX system calls, ensuring that crash recovery either yields the complete old file or complete new file without corrupt half-writes.

HintExpand

Create temp in same filesystem -> write -> fsync file -> rename -> fsync directory.

Worked solutionExpand

The atomic, crash-resilient publication protocol proceeds as follows:

  1. Create temporary file in same directory: Call int fd = open("/etc/daemon.conf.tmp.PID", O_WRONLY | O_CREAT | O_EXCL, 0644). The temporary file must reside in the same filesystem/mount as /etc so that subsequent rename is a local directory update rather than a cross-device EXDEV copy.
  2. Write complete contents: Execute a write loop ensuring all buffer bytes are written, handling partial writes and interrupted system calls (EINTR).
  3. Sync file data and metadata: Call fsync(fd) and check the return code. This guarantees all file data blocks and the inode are durable on non-volatile media before publication. Close fd.
  4. Atomic rename: Call rename("/etc/daemon.conf.tmp.PID", "/etc/daemon.conf") and check the return code. This atomically swings the directory pointer from old to new.
  5. Sync containing directory: Open the directory int dfd = open("/etc", O_RDONLY | O_DIRECTORY), execute fsync(dfd), and close dfd. This commits the updated directory entry to durable storage.

References & Further Reading