NFS Soft Mounts and Silent Write Loss

// Explains why NFS soft mounts open you up to silently dropped writes.

Overview

NFS should always be mounted using the hard option unless specifically advised by the application vendor to use soft, or if the mount is read-only.

The Core Problem

With a hard mount, a write call blocks indefinitely until the server responds. The application stalls, but it never receives a success response until the server confirms the write. No response = no acknowledgment = the caller waits.

With a soft mount, the client gives up after a configurable timeout (timeo × retrans) and returns an error to the caller. The risk: most applications don't treat write errors as fatal. They assume write() succeeds unless the system is catastrophically broken. The result is unchecked errors--and lost data the application believes is persisted.

The Write-Loss Sequence

sequenceDiagram
    participant App
    participant NFSClient as NFS Client
    participant Server as NFS Server

    App->>NFSClient: write()
    NFSClient->>Server: RPC WRITE request
    Note over Server: Server slow / rebooting / network blip

    loop timeo × retrans
        NFSClient-->>Server: retry RPC...
        Server--xNFSClient: no response
    end

    NFSClient-->>App: EIO (timeout)
    Note over App: error ignored or swallowed by runtime
    App->>NFSClient: fsync() / close()
    NFSClient-->>App: EIO (also ignored)
    Note over App: App proceeds — believes data is durable
    Note over Server: Data was never written

Why Applications Miss the Error

The failure pattern isn't unique to naïve code. Several layers can swallow the error:

  • Buffered I/O (stdio, BufferedWriter): the write to the buffer succeeds; the flush to the file descriptor fails silently if the error isn't explicitly checked.
  • Interpreted runtimes (Python, Ruby, JVM): file.close() raises an exception in well-behaved libraries, but many callers use context managers or finalizers that suppress it.
  • Log frameworks: many loggers treat write failures as non-fatal by design — the last thing you want is the logger crashing the application.
  • Database WAL writers: typically do check, but only if they mount with sync or use O_SYNC/O_DSYNC.

Hard Mount Tradeoffs

Hard mounts are not free. The tradeoffs are real:

ScenarioHard Mount BehaviorSoft Mount Behavior
Transient network blipBlocks, recovers transparentlyTimes out, returns error
Server rebootBlocks until server returnsTimes out after timeo×retrans
Server permanent failureBlocks indefinitely (hung process)Times out, returns error
NFS server storage failureBlocks indefinitelyTimes out, returns error

The hung-process case is the real argument for soft mounts: an unresponsive server on a hard mount can leave kernel threads unkillable. The correct mitigation is hard,timeo=<n>,retrans=<n> with monitoring — not soft mounts.

A Second Window: Async Server Writes

Even on a hard mount, data can be lost if:

  1. Client sends WRITE RPC with UNSTABLE flag (async, NFS v3 default)
  2. Server ACKs before committing to stable storage
  3. Server crashes before the COMMIT RPC completes

The NFS client is expected to track unstable writes and reissue them on reconnect, but a server crash before COMMIT — combined with a server-side write cache that wasn't flushed — can still lose data. This is distinct from the soft-mount problem but worth noting for completeness.

Practical Guidance

Soft mounts are defensible only for:

  • Read-only workloads
  • Scratch/disposable data
  • Workloads where stalling is worse than data loss (e.g., some streaming pipelines)

For durability, the standard recommendation is:

hard,timeo=600,retrans=2,nointr

nointr prevents SIGINT from killing the blocked syscall, preserving the hard-mount guarantee. timeo/retrans bound retry duration without switching to soft semantics.

See Also

TERMINAL comments.log
# opening comments.log…