NFS Soft Mounts and Silent Write Loss
// Explains why NFS soft mounts open you up to silently dropped writes.
Overview
NFS should always be mounted using the hard option unless specifically advised by the application vendor to use soft, or if the mount is read-only.
The Core Problem
With a hard mount, a write call blocks indefinitely until the server responds. The application stalls, but it never receives a success response until the server confirms the write. No response = no acknowledgment = the caller waits.
With a soft mount, the client gives up after a configurable timeout (timeo × retrans) and returns an error to the caller. The risk: most applications don't treat write errors as fatal. They assume write() succeeds unless the system is catastrophically broken. The result is unchecked errors--and lost data the application believes is persisted.
The Write-Loss Sequence
sequenceDiagram
participant App
participant NFSClient as NFS Client
participant Server as NFS Server
App->>NFSClient: write()
NFSClient->>Server: RPC WRITE request
Note over Server: Server slow / rebooting / network blip
loop timeo × retrans
NFSClient-->>Server: retry RPC...
Server--xNFSClient: no response
end
NFSClient-->>App: EIO (timeout)
Note over App: error ignored or swallowed by runtime
App->>NFSClient: fsync() / close()
NFSClient-->>App: EIO (also ignored)
Note over App: App proceeds — believes data is durable
Note over Server: Data was never writtenWhy Applications Miss the Error
The failure pattern isn't unique to naïve code. Several layers can swallow the error:
- Buffered I/O (
stdio,BufferedWriter): the write to the buffer succeeds; the flush to the file descriptor fails silently if the error isn't explicitly checked. - Interpreted runtimes (Python, Ruby, JVM):
file.close()raises an exception in well-behaved libraries, but many callers use context managers or finalizers that suppress it. - Log frameworks: many loggers treat write failures as non-fatal by design — the last thing you want is the logger crashing the application.
- Database WAL writers: typically do check, but only if they mount with
syncor useO_SYNC/O_DSYNC.
Hard Mount Tradeoffs
Hard mounts are not free. The tradeoffs are real:
| Scenario | Hard Mount Behavior | Soft Mount Behavior |
|---|---|---|
| Transient network blip | Blocks, recovers transparently | Times out, returns error |
| Server reboot | Blocks until server returns | Times out after timeo×retrans |
| Server permanent failure | Blocks indefinitely (hung process) | Times out, returns error |
| NFS server storage failure | Blocks indefinitely | Times out, returns error |
The hung-process case is the real argument for soft mounts: an unresponsive server on a hard mount can leave kernel threads unkillable. The correct mitigation is hard,timeo=<n>,retrans=<n> with monitoring — not soft mounts.
A Second Window: Async Server Writes
Even on a hard mount, data can be lost if:
- Client sends WRITE RPC with
UNSTABLEflag (async, NFS v3 default) - Server ACKs before committing to stable storage
- Server crashes before the COMMIT RPC completes
The NFS client is expected to track unstable writes and reissue them on reconnect, but a server crash before COMMIT — combined with a server-side write cache that wasn't flushed — can still lose data. This is distinct from the soft-mount problem but worth noting for completeness.
Practical Guidance
Soft mounts are defensible only for:
- Read-only workloads
- Scratch/disposable data
- Workloads where stalling is worse than data loss (e.g., some streaming pipelines)
For durability, the standard recommendation is:
hard,timeo=600,retrans=2,nointrnointr prevents SIGINT from killing the blocked syscall, preserving the hard-mount guarantee. timeo/retrans bound retry duration without switching to soft semantics.