RWF_UNCACHED and buffered I/O without page-cache pollution

The page-cache problem

Buffered I/O is convenient because the kernel page cache absorbs reads and writes. The same convenience can become a problem when a large one-off workload pushes useful cache pages out of memory.

Backup scans, media processing, database-adjacent file walks, and large sequential copies can read many gigabytes that will not be reused. If those reads populate the page cache, they compete with hot application data. The work finishes, but the rest of the machine may be colder and slower afterward.

That is the problem Jens Axboe's RWF_UNCACHED patch set tries to address: keep the simplicity of buffered I/O while reducing page-cache pollution for data that the caller does not want retained.

Buffered versus direct I/O

Direct I/O already lets applications bypass much of the page cache, but it comes with constraints. Alignment, block sizes, filesystem behavior, and application buffering become more demanding. It is a useful tool, but not a drop-in replacement for ordinary reads and writes.

Buffered I/O remains easier for most programs. The kernel handles caching, readahead, writeback, and interaction with the filesystem. The missing knob is intent: sometimes the application wants buffered semantics without long-term cache residency.

RWF_UNCACHED attempts to express that intent at the per-operation level.

What the flag means

The proposed flag belongs with the preadv2 and pwritev2 family of per-I/O flags. Instead of changing how an entire file is opened, the caller can mark a specific operation as uncached.

Conceptually, this is useful for mixed workloads. A program may normally benefit from buffered I/O but occasionally perform a large scan that should not displace hotter data. A per-call flag is more precise than opening a second file descriptor with a completely different mode.

The exact semantics depend on kernel implementation, filesystem support, and the final accepted API. The durable idea is the same: give applications a way to say "do this buffered operation, but do not keep the pages around for me."

Fast storage changes the cost

On slow disks, caching often hides latency. On fast NVMe storage, the relationship is more complicated. Reading from storage may be fast enough that polluting memory with low-value cache pages hurts more than it helps.

Large memory machines also change expectations. The page cache can hold a lot, but production machines run multiple workloads. A batch job that consumes cache aggressively can still disturb databases, search indexes, build systems, or language runtimes sharing the host.

Uncached buffered I/O is interesting because it targets this modern shape: fast storage, large memory, and mixed workloads where cache residency should follow reuse rather than accident.

Cautions

This is kernel-level work, so the practical advice is conservative. Do not build application correctness on a patch-set detail before the API is stable in the kernels you deploy. Treat it first as a performance and cache-behavior tool.

It also does not remove the need for measurement. A workload that looks one-off may actually benefit from readahead or later reuse. Another workload may be limited by CPU, decompression, checksums, or userspace copying rather than page-cache residency.

The right test watches both the target program and the rest of the host: throughput, major faults, cache pressure, tail latency, and whether hot services stay warm after the scan.

What uncached changes

The interesting part of RWF_UNCACHED is that it tries to avoid the page-cache side effects of a specific read or write without forcing the whole file descriptor into direct I/O rules. That is a narrower tool than O_DIRECT.

struct iovec iov = {
    .iov_base = buffer,
    .iov_len = length,
};

ssize_t bytes = preadv2(fd, &iov, 1, offset, RWF_UNCACHED);

The appeal is workload isolation. A backup, scan, or one-off export may need to read a lot of data without evicting the hot working set from cache. The risk is that cache behavior is subtle: filesystem support, alignment, writeback, readahead, and kernel version all affect what the flag can promise.

That is why this kind of interface should be treated as a performance hint with measurable effects, not as a portable correctness primitive.

Measurement before adoption

The reason to care about uncached buffered I/O is protecting the cache from work that is large, sequential, and unlikely to be reused. That hypothesis has to be measured. Look at cache hit rates, major faults, throughput, tail latency, and whether the supposedly cold workload actually harms the hot workload.

If the measurements do not show cache pollution, the flag is probably unnecessary complexity. If they do, the next question is whether the application can isolate that workload at a higher level with scheduling, batching, replicas, or separate workers. Kernel flags are most useful when the workload shape is already well understood.

The benchmark should include the workload that might be harmed, not only the workload using the flag. A scan becoming slightly faster is not a win if it still evicts the working set or increases tail latency for the interactive path.