Determinate Nix best practices

At Determinate Systems, we’re building Determinate Nix to be the best Nix for teams: fast, safe, sensible by default, and offering powerful features available in no other distribution. To get the maximum benefit out of it, we strongly recommend following the practices below.

Use lazy trees

Lazy trees enable Nix to copy only the files that your expression actually needs instead of copying the entire source tree into the Nix store before evaluating. They are especially useful in larger repositories. In Nixpkgs, for example, we routinely see evaluations run 3x or more faster while using one twentieth or less of the disk space when lazy trees are enabled.

Lazy trees are stable and enabled by default, so the best practice is to stick with the default and leave them on. You should disable lazy trees only if you want to compare performance with the feature off. Be skeptical of any advice that tells you to disable them permanently. If an evaluation misbehaves with lazy trees enabled, we’d recommend reporting that to us and disabling only temporarily rather than as a long-term fix.

Use parallel evaluation

Parallel evaluation spreads evaluation work across your processor cores, which cuts wall time substantially for expensive operations like nix search, nix flake check, nix flake show, and nix eval --json. We frequently measure speed-ups of 5x or more for some Determinate Nix operations.

Like lazy trees, parallel evaluation is enabled by default for all Determinate Nix users. To get the most out of it, we recommend giving any CI runners or builders running Determinate Nix more than a few cores, since evaluation can make efficient use of them. For expensive expressions that the evaluator can’t parallelize on its own, especially ones that rely on import-from-derivation (IFD), we recommend using builtins.parallel directly.

Let Determinate Nixd collect your garbage

Determinate Nixd handles garbage collection for you. It keeps at least 30GB free for system updates, aims for a steady state of 5 to 20% free disk space, and switches to an urgent mode if you drop below 5%. Because it runs continuously in the background, it deletes a little at a time rather than a lot at once, which keeps its impact on your system low.

We recommend resisting any urge to add an additional garbage collection process on top of that, for example a nix-collect-garbage cron job or a systemd timer. If you genuinely need to manage garbage collection yourself, turn Determinate Nixd’s collector off explicitly by setting garbageCollector.strategy to disabled in its configuration.

Don’t mount your Nix store with online discard

If your Nix store lives on ext4, mount it with nodiscard and reclaim free space with fstrim on a schedule instead. The discard mount option and Nix garbage collection are a bad combination.

Garbage collection deletes an enormous number of files at once, and with discard each freed extent becomes a synchronous discard request to the block device. On network-backed or thin-provisioned storage, like Google Persistent Disk, Amazon Elastic Block Store (EBS), or a thin Logical Volume Manager (LVM) volume, those requests take roughly 10 milliseconds each. That caps the filesystem at about 100 discards per second, keeps the device 97 to 99% busy, and starves every other read and write on the machine.

The effect on a busy store is severe. Nix evaluations that normally take seconds can stretch into tens of minutes whenever garbage collection is running, and garbage collection itself slows to a crawl.

To fix it, drop discard from the mount options for the filesystem holding /nix:

/dev/nvme0n1p1  /  ext4  defaults,nodiscard  0  1

Then enable a periodic trim instead, which most distributions ship as a weekly fstrim.timer:

systemctl enable --now fstrim.timer

We also recommend passing --minimum 1M to fstrim so it skips the smallest free extents, which generate the most discard requests for the least reclaimed space.

nodiscard doesn’t stop ext4 from reusing freed blocks, so your store keeps working exactly as before. The only difference is that you tell the underlying device about that free space on a schedule rather than in the middle of a garbage collection run.