Curious bout your use case for building all software from source, is it because you're worried about the supply chain since nixpkgs builds don't have reproducibility guarantee?
I was already building the vast majority of it from source because I enable CPU optimizations for the specific microarchitecture in the machine (nixpkgs.hostPlatform.gcc.arch and nixpkgs.hostPlatform.gcc.tune), so once I learned about the risk of supply chain attack on the nix cache, disabling it entirely was a pretty small change.
So far, I'd say the biggest negative (aside from the build times that I was already experiencing due to the optimizations) is that GNU savannah will temporarily IP ban you when you download too much from them. For example, building the grub that is used for the ISOs downloads like 70+ patches from GNU Savannah which frequently triggers the IP ban.
One suggestion for improving the cache's posture: Give each dedicated builder machine a TPM. On each builder, generate a TPM-backed signing key. Only use those TPM-backed keys for signing packages that end up in the cache. That way we'd ensure that signed packages _must_ have been built on the dedicated build machines (due to the TPM) rather than potentially uploaded from a compromised developer machine using a compromised signing key.
It seems the current situation is there is a single signing key that has been in use since 2015 which implies that key is not tied to specific hardware and therefore could be leaked/exfiltrated/copied: https://github.com/NixOS/rfcs/pull/149/files#diff-ac4cc42774...
A reasonable suggestion, for any code signing system really. We need all the help we can get these days.
Have you measured the gains from your source-based approach? I was into Gentoo for tuning a long time ago when hardware wasn't as powerful, but I hadn't considered optimizing at that level with nix.
