Cow

Gracefully provided by Quaritsch Photography on Unsplash, via https://unsplash.com/photos/1_6rJHQ2Gmw

NixOS Configuration

This is a bare minimum nix configuration for koumakan.

I'm still very, very new to Nix and its ecosystem so pointers to better way of doing things are very much appreciated.

The canonical URL of this site is https://flake.soopy.moe.

Documentation

Documentation and other tips can be found in this book. See the sidebar on the left for a table of contents.

couldn't find what you needed? suffer with me! see the How 2 Nix section in this repo.

✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓

Tips and Tricks

formerly known as tops and bottoms

This section outlines things that I've learned from various sources and some pure guesswork.

To learn Nix is to learn to suffer, and to learn the way of numbing the pain

— Cassie circa. 2023

There might be more undocumented things. Interesting things are usually marked with #‍ HACK:.

Of course, I might completely miss stuff. in that case, feel free to open an issue.

To get started, look at the sidebar to the left.

Overriding Packages

The nix pill confused me and i thought i had to make overlays to do overrides but no

in packages (i.e. environment.systemPackages), just do

{pkgs, ...}: {
  environment.systemPackages = with pkgs; [
    (discord.override {withOpenASAR = true;})
  ];
}

This works as well

security.sudo.package = (pkgs.sudo.override {withInsults = true;});

Overlays

overlays are useful when you want to refer to a package globally, or to fix a broken package locally.

you might also want to use overlays when something hasn't made it into nixos-unstable or whatever you're on yet, but you desparately need said thing.

the gist of overlays is as thus:

overlay is just final: prev: {} functions

dumbed down idea is that you do pkg = prev.pkg.override and refer to everything else from final

idea is like final = prev // overlay(prev, final)

(it's a recursive definition)

(poorly made) example overlays can be found here

currently in-use and slightly better overlays can be found in this repo! head over to /overlays to see them.

note: replace self: super: with final: prev: for consistency

UPDATE: we don't really use overlays anymore. If you'd like an example, please reach out and we can add some here.

concept and quote content by @natsukagami

Overlaying python packages

In some situations a python package may be bugged. This might have been fixed upstream by Nixpkgs devs, but has not reached nixos-unstable or whatever.

While overriding normal packages is relatively straightforward, doing so with python is most definitely not.

We have done this recently with the help of Scrumplex (thank you!) because a package was broken on nixos-unstable. Someone made a fix and it was merged, but it has yet to make it to nixos-unstable. This was blocking our systems from building so we decided to just say sod it, we're doing this.

Again, overriding simple packages that are not inside any package groups is wildly easier than this operation. Since not every package group is the same, this sample only focuses on Python because we only have experience with that.

Copy-paste the new package definiton next to the place where you're defining the overlay. We will be referencing it as ./package.nix.

final: prev: {
  # this does not work because the package uses python3Packages. this is defining standalone package.
  pyscard = prev.python3Packages.callPackage ./package.nix {};

  python3 = prev.python3.override {
    self = final.python3;

    # to their credit we do have this thing here which was already great
    packageOverrides = (final': prev': {
      # we cannot use final'.pkgs.callPackage here because it's missing buildPythonModule or something.
      pyscard = final.python3Packages.callPackage ./package.nix {
        inherit (final.darwin.apple_sdk.frameworks) PCSC; # apple carp
      };
    });
  };

  # probably some `rec` carp
  # IMPORTANT: you need this! this is needed to let nix know we want to use our overrided python3 package earlier.
  #            if you don't add this, you will still be building the old package like nothing changed at all.
  #            Yes, nix is this sad.
  python3Packages = final.python3.pkgs;
}

A full example is accessible here.

"Global"/Extra Options

a way of passing additional options "globally" to modules is by using extraOpts.

in nix flakes, this is accomplished by using specialArgs in nixosSystem.

for example, check out these few lines in our flake.nix: [source]

# note: unrelated attributes stripped and removed.
# note2: this code is now out of date from our code, but can still be referenced.
{
  outputs = { ... }:{
    nixosConfigurations = {
      koumakan = lib.nixosSystem {
        specialArgs = {
          _utils = (import ./global/utils.nix) { inherit pkgs; };

          someOtherArg = {thisCanBe = "LiterallyAnything";};
        };
      };
    };
  };
}

With this, you can now do this in other imported nixos modules.

{ someOtherArg, ... }: {
  users.users.${someOtherArg} = {};
}

this avoids the horror of import ../../../utils/bar.nix; and various other things.

refer to nixpkgs:nixos/lib/eval-config.nix and nixpkgs:lib/modules.nix#122 for more info

pointers by @natsukagami

using sops-nix or other stuff to pass big chungus files to services with DynamicUser=true

afaik this is not possible.

The option that makes the most sense, LoadCredentials only supports files up to 1 MB in size. (relevant documentation (systemd.exec(5)))

Without that option, we are only left with giving a user access to the file somehow.

Doing that directly via systemd is not possible either. We cannot get the dynamic user's id in a ExecStartPre hook with the + prefix to chown the file. The user is ran with root privileges and there are no signs of the final ephemeral user id. the same happens with ones prefixed with !.

cassie in marisa in ~ took 1s
✗ 1 ➜ systemd-run -pPrivateTmp=true -pDynamicUser=true --property="SystemCallFilter=@system-service ~@privileged ~@resources" -pExecStartPre="+env" -pPrivateUsers=true -t bash

Running as unit: run-u1196.service
Press ^] three times within 1s to disconnect TTY.
LANG=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin
LOGNAME=run-u1196
USER=run-u1196
[...]
^C%

cassie in marisa in ~ took 2s
➜ systemd-run -pPrivateTmp=true -pDynamicUser=true --property="SystemCallFilter=@system-service ~@privileged ~@resources" -pExecStartPre="\!env" -pPrivateUsers=true -t bash

Running as unit: run-u1200.service
Press ^] three times within 1s to disconnect TTY.
LANG=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin
LOGNAME=run-u1200
USER=run-u1200
[...]
^C%

cassie in marisa in ~ took 2s
➜ systemd-run -pPrivateTmp=true -pDynamicUser=true -pSystemCallFilter=@system-service -pSystemCallFilter=~@privileged -pSystemCallFilter=~@resources -pExecStartPre="\!bash -c 'echo \$UID'" -pPrivateUsers=true -t bash -c "ls"

Running as unit: run-u1236.service
Press ^] three times within 1s to disconnect TTY.
0
^C%

cassie in marisa in ~ took 4s
➜ systemd-run -pPrivateTmp=true -pDynamicUser=true -pSystemCallFilter=@system-service -pSystemCallFilter=~@privileged -pSystemCallFilter=~@resources -pExecStartPre="+bash -c 'echo \$UID'" -pPrivateUsers=true -t bash -c "ls"

Running as unit: run-u1241.service
Press ^] three times within 1s to disconnect TTY.
0
^C%

So now, we are left with the only option, which is to create a non-ephemeral user, assign it to the unit and disable DynamicUser. This step is a little involved, you will have to add a user option to the service and forcibly disable DynamicUser.

I opted to replace the entire module file with my own under a different name, as I had to fix a mistake in it anyways. Here's the link to the modified source file. For clarity's sake, this is the diff of the changes made.

Misc tips

This page contains stuff that I couldn't be bothered to move to the new format is probably outdated or just short tips.

previously: tops and bottoms

@ (at) syntax

very simple.

args@{a, b, c, ...}: {
  # args.a and a are the same
  some = "value";
}

nginx regex location

{
  locations."~ \.php$".extraConfig = ''
    # balls
  '';
}

from nixos wiki

adding a package with an overlay to a package set

for package sets with a scope, you will have to do something like

final: prev: {
  nimPackages = prev.nimPackages.overrideScope (final': prev': {
    sha1 = final'.callPackage ./sha1.nix {};
    oauth = final'.callPackage ./oauth.nix {};
  });
}

There's an alternative method that i used to use here:

https://github.com/soopyc/nix-on-koumakan/blob/30e65402d22b000a3b5af6c9e5ea48a2b58a54e0/overlays/nim/oauth/default.nix

however i do not think that's the best way lol

what the hell is an IFD??

IFD stands for import from derivation.

nixos/nixpkgs really need better and significantly less scattered documentation while improving manual readability.

Useful links

Builtin stdlib functions search engine: https://noogle.dev/

Pitfalls

"There are pitfalls in this language???!??!?"

-- The uninitiated

importing nixpkgs with an empty attrset

ever had this in your flake.nix

{
  outputs = { nixpkgs, ... }@inputs: let
    pkgs = import nixpkgs {};
    lib = nixpkgs.lib;
  in {
    # ...
  };
}

... and got fucked with this?

error:
       … while checking flake output 'nixosConfigurations'

         at /nix/store/lz2ra1180qfffmpwg41jpyg1z602qdgx-source/flake.nix:50:5:

           49|   in {
           50|     nixosConfigurations = {
             |     ^
           51|       koumakan = (import ./systems/koumakan { inherit pkgs lib inputs; });

       … while checking the NixOS configuration 'nixosConfigurations.koumakan'

         at /nix/store/lz2ra1180qfffmpwg41jpyg1z602qdgx-source/flake.nix:51:7:

           50|     nixosConfigurations = {
           51|       koumakan = (import ./systems/koumakan { inherit pkgs lib inputs; });
             |       ^
           52|     };

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: attribute 'currentSystem' missing

       at /nix/store/5c0k827yjq7j24qaq8l2fcnsxp7nv8v1-source/pkgs/top-level/impure.nix:17:43:

           16|   # (build, in GNU Autotools parlance) platform.
           17|   localSystem ? { system = args.system or builtins.currentSystem; }
             |                                           ^
           18|

just don't!!!11 remove the pkgs definition. (note that this only applies to pkgs = import nixpkgs {};)

explanation

you shouldn't ever really import nixpkgs with an empty attrset either

that causes it to fall back on guessing your system using builtins.currentSystem, which is impure and so not allowed in pure evaluation mode

—- @getchoo

Utility Functions

In this section you will find various utility functions available in this flake.

You are free to use them as you wish if you find them useful.

Watch out! Here's a hint box!

Getting started

You first need to import this flake as an input.

If you don't know how to do so, you should not be here. Please refer to various Nix documentation first, then come back. Using these utilities when you're just starting out causes unnecessary pain later on when it doesn't match your needs.

For NixOS users, it is possible to make the utils module "globally" available in your NixOS configuration modules. To do so, please refer to Tips/"Global" Options.

We are dogfooding on these functions ourselves so they should be relatively error-free. If you encounter unexpected behavior though, do reach out and open an issue/send us a message. We won't bite. Promise.

_utils.mkVhost

freeformAttrset -> freeformAttrset

make a virtual host with sensible defaults.

pass in an attrset to override the defaults. the attrset is essentially the same as any virtual host config.

Example

services.nginx.virtualHosts."balls.example" = _utils.mkVhost {};

_utils.mkSimpleProxy

{port, protocol, location, websockets, extraConfig} -> freeformAttrset

make a simple reverse proxy

takes a set:

{
  port ? null,
  socketPath ? null,
  protocol ? "http",
  location ? "/",
  websockets ? false,
  extraConfig ? {}
}

Provide either a socketPath to a UNIX socket or a port to connect to the upstream via TCP. Note that both of these options are mutually exclusive in that only one can be specified.

It is recommended to override/add attributes with extraConfig to preserve defaults.

Items in extraConfig are merged verbatim to the base attrset with defaults. They are overridden based on their priority order (i.e. via lib.mk{Default,Force,Order}).

_utils.genSecrets

namespace<str> -> files<list[str]> -> value<attrset> -> attrset

generate an attrset to be passed into sops.secrets.

Example

{ _utils, ... }:
let
  secrets = [
    "secure_secret"
    # this is a directory structure, so secrets will be stored as a file in /run/secrets/service/test/secret.
    "service/test/secret"
  ];
in {
  sops.secrets = _utils.genSecrets "" secrets {}; # it's recommended to use a namespace, but having none is still fine.
  # -> sops.secrets."secure_secret" = {};
  #    sops.secrets."service/test/secret" = {};
  sops.secrets = _utils.genSecrets "balls" ["balls_secret"] {owner = "balls";};
  # -> sops.secrets."balls/balls_secret" = {owner = "balls";};
}

See https://github.com/soopyc/nix-on-koumakan/blob/b7983776143c15c91df69ef34ba4264a22047ec6/systems/koumakan/services/fedivese/akkoma.nix#L8-L34 for a more extensive example

_utils.setupSecrets

attrset<nixos config attr> -> {namespace<str> ? "", secrets[list<str>], config ? freeformAttrset} -> secretHelpers

This is a higher-level setup that wraps around _utils.genSecrets and provides some additional helper functions. Usage of this function should make more sense than just using genSecrets.

NOTE: does not support overriding config for only 1 path. might implement when demand arises.

The definition of secretHelpers is defined as follows:

secretHelpers = {
  generate    = {}; # => {sops.secrets.* = <sopsConfig>} (inline module)
  get         = path: ""; # => actual path of the secret, usually /run/secrets/the/secret

  placeholder = path: ""; # => placeholder string generated by sops-nix, for that secret path to be used in templates.
  getTemplate = file: ""; # => actual path of the template, realized at activation time, similar to the get function.
  mkTemplate  = file: content: {}; # => {sops.templates.* = ...;}
  #             ^ => filename of the template. can be any arbitrary string.
}

Example

{ _utils, config, ... }: let
  secrets = _utils.setupSecrets config {
    namespace = "balls";  # for us, the namespace is just the top level element in our secrets yaml file.
    config = {
      owner = "jane";
    };
    secrets = [ "my/definitions/gock" "my/sizes/gock" ];
  };
in {
  imports = [
    secrets.generate
    (secrets.mkTemplate "my-secret.env" ''
      MY_GOCK_SIZE=${secrets.placeholder "my/sizes/gock"}
    '')
  ];

  some.service.settings.gock.file = secrets.get "my/definitions/gock";  # resolves to the path of balls/my/definitions/gock.
  some.service.settings.envFile = secrets.getTemplate "my-secret.env";
}

_utils.mkNginxFile

{filename<str> ? "index.html", content<str>, status<int> ? 200} -> {alias<str>, tryFiles<str>}

Helper function to generate an attrset compatible with a nginx vhost locations attribute that serves a single file.

Example

Without filename

services.nginx.virtualHosts."example.com".locations."/" = _utils.mkNginxFile {
  content = ''
    <!doctype html><html><body>We've been trying to reach you about your car's Extended Warranty.</body></html>
  '';
};

With filename

services.nginx.virtualHosts."filename.example.com".locations."/filename" = _utils.mkNginxFile {
  content = "the filename doesn't really matter, but it's there to help you figure out where your things are";
  filename = "random.txt";
}

_utils.mkNginxJSON

filename<str> -> freeformAttrset ==> attrset

Simple wrapper around mkNginxFile that takes in an attrset and formats it as JSON.

Note that the function signature is different in that it doesn't take in only one attrset. This may change in the future.

Example

services.nginx.virtualHosts."balls.org" = _utils.mkVhost {
  locations."/" = _utils.mkNginxJSON "index.json" {
    arbitraryAttribute = "arbitraryValue";
    doTheyKnow = false;
  };
};

Ports

This section mainly focuses on our existing port definition stuff. We try to not use ports as much and to use unix sockets, but sometimes it's just not possible.

Note: most of this document focuses on koumakan.

Defined port ranges

• 20xxx: Prometheus/Metrics
  • 2009x: core metrics, node metrics
  • 201xx: service metrics
  • 21000: VMAuth (special case as this is not strictly metrics but a proxy)

• 3xxxx: Service ports
  • 35xxx: exposed docker container ports

External Untracked Files

due to the required secure nature of these files, we are unable to include thses sets of files/directories in this repository.

• -r-------- /etc/lego/desec: acme credentials
• drwx------ /etc/secureboot: secureboot keys
• -r-------- /v/l/forgejo/data/jwt/oauth.pem: forgejo oauth jwt private key
• -r-------- kita:/etc/radicale/users: radicale user htpasswd mappings

changelog

This section will only list removals.

as of commit 8501880 (850188052ea0968e7eb96724c2027ad998cbbefb)

• nitter/guest_tokens.json managed in-tree

Certificates presets

{...}: {
  gensokyo.presets.certificates = true;
}

This enables and set some ACME related configurations to a common value.

This requires the following secrets to be set:

lego:
    cf_token: # generate from cloudflare

Nginx presets

{...}: {
  gensokyo.presets.nginx = true;
}

This enables nginx and related default configurations.

VictoriaMetrics presets

{...}: {
  gensokyo.presets.vmetrics = true;
}

This enables vmetrics and some default configurations. Afterwards, you can add new scrape configs like below.

{...}: {
  services.vmagent.prometheusConfig.scrape_configs = [{
    job_name = "nginx";
    static_configs = [{targets = ["localhost:${builtins.toString config.services.prometheus.exporters.nginx.port}"];}];
    relabel_configs = [{
      target_label = "instance";
      replacement = "${config.networking.fqdnOrHostName}";
    }];
  }];
}

Prerequisites

You need to do the following things when adding a new host.

Secrets

Include the follow secret configuration.

vmetrics:
    auth: # openssl rand 129 | base64 -w0 | tr "/=+" "-_."

Then add to koumakan.