About

A page explaining the content and implementation behind the website

This page serves as a much longer, more detailed account of the technologies and software stack this server (as in the one hosting the website) is using compared with the brief descriptions present on some of the other pages.

Introduction

As discussed on the site's Index page: this site doesn't really have a purpose—it more or less only exists because it can (and the fact that it takes minimal extra effort to actually run / host), not as some grand project or useful tool; following on from that, the reason I had to have a reverse proxy setup in the first place (as alluded to in the site's index) was because I needed to route traffic to the individual internal webservers / services depending on the specific subdomain each request was intended for, as well as some other requirements mentioned in the following section.

Nginx

Nginx (specifically the open source version) is the reverse proxy being used here to handle all the incoming HTTP / HTTPS traffic hitting ports 80 and 443 respectively (even if all plain unencrypted HTTP requests just get sent a 301 redirect to the HTTPS version). While the primary purpose served by Nginx is to proxy the traffic to internal ports and sockets (which requires reading the HTTP headers to determine which of the services to send the requests to), that's not all it does—it also handles the terminating of SSL and the buffering of data, which are important in their own right as described in the following paragraphs.

While i could let each of the individual services handle SSL encryption and certificate renewal themselves, this would be a nightmare to maintain and run, not because i necessarily distrust any of the services (although letting each of them have one of my SSL certificates wouldn't be particularly secure), but because having everything managed centrally all in one place makes this significantly easier (especially considering that there's a native NixOS module for it) and allows the configuration to be relatively straightforward thanks to how well NixOS (we'll talk about it shortly) integrates with Lego and Let's Encrypt to obtain and manage SSL certificates signed by a trusted CA. This allows each of the services to just receive a regular plaintext HTTP stream directly from Nginx and not have to deal with the encryption themselves.

Nginx also buffers the data being sent to and from the backend services so that all the data is sent to the client or webserver at once, instead of the service having to wait for a slow connection to send or receive the full payload, potentially slowing down or hanging it from all the waiting. Nginx also deals with compressing most uncompressed MIME types like UTF-8 encoded HTML or CSS files to reduce page load times, although this is mainly for when serving static content.

Finally, it's worth noting that just serving static files from disk also allows for another powerful optimisation, where the data living inside those files can be streamed directly from the disk (or OS page cache) to a network socket via a dedicated syscall as part of the Linux kernel, avoiding the expensive operations involved in copying the data from the kernel to the userspace Nginx worker process, then back again through another syscall—adding yet more overhead due to the constant context switches between the kernel and userspace for every chunk in the file sent; this is why the site is built using a custom Python script to compile a bunch of unique content into a list of static files that contain a lot of redundant code, so that the single directory containing the files (as well as some other required assets) produced from the package can be directly served by Nginx with incredible performance due to not having to template the files at runtime.

NixOS

The server itself is running NixOS, which is a Linux distro built on top of the Nix package manager for declarative package builds and system configurations, among other things; this allows the entire system setup to be managed inside a single Git repo and completely rebuilt on another machine or blank disk without having to manually track changes or hunt down specific configuration files manually, thus preventing old unused packages or files from accumulating on disk without an easy way to remove them without reconfiguring the entire system from scratch.

The server is also a VPS running on a cloud provider instead of being run on physical hardware, mainly because I didn't want to have to deal with the Raspberry Pi's hardware running something that isn't Raspberry Pi OS (formerly Raspbian) anymore, and as I wanted a static IPv4 and IPv6 address from a datacentre to not have to point my domain at my main residential IP and run everything off of that.

The services

This section documents some of the other things running on the server and bits about their setup.

Naturally the server requires an SSH server running to access the thing in the first place (because I'm not managing it over the recovery serial connection), so is running the standard OpenSSH setup with some hardening applied. This also significantly improves security as now the only way in is via SSH key authentication where all the user accounts including root don't have passwords so can't do password based authentication at all (with the root user locked anyway), making accessing a shell only possible via the SSH daemon, even with a login prompt available. Having an SSH key used for logging into the remote server and a secure way of storing it locally inside of KeePassXC via the SSH agent makes it relatively simple to also use it for Git commit and tag signing (although with a different key), so code and changes can also be verified on the server and by anyone with a copy of the public key.

Compared to the rest of the system, setting up Forgejo was super simple as NixOS has first class support for it via the services.forgejo option, so it was just a case of enabling it with services.forgejo.enable = true;, setting some basic Forgejo options in the services.forgejo.settings block, poking a hole in the firewall via networking.firewall.allowedTCPPorts so the embedded SSH server is reachable (as it can't be proxied by Nginx), and finally pointing Nginx to Forgejo's HTTP Unix socket with services.forgejo.settings.server.PROTOCOL = "http+unix";, proxyPass = "http://unix:/run/forgejo/forgejo.sock"; and users.users.nginx.extraGroups = [ "forgejo" ];.

To get Seafile working on the server without a native NixOS module available (unlike with Forgejo where this is very well supported) I has to run it inside of Podman via the Docker deployment method, as Podman is a drop in replacement for Docker that supports the same OCI-compliant containers / images that Docker uses, while having better security as it's a daemonless tool that instead relies on Systemd to manage container life cycles, making it the preferred backend to use and the default on NixOS. The files and guide provided by Seafile to run it using Docker assumed the use of Docker Compose, which I wasn't using—so instead I had to translate all the provided compose files into the equivalent Nix code for the virtualisation.oci-containers option, then figure out how to securely pass down environmental variables containing secrets as it's done differently to how it's done in Docker Compose, and then finally hardcode the containers' internal IPs as port forwarding to localhost decided to break...

Lastly, there are a few (or even no, this is likely to change) other, much smaller services hosted here that aren't mentioned anywhere, either because they're too small to deserve a mention, new / likely to be removed, or because I don't want to reveal their existence (and they don't have a publicly accessible webpage to link to).

Remarks

Similar to the Notes section on the Index page.

This site uses a lot of Em Dashes (—), which I initially wanted to format as " - " using a regular dash or Hyphen (-), but that's technically wrong and the grammatically correct version is "—" with no spaces, and even though it looks weird, I eventually opted for styling it that way. Em dashes can also be a big sign of AI generated writing, which is a shame as some people intentionally omit them from their writing for the sake of making the text look more "human", despite the fact that it's perfectly fine English grammar, so I felt I should at least acknowledge their use and point out that they don't mean the words themselves aren't my own.

Just to add: there's also at least one Oxford comma in here, where most of the previous points still apply.

You also might notice that the writing style used throughout the text isn't particularly consistent, where some parts are significantly more or less formal or technical than others (even between consecutive paragraphs), and where some of it might just be incorrect (either grammatically or factually). The reason for the subtly shifting tone is because these documents don't really have a concrete definition for how they should be written, so it remains mostly formal but some sections are allowed to slip, but they should at least remain in perfect English so nothing like using a plus symbol as a conjunction. The structure may also be somewhat strange as most of the website was written out of order and repeating specific words, phrases, or punctuation close together has been attempted to be avoided so some parts may be slightly awkward, as well as some rules changing midway through writing making some grammatical styles or HTML structures potentially differ to the rest of the text. Also adding that some words or spellings may get unintentionally conflated between British English and American English, where the canonical dialect (yes that's the correct term) for this site should be British English, as stated in the HTML header.

I also wanted to maybe include the date that each page was generated or updated in the footer, but because of how Nix builds it's packages where the time inside the build environment is set to the Unix epoch (or midnight on the date 1/1/1970) and doing it at runtime via substituting a string with Nginx cancels the sendfile optimisation and incurs a performance penalty (as well as me being too lazy to update the timestamps manually), I decided to abandon this idea.