There’s No Place Like ~

Around the time the generative AI craze was revving up, I read that Google could use the content of files in Google Docs as AI training data. This alarmed me at the time, although the claim was exaggerated. It made me think about who ultimately had access to my data, from the personal to the sensitive. I wanted to be the one controlling my data, so I downloaded everything I could off Google and other cloud platforms. With everything now sitting on my hard drive, I started searching for alternatives.

After some research, Nextcloud came out on top as a popular alternative, and it was just what I was looking for as a solution. The prospect of setting up a server on an old Raspberry Pi 2 I had lying around sounded fun, so off I went. I’ve been tinkering with my home server ever since—almost four years now. I’ve since upgraded the hardware twice, I now use and Intel NUC, and I’ve switched around using Arch, Debian, and even Nix. There are plenty of things I’ve picked up along the way and many things I’ve yet to learn.

Quite the antique!

Before starting my self-hosting journey, the closest thing I had managed was a Minecraft server. I had many hiccups and bumps starting out, but we live and we learn. Access and administration have been huge areas of growth due to this ongoing project. I did have some experience working in remote environments, so SSH, containers, and the basics of LAN networking weren’t unfamiliar. The challenge would be getting things up and running on my own. Fortunately, with my prior experience with installing Linux and the numerous resources available, getting started turned out easier than I initially expected.

Once I began, what became my primary issue was accessibility. I was still in university at the time, and running a server on the school network was a no-go. My first—and in hindsight overly risky—thought was opening up the server for access over the internet. In taking this route, it became very necessary to learn new skills and software, particularly in the realm of security.

I started with hardening the system itself. Locking down the SSH configuration using Mozilla’s guidelines and setting up key-based authentication increased my security posture significantly with just a few changes. My password manager’s built-in key agent has made managing keys such a breeze. The simple act of disabling password authentication shuts down common SSH brute force attacks, the most common attempted attack I observed on my system logs. For hardening the rest of the OS, I used the CIS Benchmark provided for my operating system version. It provided information about the system and security implications I never would have come up with.

System hardening, though important, wasn’t my biggest worry since ideally I would be the only one accessing the server or its services. Preventing access from bad actors on the outside network was a big priority. The first issue was deciding how to expose the server in the first place. I knew I had to forward ports from my router, but I didn’t want to set up forwarding rules for every service I planned to run. Not only would this be annoying eventually, it would largely increase the attack surface of the network. Turns out, it was one of those problems with a simple solution. I learned about the concept of reverse proxying. I would only need to open up ports 80 and 443 to web traffic and proxy it to the correct internal ports. My proxy of choice is Caddy. It’s simple configuration format makes it easy to create rules as simple or complex as you want. I may try my hand at NGINX or Apache in the future, but as for now, Caddy provides all the features I need without much hassle.

One perk of Caddy is its integration with authentication middleware. Some of the services I planned to run didn’t have built in authentication, and I would rather not leave these services open. I decided to go with Authelia to provide authentication in these circumstances and facilitate SSO with compatible services. I’ve also tried Authentik, another popular open-source user management service, but I decided to stick with Authelia as it scales down to my smaller use case better. Adding another layer of dedicated authentication to network facing services does supply peace of mind.

Once I knew what ports actually needed to be accessible, I could lock down the firewall. UFW is a go-to tool for configuring firewall rules and works well with other security services since it uses iptables as a backend. I initially used Fail2Ban to analyze the network traffic and ban IPs, but I have since switched to Crowdsec. It offers more robust security features and integrations, great support, and is highly configurable. Getting detailed information on blocked IPs provides very useful insights on ways to further tailor firewall rules.

With security and availability remaining a top priority, I have since changed the way I access services outside my home network. The solution happened to be sitting in front of my face: VPNs. I’m surprised I didn’t think of using one sooner, considering I spent so much time explaining to family and friends how VPNs actually work and why the metric tons of ads and YouTube sponsorships about their function were misleading. Tailscale is what I currently use to provide this functionality. I know it’s another SaaS, the very thing I’ve been trying to avoid, but more on that later. Since I’m on the same LAN as my server most of the time, I don’t find myself connecting to the VPN often anyway.

However, some new challenges presented themselves when I switched to this new structure. The first and most simple to solve was domain resolution. The domain no longer pointed back to my public IP. One option would be to add records pointing to the private address of the server, but that didn’t seem very practical and unnecessarily exposes information about my network. The solution to this problem was setting up a DNS server. With Pi-hole and AdGuard Home being popular choices in this space, I decided to GO (haha) with AdGuard. The nice interface, easy DNS rewrites, access to cutting-edge protocols is what swayed me. Moreover, it’s not written in PHP, which I have my own beef with. There’s something satisfying about seeing requests sent to analysis and ad-serving domains blocked. With DNS working, it was time to tackle the next challenge: certificates.

I didn’t want to deal with my browser freaking out every time I tried to access a page, and even on a local network, encryption is important. In the prior setup, I had used certbot and Caddy to obtain certificates from Let’s Encrypt, but as far as I knew, it required at least forwarding ports 80 and 443 to function. Turns out, yet again, I wasn’t alone in my conundrum and there were solutions ready. Certificates be obtained by using a DNS challenge. This only requires owning a registered domain to work. I had to build a custom version of Caddy to do this, but it’s not as complicated as it sounds. With that, I was able to secure certificates to encrypt my local web and DNS traffic.

# The scary custom build in question
ARG VERSION=2
FROM caddy:${VERSION}-builder AS builder
RUN xcaddy build --with github.com/caddy-dns/cloudflare
FROM caddy:${VERSION}
COPY --from=builder /usr/bin/caddy /usr/bin/caddy

As it stands, this is the state of my infrastructure as of writing. I do have some future goals and projects in mind. The main one currently is getting my own VPN set up. This will require more security considerations, but it allows me to move away from 3rd party software. One option, however, could be using Headscale, an open-source implementation of the Tailscale server. Since the server is the only closed source aspect of Tailscale, replacing it with a self-hosted version would remediate my qualms.

With outside access requiring my infrastructure to function rather than a dedicated service, another goal is to set up the Pis I still have around to create a high availability environment for the most critical services. Having uptime monitors and failover options would certainly make things less frustrating when something stops working when I’m out of town.

Moving to a container based, multi-machine setup gives me the ability to experiment with Kubernetes. Even at a small scale, getting hands-on experience with the system will help me understand scaling and networking better.

The last project consideration I currently have is using my infrastructure as a sort of cybersecurity lab. Setting up and experimenting with SIEMs (currently starting with the ELK stack), virtual machine and container management, and learning more about Splunk are all on my list. I also want to write more automation scripts using python and bash to make recovery easier when something inevitably goes wrong with me messing around.

Having your own server to mess around with is not only useful but loads of fun. There are people building apps to perform common tasks and work in niche fields. With a server, you can tailor what services you run and what experience you have. You can even deploy your own app! Experimenting, failing, and trying again is all part of the process. The level of control and privacy is a major benefit in a world that attempts to harvest your data no matter what. Using your home network to its full potential should be fun and encouraged even for non-technical folks. By sharing my experience, I hope to get more people excited to join the homelab and self-hosting community.