Posts tagged: homelab Archive

The Faulted Disk: harbor Replacement Writeup

2026-10-06T21:30:00Z

The sequel to panicking-led-to-losing-my-desktop — this time the monitoring actually caught the disk dying, and nothing was lost.

What happened #

harbor is my replica pool — a 10.9T mirror (2x 12TB) that receives syncoid snapshots from tank. One side of the mirror, a Seagate Exos ST12000NM0127 (serial ZJV4QFLB, /dev/sdb), went FAULTED with 14 read + 22 checksum errors.

mirror-0                              DEGRADED     0     0     0
  ata-ST12000VN0008-2PH103_ZTM0NFDW  ONLINE       0     0     0
  ata-ST12000NM0127_ZJV4QFLB         FAULTED     14     0    22  too many errors

The IronWolf mirror side carried the pool — No known data errors. ZFS redundancy did exactly its job.

The difference from last time #

Last failure: no monitoring, found out by accident months later, desktop died.

This failure: SigNoz + node-exporter’s ZFS collector → node_zfs_zpool_state{state="degraded"} → alert rule → Gotify → my phone. The gotify notification fired before I knew anything was wrong.

Diagnosis #

Before zpool clear or replace — check SMART. smartctl-exporter already scrapes all disks into SigNoz, so I didn’t even need sudo:

Reallocated_Sector_Ct raw = 3024 and counting
Offline_Uncorrectable value/worst still 100 but raw errors climbing
SMART overall: still PASS (SMART’s overall bit is conservative until threshold)

3k+ remapped sectors is a platter going bad — not a cable blip. Verdict: replace, don’t clear.

The swap (the annoying part) #

Hot-plug is never as smooth as it should be:

zpool offline harbor <old> to stop writes to the dead drive — actually skipped; drive fell off the bus on its own
New IronWolf ZTN1CQ07 in hand → plugged into ghost’s SATA → didn’t enumerate
Forced rescan (echo "- - -" | sudo tee /sys/class/scsi_host/host*/scan) → nothing
Moved it to the old drive’s port → nothing
Sabrent USB dock on aurora → dock enumerated as 0B device, no disk behind it → reseated + replugged → drive spun up and appeared: /dev/sdd, 10.9T, old ZFS partition table on it (used drive — part1/part9 layout)
Back to ghost, direct SATA → enumerated as sdb

Lesson: “spins up but doesn’t enumerate” on direct SATA + dock-shows-0B = seating/power problem, not DOA. The drive was fine all along.

The replace #

sudo zpool replace harbor \
  ata-ST12000NM0127_ZJV4QFLB \
  ata-ST12000VN0008-2PH103_ZTN1CQ07

Resilver: 8.80T in 18h38m, 0 errors (~154M/s). Pool stayed online and usable the whole time.

One gotcha: after resilver, zpool status showed errors: 1 data errors — but zpool status -v showed an empty error list. The corrupted data was already repaired; the counter was just stale. sudo zpool clear harbor → clean.

The monitoring that made this possible #

Wired during this session (all now in SigNoz):

node_zfs_zpool_state — pool health (node-exporter zfs collector)
smartctl-exporter — SMART attributes incl. reallocated sectors (the early-death signal)
zfs-metrics.sh textfile bridge — sanoid --monitor-* exit codes, zpool error counters, scrub ages, resilver %, syncoid last-success
Alert rules → Gotify → phone

The replication alerting even proved itself live: syncoid failed twice during the resilver window and I got paged on both. Turned out to be send-time contention — self-healed once resilver finished — but the notification path works end to end.

Remaining homework #

Buy the replacement spare (the shelf’s empty now)
10Fold datasets are garbage + have zero snapshots — destroy
Every scheduled job emits cron_last_success_epoch now — if a job dies silently again, I’ll know

Same failure as April, opposite outcome. Redundancy did the protection; monitoring did the detection. You need both.

Co-authored with Devin, who ran the monitoring stack, the SMART diagnosis, and the alerting loop.

💭 Docker | Atuin Docs

2025-11-22T20:58:59Z

Here's my thought on Docker | Atuin Docs --- A 2-fer... So I want to host atuin and share shell history across my machines on my tailnet I think... it's such a nice interface but I barely use it so I'm sure the features will improve some aspects of my life. And secondly, this section on using systemd is interesting... I hadn't thought to use systemd to schedule container upgrades - sounds incredibly trivial and probably a widely practiced thing now that I say it out loud... Saving here for implementing something... sometime... --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Homelabbing Realization - Configs and Git

2025-11-13T05:48:15Z

Realization #

I’m back to brainstorming migrations away from Docker Compose to something that solves multi-node networking and secrets but I don’t want to go to k8s… so my options are probably Docker Swarm or Nomad. As I wrestle mentally here I was thinking about one of my issues with multi-node setup is that I still keep everything out of one git repo, so if something gets managed remotely then I get out of sync. Best example is dashy, Due to some constraint of wanting to be as close to zero-day deployment-ready as possible, I think I overcomplicate some of the simple stuff - like it’s ok to just have to put a file on a server for a container to start the first time… That doesn’t mean ansible is a requirement, in fact lately I’ve just leaned into just recipes. But even then, I cornered myself into tracking my dashy config in git, so I’m hesitant to ever update it in the browser cause I’ll have to redownload the config, reconcile the formatting differences in my repo (there’s a reason for this that a linter doesn’t ssolve…), etc…. but in all of my brainstorming I just decided that my dashy config doesn’t have to live in git… it can just be on the zfs storage, which is backed up, and configured via the app itself… the config can simply be apart of the data and container lifecycle, it doesn’t have to be apart of the infra lifecycle.

RepoFlow Robot User

2025-11-10T09:24:25Z

I’m trying to lean into repoFlow at home, and model some slightly better patterns than open borders to artifacts across all my services. As I’m getting repoflow going I see that I can make users in the console, but then I do have to invite users to workspaces with a manually generate invite link.

The workspaces an abstraction layer, and in each workspace can be many kinds of repositories. For my purposes I’ll probably have 2 workspaces, even though 1 would almost certainly suffice.

There doesn’t seem to be a way to add a user to a workspace from the admin setting so for a while I was pretty confused why I couldn’t add my robot user to a docker repository in my main workspace…

After generating the invite link and accepting though then I see in the workspace settings a simple click to give the robot user access to all repos (if you want)

Can manage allows the user to mutate tags (ie. push latest docker tags). If the user only has Can deploy then they can only create new artifacts (ie. not mutate any tags)

Proxy Pull Docker Images From Self-Hosted Container Registry Through Self-Hosted Repoflow

2025-08-12T06:59:02Z

This post is a short write-up of an issue I had while exploring Reploflow - a super solid artifactory-esq replacement for the homelab (and enterprise!). I’ve been playing with it and have tried to setup a few artifact repos that I’m familiar with - docker and pypi.

PyPi #

This isn’t a post about repoflow, but I had an issue with pypi and Tomer was EXTREMELY responsive in helping to fix my issue and patch repoflow within a day of me reporting the issue via email!

Docker #

For docker I have a few options I just wanted to see work - obviously proxy pull from Dockerhub for ease, but also in the interim I’d like to proxy pull from my original container registry to avoid having to update any homelab config just yet. Now, obviously I’ll have to update all my build pipelines to start using repoflow, and my homelab configuration to reference the repoflow docker repository rather than my existing registry.

But there’s something to bear in mind if maintaining that middle registry - turns out a there’s a default namespace in dockerhub /library/ and repoflow expects this standard across any docker registry, so it’s pull paths fully resolve to an incorrect tag if you don’t take this into account in your existing infrastructure.

TLDR: if you tag an image myregistry/ubuntu:latest and push it then try to pull it back through repoflow then repoflow expects the image to be myregistry/library/ubuntu:latest. I think repoflow should support configurating the namespace/org for the images - but in the meantime I can go find all my images and update my build scripts to add /library/ to my tags… or while I’m doing that I can just push to repoflow directly… All in all, nice little lesson on namespaces in docker repos and third-party tooling.

💭 My Ultimate Self-hosting Setup

2025-07-25T19:56:38Z

Here's my thought on My Ultimate Self-hosting Setup --- great post on self-hosting. I think in a lot of similar ways and had a siar journey. There's things I want to accomplish as time goes on, looking for motivation to get a post like this going myself --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Using restic to backup my home directory

2025-07-16T09:12:26Z

# Intro I need to backup my personal $HOME to my NAS cause there's a lot in there, and mostly my git projects with .env files all over. Plus some docker data ## Why Not ZFS? GREAT QUESTION! It's because I struggled getting all the syncoid/sanoid requirements installed on my Aurora OS.... I like the immutable desktop trend, but I don't understand rpm-ostree enough to properly get all the lower level kernel stuff setup in a way I trust. So because I don't have ZFS at $HOME what's a boy to do? `rsync` would probably work fine especially if I [[rsync-like-a-pro]] but I figured I could cobble together some other tech and broaden my horizons... ## Enter restic Apparently [restic](https://restic.net/) has been around for a while, and you can run it in a container... awesome that's all I needed to hear... ## How? I'll use a docker compose stack for this... typically I think running `restic` right on the host is the way to go, but on my desktop I float between my host and distroboxes and to keep things as simple as possible, even though less efficient, I want to manage everything through containers as my homelab gets built out and eventually I'll have a more full-featured ecosystem. ## The Value You can find code below and an example repo that more or less is what I use for my desktop backup but that's primarily what I was after.... backing up the data on my desktop to my NAS in a way that: 1. I wouldn't have to "remember" how I was doing it - I accomplish this by having good git repo organization and README files that explain my own usage patterns 2. Tracked snapshots - I need some kind of snapshotting - `zfs` has spoiled me, and turns out `restic` has some amount of support for it 3. Running in docker would be ideal for now so that I don't need to worry about installing specific binaries across machine while I build out my patterns With this setup I get to backup my $HOME to my NAS which contains 1. docker volume info for all the AI workloads I run on my desktop and 2. all my sensitive info in my git repos is at least backed up in a way that I can recover `.env` files with relative ease... # Code Check out the [Example GH Repo](https://github.com/pypeaday/docker-compose-restic) but I'll drop key file contents here for a quick read if you're interested in some of the setup... but the blog post mostly ends here ## The Files We have a `docker-compose.yml` of course ```yaml services: backup: image: restic/restic:latest network_mode: host container_name: restic_backup hostname: restic-backup-runner env_file: - .env environment: # The location of the backup repository inside the container - RESTIC_REPOSITORY=/target # The location of the password file inside the container - RESTIC_PASSWORD_FILE=/password volumes: # --- Source and Config Mounts --- - "${BACKUP_SOURCE}:/source:ro" - "${RESTIC_PASSWORD_FILE}:/password:ro" - "./backup-and-prune.sh:/usr/local/bin/backup-and-prune:ro" # --- Persistent Data Mounts --- - "./.ssh:/root/.ssh:rw" - "./.cache:/root/.cache:rw" # Set the default entrypoint to our new script. This will be executed when # the container starts, unless overridden. entrypoint: ["/usr/local/bin/backup-and-prune"] ``` Your `.env` file will need to look like this ```bash ``# HOST related variables # The source directory to back up (absolute path) BACKUP_SOURCE=/home/nic # --- Restic Configuration --- # The file containing the restic repository password (absolute path on host) HOST_RESTIC_PASSWORD_FILE=/home/nic/projects/personal/homelab-mono/dataops/docker/.restic-password # The SSH private key to use for connecting to the NAS (absolute path on host) # It's recommended to use a dedicated key for this purpose. SSH_PRIVATE_KEY_FILE=/home/nic/.skm/ghost/id_rsa # Container Env Vars # the container makes the sftp connection using my credentials, but nonetheless the container needs them, so this isn't envrc stuff # --- SFTP/SSH Connection Details for the NAS --- SFTP_USER=nic SFTP_HOST=ghost # The path on the NAS where the restic repository will be stored SFTP_PATH=/tank/encrypted/nas/nic-home/ # --- Restic Configuration --- RESTIC_REPOSITORY=/target RESTIC_PASSWORD_FILE=/password ``` And then the backup script that the container will execute is something like this: ```bash #!/bin/sh set -e # Exit immediately if a command exits with a non-zero status. # Construct the repository path from environment variables passed by docker-compose REPO="sftp:${SFTP_USER}@${SFTP_HOST}:${SFTP_PATH}" # 1. Run the backup # ----------------- echo "--- Starting backup for ${BACKUP_SOURCE} ---" restic backup /source --verbose -r "${REPO}" echo "--- Backup complete ---" # 2. Clean up old snapshots according to the policy # -------------------------------------------------- echo "--- Pruning old snapshots --- (Policy: keep last 7 daily, 4 weekly, 6 monthly)" restic forget \ --prune \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 6 \ -r "${REPO}" echo "--- Backup and prune process finished successfully ---" ``` ## Systemd Finally there's a systemd unit and timer file in there so you can setup a systemd service for the backup ### service ```bash [Unit] Description=Run Restic backup to NAS using Docker Compose # We are running this as a user service, so we assume that the system-level # docker.service is already running. After=network-online.target [Service] Type=oneshot # Set the working directory to where your docker-compose.yml and .env file are located WorkingDirectory=/home/user/nas-backup/docker # The command to execute. We use the full path to docker-compose for reliability. # You may need to adjust this path if 'docker compose' is installed elsewhere. ExecStart=/usr/bin/docker compose run --rm backup [Install] WantedBy=default.target ``` ### timer ```bash [Unit] Description=Run Restic backup job daily [Timer] # Run daily at 2:00 AM OnCalendar=daily # Or uncomment for a specific time: # OnCalendar=*-*-* 02:00:00 # Run the backup immediately if the last scheduled run was missed (e.g., if the computer was off) Persistent=true [Install] WantedBy=timers.target ```

Jellyfin

2025-07-11T20:39:00Z

Jellyfin is the media server software I run at home

Ghost

2025-07-11T20:23:35Z

# Intro Ghost is my primary application server. These are the specs... > I wrote about some of the specs in [[homelab-journey-part-1#current-homelab-setup]] but here is where I will keep up to date information ## Why Why `ghost`? I don't know... I was going for a theme at one point it time, or several points rather... and after giving up on the theatrics of a fully consistent harry potter naming scheme across all my homelab entities, I decided to just come up with some short aliases that were easy to type. So `ghost` was born, and the backup server is `ghost-vault`. ## CPU I have a `AMD Ryzen 7 5700G with Radeon Graphics @ 16x 4.426GHz` in this bad boy... why this chip? Well I knew I wanted many cores, and I was previously on a Ryzn 5 3600, which was fine but I noticed that as I started to develop with containers more that I was going to want to take advnaatage of more cores... I also was pretty sure that the built-in graphics would be enough for the acceleration requirements I had in mind - which was primarily [[jellyfin]]. At this point in time, I think AMD was still beating Intel on most of the benchmarks I tended to care about (I was watching channels like Bitwit and LTT at the time of building this machine) So because of the moderate amount of cores and built-in GPU, I paid $229 for it according to my Amazon history, and it's currently priced at $174, so that's a pretty sweet deal if you're on AM4 and looking to upgrade in July of 2025 ## Memory I maxxed her out at 128 Gee Beez. I snagged that kit of Crucial that's all black that we've all seen on Amazon. I paid aboaut $90 for it, it's currently priced way high for some reason, which is unfortunate. It's been totally fine for me as far as memory performance goes - in that it's stable and I don't think about it... ## Boot I have a 1 TB NVMe in here to boot from. I think it's a Crucial P3 ## Storage Storage is the interesting bit, it's supposed to be the most rock solid but it's also been the most fluid for me from a hardware perspective, and as of my most recent case migration I think I'm in a good spot.. > The case is a Sagitatius 8-bay NAS Chassis from aliexpress I use [zfs](https://openzfs.github.io/openzfs-docs/Getting%20Started/index.html) and recommend you do to.... but this isn't a post about that My primary zpool is a 12TB ZFS Mirror, and I have another 12TB ZFS Mirror as an on-prem replica plugged into this same box. There is a 4TB drive that serves as frigate's media directory - I wanted a dedicated drive due to the write-intensive nature of the NVR. > I backup to an offsite box using syncoid ## OS I am currently running Ubuntu Server 24.04 I am eager for Ubuntu to get zfs in their release that contains the zfs data corruption with encrypted datasets but I don't expect to see it until 26.04 at the earliest _fingers crossed_. ## KVM I recently purchased a [jetkvm](https://www.jetkvm.com/) and it's exactly what I need for both my main server and my backup off-site... An amazingly simple and elegant solution to remote KVM. ## Applications Currently I use `docker compose` to manage practically everything. Stay tuned for more about those things I also have a few ansible playbooks for setting up my shell and some utilities on my computers, including the server. They're in [github](https://github.com/pypeaday/ansible-playbooks) Find more at [[the-homelab]]

The Homelab

2025-07-11T20:22:51Z

# The Lab This is the landing page for my homelab posts. It isn't a feed because I will write things here and link out to relevant parts of the homelab. I'll probably have notes about the hardware mostly here. ## Primary Application Server My main server is called [[ghost]] ## Current Homelab Setup > I copied this our of [[homelab-journey-part-1#current-homelab-setup]] for now to stub out what this page will link to ## Ad blocking | Pi-Hole Hardware: pi-3 I have a Raspberry Pi 3 hardwired in that is running pi-hole so that's been a part of my infrastructure now for a long time. ## Router | OPNSense !!! note "stats" CPU: i5-8700 Memory: 8GB of whatever was in the box Storage: 1TB SSD My router is an old Optiplex SFF 5060 I got from Amazon for less than $100. So I'm running a x86 system for my router with OpenSense on it. Just seemed it to be another home lab standard and I'd like to someday bring pi-hole over there or use AdGuard or integrate those two things maybe a little bit more completely but right now they are mostly separate. ## Networking | VLANs There's a managed switch in there for some VLAN tomfoolery for my cameras and an IOT network type of a thing. ## Networking | VPN | Tailscale They're on tailscale as well so my DNS happens over tailscale. ## My Desktop !!! note "stats" CPU: Ryzen 7 5700X GPU: Nvidia 3090 Memory: 64 GB Storage: 4TB SSD + 2 TB HDD for zfs backup OS: Universal Blue Aurora My daily-driver desktop is somewhat apart of my homelab now as I have a 3090 in there with 20 GB of VRAM so I can run some heavier LLMs for self-hosted AI workloads. Everything in that space runs in a container and then I manage docker containers with compose stacks and portainer for visibility. ### Applications Some of the apps I run here are ollama, open-webui, automatic1111's stable diffusion webui, whisper-webui

💭 Docker Brings Compose to the AI Agent Era | Docker

2025-07-11T11:49:41Z

Here's my thought on Docker Brings Compose to the AI Agent Era | Docker --- Seems like docker is leaning harder into compose - which is great for me as a heavy compose user. I had heard about some of the LLM enablements directly through docker desktop - for example taking fuller advantage of the host's compute power without paying the penalty of the nvidia runtime or something... I can't claim to have followed it all but I heard "run LLMs in docker directly for a bigger boost" and I'm in. --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

💭 restic · Backups done right!

2025-07-08T11:53:41Z

Here's my thought on restic · Backups done right! --- Welp, I need to be backing up my desktop home directory, specifically a directory of docker volume data... I thought duplicati made sense, but that mostly targets cloud-based backends, and rsync is great but I wasn't sure how i wanted to manage the job. Through a little AI chat I learned about restic and am working on a compose stack to run restic in a container to regularly backup my home directory via sftp to my NAS to then be swept up in my NAS backup workflow --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Thoughts To Nostr

2025-07-08T08:30:35Z

I am trying to use [[temporal]] at home for running automated pipelines. It's definitely overkill for my use case but this is America... So my first project is a system that takes my [[thoughts]] and posts them to [[nostr]] as I briefly discussed in [[testing-a-postiz-change-locally]]. I have temporal infrastructure setup in homelab-compose and then homelab-temporal should probably be homelab-temporal-pipelines or something, but it's the app code. I also have a repo homelab-social-media-pipelines - which has been consolidated down into homelab-temporal. once that all works I can destroy the homelab-social-media-pipelines repo NOTE: everything is moving to homelab-mono ## Issue 01 I am currently hitting a stupid python issue: ``` nic in homelab-temporal  main  ×2  ×4  ×7 via  v3.12.8(homelab-temporal)  (dev) 󰒄 ⬢ [devbox] ❯ python -c "from homelab_temporal.postiz.postiz_client import PostizClient" (homelab-temporal) nic in homelab-temporal  main  ×2  ×4  ×7 via  v3.12.8(homelab-temporal)  (dev) 󰒄 ⬢ [devbox] ❯ python scripts/postiz/simple_post.py Traceback (most recent call last): File "/home/nic/projects/personal/homelab-temporal/scripts/postiz/simple_post.py", line 3, in from homelab_temporal.postiz.postiz_client import PostizClient, process_for_nostr, process_for_x ModuleNotFoundError: No module named 'homelab_temporal' ``` very dumb venv problem... ## Resolution need to use `python -m` because the script is outside the project root? see [[2025-07-08-notes]] and the chatgpt notes for more on this but it works now ## Thoughts Order as of [[2025-07-11-notes]] I have the thoughts ordering correct - we post the oldest thoughts up to the newest ones. Now I want more intros for the notes so they don't get too repetitive ## TODOs [x] - destroy the homelab-social-media-pipelines repo [] - think about how to structure things more... monorepo in homelab-temporal might be hard to manage with AI tools due to context management... unsure though [] - create workflow and activities for thoughts to nostr... revisit diagram to see what you need to build out in the temporal repo [] - do something like this for github stars and then blog posts --- I need to update my templates for nostr og images using this as fodder ```html Nostr Design ``` ## Dockerizing I think I wnt to dockerize my workers and workflows... build them out of the `homelab-temporal` repo but then run them in compose stacks... that can get stitched together better but for now that'll probably be a good target. I have things to think about though... for example the blog builder needs my ssh keys, or an ssh key for a user... also git is a little mess up anyways - temporal [link](https://temporal-ui.paynepride.com/namespaces/default/workflows/blog-build-ee488b69-6cdf-4819-bccd-df183cf8ec13/01981101-e48b-712a-96db-328ce720c13c/history) ## Temporal I haev temporal workflows and stuff that all seem to work on [[ 2025-08-08-notes ]] I had issues with my nostr relay being too public, so I shut it down, reconfigured postiz and stuff and got it workign again, more restricted and in-memory database

💭 HomeBox

2025-07-07T10:48:12Z

Here's my thought on HomeBox --- I stumbled onto homebox this morning scrolling a newsletter and it looks like something I've been lightly thinking about for a while. I have wanted a way to track all the things in my house, but I haven't given it enough thought to come up with the specific manner in which to do it. I've seen other asset trackers but today this one excited me. I spun it up at home quick and it looks great - It'll take quite a lot of time to fill it out but here's a few features I love out of the gate: 1. the default setup seems very useful - pre-populated locations and tags for cataloging assets reduces the barrier to entry for me 2. it's pretty nice lookin 3. there's a label generator including qr codes that take you to the record in your homebox... I had an idea for something like this a while ago - qr codes to link to instructions for things around the house (like cleaning the oven or something). So it's neat to see that similar idea played out in another more useful way! --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

💭 Self-Hosting A Cluster On Old Phones | Hackaday

2025-07-06T11:30:17Z

Here's my thought on Self-Hosting A Cluster On Old Phones | Hackaday --- Came across a blog and article on using a mobile OS to self-host a cluster on old phones... I happen to keep several old android phones lying around and although I'm not presently super interested in figuring out an android/arm based homelab, I am excited for the next coming years when I'm sure it'll only get easier, phones will only be more powerful, and perhaps we'll be running full blown CV pipeline in k8s clusters running on Pixel devices! --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Github Star Sorter

2025-07-01T08:01:31Z

I wrote a simple app to sort, tag, and create tickets at home for my github stars… As of right now I have 1.3k stars and no convenient way to view them or remember which ones were ones I actually wanted to do something with. This simple UI makes scrolling the stars list easy, updates is easy, and the kicker for me is Kanboard integration so I can click a button from the stars UI and get a ticket made in my kanboard instance so that the work I want to do is in front of me and not hidden by Github’s shockingly HORRIBLE UI.

GH Link

temporal

2025-06-24T21:21:43Z

Temporal is a workflow orchestrator that I’m interested in

Traefik and gRPC for Temporal at home

2025-06-24T20:55:42Z

# Temporal Networking Woes Someday I will potentially write up some stuff about [[temporal]] but for now it's a workflow orchestrator that I'm interested in and I wanted to PoC it in my homelab by replacing a simple cron job to build my blog with a temporal workflow and activity. I was IMMEDIATELY thwarted by a networking error that feels crazy to me but maybe won't be to others... so enjoy the problem and if you're here from a Google search, I hope it's helpful. Getting started with Temporal I wanted to have the server and associated components on my server, but I expected to be running workflows and workers on another computer. For most of my apps I slap some [[traefik]] labels on and we're good to go. But I got a wild set of errors trying to run a worker on my desktop and connect it to the temporal server on my server over https.... it ended up being because I needed to setup a special protocol in traefik for the reverse proxy to appropriately proxy the grpc call. So I had a normal set of labels on my temporal container, went to run a worker and got this nasty work of art ``` RuntimeError: Failed client connect: `get_system_info` call error after connection: Status { code: Internal, message: "protocol error: received message with invalid compression flag: 73 (valid flags are 0 and 1) while receiving response with status: 500 Internal Server Error", metadata: MetadataMap { headers: {"content-type": "text/plain; charset=utf-8", "content-length": "21", "date": "Wed, 25 Jun 2025 02:12:44 GMT"} }, source: None } ``` Thank God I had kind of seen this at work one other time in my life, and we never fixed the issue there but I knew what it was... we had traefik acting as a load balancer (as do I at home) and the traefik load balancer was "balancing traffic using https"... no, I do not really know what I'm saying. But what I do know is that in order for the Temporal Server and Worker to communicate, traefik has to properly route the traffic for gRPC instead of https. !!! warning "" I can't stress enough how little I know about the difference between https and grpc or why the load balancer cares... there's clearly something in the networking stack that is extremely relevant here But the fix is NOT hard thankfully, and it's even on [traefik's website](https://doc.traefik.io/traefik/user-guides/grpc/#traefik-configuration) - the warning annotation there tells you what you need - which is one more traefik label... ```yaml traefik.http.services..loadbalancer.server.scheme=h2c ``` So after setting the loadbalancer.server.scheme to `h2c` I could run my worker just fine! ![20250625021838_4d7104c2.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250625021838_4d7104c2.png)

Nextcloud PHP Opache Memory Consumption

2025-06-23T08:46:20Z

Intro

Today I was combing through Nextcloud, just taking a gander at the apps, updates, etc.

The OPcache buffer is nearly full. To assure that all scripts can be hold in cache, it is recommended to apply “opcache.memory_consumption” to your PHP configuration with a value higher than “128”

So I know nothing about php and I had no idea where to set this… Also I’m running Nextcloud in a container so any updates will be blown away when the container is removed (say next time I update Nextcloud)…

After a little duckduckgoing I came across this reddit post where a guy had the same issue…

The long and short of it is that on single instance nextcloud deployments this php memory consumption value can be too low for Nextcloud to keep up with the demand… You can see in the message they just recommend upping the value.

So I had to think about how to persist the change and have it take effect now ideally without affecting any users too much… So here’s what I did - made the change in 2 places:

In the container #

I exec’d into the container and used this handy sed command sed -i 's/^opcache\.memory_consumption\s*=\s*128$/opcache.memory_consumption=256/' /usr/local/etc/php/conf.d/opcache-recommended.ini to update the value in the running container - and I guess php maybe is loaded up dynamically or something? but the warning did go away….

So how to persist this change when the filesystem is blown away when the container is updated?

In the config #

I set the env var PHP_OPCACHE_MEMORY_CONSUMPTION in the docker-compose.yml file to 256 and voila… the change should persist…

Quit issueing 200s for 500s

2025-06-22T06:55:21Z

Stop this nonsense

💭 Control Your n8n Instance Remotely with Telegram Bot Commands …

2025-06-22T00:38:53Z

Here's my thought on Control Your n8n Instance Remotely with Telegram Bot Commands | n8n workflow template --- I'm doing a lot of thinking and brainstorming about how I want to run some pipelines I have in my head, at home... things like image gen for blog posts, posting to various social media, click to generate qr codes and send emails, etc... things that require a lot of integration. I tried n8n a while ago with 1 use case, it didn't work but times have changed and whatever was wrong for that use case is now just fixed with a few updates... so I'm strongly considering n8n as a platform I build my workflows on, and using telegram as a simple entry-point into workflow execution is kind of intriguing... imagine sending a message to telegram to generate an image with SD based on a picture you took with our phone, maybe do something else with that image (turn it into a qr code or something), and then email it? all with a text message and pic? that sounds pretty cool... not sure what I"ll have spun up but I am excited to be making some progress on the automation ideas --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Thinking In Diagrams

2025-06-20T06:55:31Z

# Intro Something I didn't appreciate earlier on in my career (and I'm only 8 as far as the age of that career goes anyways) was thinking through problems with diagrams... I've often approached a problem with code first, and immediately started implementing a solution. There's nothing actually wrong with that especially when considering: 1. people think differently 2. some people think with their fingers 3. recognizing that the first thing you build should be thrown away should give you freedom to "just start" Now with that said, let's chat about how awesome diagrams are though... 1. they put context in front of you without details 2. you can express many more components of a system in a consumable way when compared to code 3. diagrams help prioritize work and bubble up maybe unforseen dependencies 4. they force you to wireframe before getting lost in the details ## Wireframing One of my biggest issue when I "just start coding" is not having a direction... I can put a few place holders `def this:` `def that:` in a few python modules, but without having every place holder in front of me it is hard to keep them straight, know the order or dependencies, etc. Diagrams aren't beholden to a text file or series of them, and you can fit much more on your screen because you have all the X-Y available to you instead of the height of your monitor times your formatted lined length (usually 120 for me). Diagrams also make it easy to spew out onto a page the different components you need for a solution, without committing to an order (something that I cannot express well in scripts) ## A Plan Diagramming with purpose can get you to the place of having almost a paint-by-number experience building your app/system/etc. Beecause you separate the implemetation from the design and when you implement BASED on a design rather than implement to DISCOVER a design you will be more efficient and in my experience I've improved my ability to forsee issues that I wouldn't have seen, or have historically missed, by putting the design down in front of me, the details of any integrations, etc. before starting to "just code". ## Good Enough A final note about diagrams for right now is that they're easy to leave as "good enough". The point was to get a guide for solving a problem. Diagrams don't have to be picture perfect, even for documentation's sake. They can even remain somewhat unfinished (permitted you don't get yourself into a bind of treating diagrams as docs that you didn't clean up) Here's an example of what I mean - I'm working on a simple workflow to take [my-thoughts](/my-thoughts) and post them to [[nostr]]. I started with a diagram like this... ![20250622112223_f1838d9b.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250622112223_f1838d9b.png) > In parallel I'm working on posting my blog to other social media - this small > workflow is apart of a larger initiative I'm working on at home AS you can see this diagram isn't super detailed or pretty. But what it does it help me see the high level points of bringing my published thoughts to nostr as posts. After I have the simple steps, I've started diagramming out more details of each step: ![20250622112415_f18663e3.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250622112415_f18663e3.png) The diagram is easy to put notes on, add some color to for problems not yet solved, etc. And then with this diagram in front of me, it's easy to begin building small-scoped components that will be assembled to my larger system, but without the pressure of constructing every part of the system at once # Fin Diagrams help me break up a problem into manageable chunks so that I can put the things I need later on the back-burner and can focus on one achievable thing at a time.

💭 pypeaday/whose-turn-is-it: Simple uv webapp to determine order…

2025-06-20T01:23:26Z

Here's my thought on pypeaday/whose-turn-is-it: Simple uv webapp to determine order fo players for an event --- We pray together as a family every night, but every single flipping night my daughter's fight about the order we go in... well, NO MORE! I built a simple uv fastapi app that randomly sorts a list from an environment var `PLAYERS` and presents it in the web page at `/generate`. Super easy `uv run` at home... I love that with enough knowledge, or the right kind of knowledge, you can build like anything you want! --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Homelab Youtube Podcasts to AudioBookShelf Pipeline

2025-06-18T13:53:12Z

I wrote about using [[homelab-pinchflat]] for at-home mashup creation and a song flow for church. But I use it for more than, that and here's another prime use case There are podcasts on YouTube I want to follow - The Standup with Prime, or Shawn Ryan's show. Basically I use pinchflat to download the audio and serve up those files in ABS (audiobookshelf)

source_up for nested .envrc

2025-06-17T09:52:17Z

TLDR

I have a use case for nesting directories with cascading .envrc files. and ran into an issue where the parent environment was being reset by the child environment. The fix is source_up

Context

I manage some infrastructure at home with a folder like this:

.
├── homelab
│   ├── .envrc
│   └── stack1
│       ├── .envrc

I do this so that the “homelab-wide” terraform variables can be exported with TF_VAR_the_variable in the homelab/.envrc and then the stack specific variables in stack1/.envrc get exported like TF_VAR_stack_specific_var.

For example, my cloudflare account id is needed in every stack, but the cloudflare zone id is different per stack - so instead of copying the account id var to every stack (however the vars are managed), I can set it in a parent .envrc and share it in nested stack directories

Problem #

# homelab/.envrc
#!/bin/bash
export VAR_FOO=BAR

So when I cd homelab I get

nic in /tmp/homelab   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ direnv allow
direnv: loading /tmp/homelab/.envrc
direnv: export +VAR_FOO

nic in /tmp/homelab   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ env | grep VAR
VAR_FOO=BAR

You can see VAR_FOO is set… but now I’ll cd stack1

nic in /tmp/homelab   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ cd stack1
direnv: error /tmp/homelab/stack1/.envrc is blocked. Run `direnv allow` to approve its content

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ direnv allow
direnv: loading /tmp/homelab/stack1/.envrc
direnv: export +VAR_BAZ

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ env | grep VAR
VAR_BAZ=BAM

As you can see VAR_FOO was unset, and VAR_BAZ is set… so homelab/.envrc was blown away!

Fix #

Edit stack1/.envrc with a source_up command

# stack1/.envrc # NEW
#!/bin/bash
source_up  # source the parent environment
export VAR_BAZ=BAM

After the edit

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ vim .envrc
direnv: error /tmp/homelab/stack1/.envrc is blocked. Run `direnv allow` to approve its content

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET  took 5s
⬢ [devbox] ❯ direnv allow
direnv: loading /tmp/homelab/stack1/.envrc
direnv: loading /tmp/homelab/.envrc
direnv: export +VAR_BAZ +VAR_FOO

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ env | grep VAR
VAR_FOO=BAR
VAR_BAZ=BAM

Both vars are set!

Stable Diffusion QR Codes for Josh

2025-06-17T06:27:28Z

After my blunder

(fjords of Finland:1.3), (steep cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1),(serene nature:1.2), (8K landscape photography:1.3), (gaming concept art:1.2), (ultra realistic: 1.2)

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1),y Steps: 30, Sampler: Euler a, Schedule type: Automatic, CFG scale: 7, Seed: 784129766, Size: 1536x1536, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 0.8, Pixel Perfect: True, Control Mode: Balanced”, ControlNet 1: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 0.75, Pixel Perfect: True, Control Mode: Balanced”, Version: v1.9.4

Final Candidates

Turns out upscaling can mess up the design… so I generated new ones at the high res instead of using AI to upscale

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: Euler a, Schedule type: Automatic, CFG scale: 7, Seed: 1827707865, Size: 1536x1536, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 1536, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 1.0, Pixel Perfect: False, Control Mode: Balanced”, ControlNet 1: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 0.88, Pixel Perfect: False, Control Mode: Balanced”, Version: v1.9.4

I turned the brightness down on the above one a little and got a better result

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: Euler a, Schedule type: Automatic, CFG scale: 7, Seed: 1827707865, Size: 1536x1536, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 1536, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 0.85, Pixel Perfect: False, Control Mode: Balanced”, ControlNet 1: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 0.75, Pixel Perfect: False, Control Mode: Balanced”, Version: v1.9.4

Some favorites

(fjords of Finland:1.3), (steep cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (green orange:1.0), (serene nature:1.2), (8K landscape photography:1.3), (gaming concept art:1.2), (ultra realistic: 1.1)

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: Euler a, Schedule type: Automatic, CFG scale: 7, Seed: 2926433856, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 1.0, Pixel Perfect: False, Control Mode: Balanced”, ControlNet 1: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 0.85, Pixel Perfect: False, Control Mode: Balanced”, Version: v1.9.4

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: Euler a, Schedule type: Automatic, CFG scale: 7, Seed: 1827707865, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 1.0, Pixel Perfect: False, Control Mode: Balanced”, ControlNet 1: “Module: none, Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.25, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.25, Guidance End: 0.88, Pixel Perfect: False, Control Mode: Balanced”, Version: v1.9.4

Codes and prompts

prompt: (breathtaking scenic view:1.2), (fjords of Finland:1.3), (steep forested cliffs:1.1), (crystal-clear blue water:1.2), (reflection of sky on water:1.1), , (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (birch trees:1.0), (serene nature:1.2), (ultra-realistic:1.3), (8K landscape photography:1.3), (photorealistic:1.2), (atmospheric lighting:1.2)

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: DPM++ 2M SDE, Schedule type: Karras, CFG scale: 7, Seed: 2206746812, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: controlnetQRPatternQR_v2Sd15 [2d8d5750], Weight: 1.15, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 0.85, Pixel Perfect: True, Control Mode: Balanced”, Version: v1.9.4

(breathtaking scenic view:1.2), (fjords of Finland:1.3), (steep forested cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), , (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (birch trees:1.0), (serene nature:1.2), (ultra-realistic:1.3), (8K landscape photography:1.3), (photorealistic:1.2), (greens and blues:1.2)

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2) Steps: 30, Sampler: DPM++ 2M SDE, Schedule type: Karras, CFG scale: 7, Seed: 2485699904, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: controlnetQRPatternQR_v2Sd15 [2d8d5750], Weight: 1.1, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 0.8, Pixel Perfect: True, Control Mode: Balanced”, Version: v1.9.4

(breathtaking scenic view:1.2), (fjord:1.3), (steep forested cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), , (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (birch trees:1.0), (serene nature:1.2), (ultra-realistic:1.3), (8K landscape photography:1.3), (photorealistic:1.2), (green:1.2)

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2) Steps: 30, Sampler: DPM++ 2M SDE, Schedule type: Karras, CFG scale: 8, Seed: 1055874558, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: controlnetQRPatternQR_v2Sd15 [2d8d5750], Weight: 1.1, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 0.8, Pixel Perfect: True, Control Mode: Balanced”, Version: v1.9.4

Top Contendor #

(breathtaking scenic view:1.2), (fjords of Finland:1.3), (steep forested cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), , (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (green orange:1.0), (serene nature:1.2), (ultra-realistic:1.3), (8K landscape photography:1.3), (gaming concept art:1.2),

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: DPM++ 2M SDE, Schedule type: Karras, CFG scale: 8.5, Seed: 1240242415, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: controlnetQRPatternQR_v2Sd15 [2d8d5750], Weight: 1.1, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 0.8, Pixel Perfect: False, Control Mode: Balanced”, Version: v1.9.4

(breathtaking scenic view:1.2), (fjords of Finland:1.3), (steep forested cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), , (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (green orange:1.0), (serene nature:1.2), (ultra-realistic:1.3), (8K landscape photography:1.3), (gaming concept art:1.2),

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: DPM++ 2M SDE, Schedule type: Karras, CFG scale: 8.5, Seed: 1240242416, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: controlnetQRPatternQR_v2Sd15 [2d8d5750], Weight: 1.1, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 0.8, Pixel Perfect: False, Control Mode: Balanced”, Version: v1.9.4

Scans!

(fjords of Finland:1.3), (steep forested cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), , (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (green orange:1.0), (serene nature:1.2), (8K landscape photography:1.3), (gaming concept art:1.2),

(blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: DPM++ 2M SDE, Schedule type: Exponential, CFG scale: 7, Seed: 4060643233, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: controlnetQRPatternQR_v2Sd15 [2d8d5750], Weight: 1.4, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 0.8, Pixel Perfect: True, Control Mode: Balanced”, Version: v1.9.4

(fjords of Finland:1.3), (steep forested cliffs:1.1), (crystal-clear water:1.2), (reflection of sky on water:1.1), , (early morning light:1.2), (golden hour glow:1.1), (pine trees:1.1), (green blue:1.0), (serene nature:1.2), (8K landscape photography:1.3), (gaming concept art:1.2),

Negative prompt: (blurry:1.3), (low resolution:1.3), (overexposed:1.2), (cartoonish:1.3), (unrealistic colors:1.3), (people:1.1), (city buildings:1.2), (vehicles:1.2), (text:1.3), (distortion:1.2)

Steps: 30, Sampler: DPM++ 2M SDE, Schedule type: Exponential, CFG scale: 7, Seed: 3713594572, Size: 768x768, Model hash: 4199bcdd14, Model: revAnimated_v122EOL, ControlNet 0: “Module: none, Model: controlnetQRPatternQR_v2Sd15 [2d8d5750], Weight: 1.4, Resize Mode: Crop and Resize, Processor Res: 512, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0.0, Guidance End: 0.8, Pixel Perfect: True, Control Mode: Balanced”, Version: v1.9.4

U-Blue Aurora Upgrade Broke HWA Docker

2025-06-15T08:42:09Z

I use Universal Blue’s Aurora distribution on my daily driver desktop, and I rebooted today which led what I imagine is a kernel update. This broke all of my docker containers using a GPU.

nvidia-container-cli: detection error: open failed: /usr/lib/libnvidia-tls.so.570.153.02: no such file or directory: unknown

Thankfully a reboot and booting into the older environment fixes everything… Linux and immutability FTW

Discorse Thread

Homelab Pinchflat

2025-06-14T08:51:21Z

# Why? I've written about [[self-hosted-media]] before and addressed some thoughts with YouTube. Lately though I've had 2 use cases for offline YouTube media and I'm now solving this problem by using [pinchflat](https://pinchflat.com/) as my `ytdl` solution + file system organization for use throughout my homelab. ## Use Case 1 At church we use pre-recorded music to sing along to, or at least we used to, thankfully as of June 2025 we're doing live-music every week. But in the before times I would strip the audio out of song files we had purchased and then made lyric videos with the instrumental tracks. As we wanted to try new songs out we didn't want to purchase them in case they didn't work out for us to use and that ends up being a waste (church budget is very tight). So using songs available on YouTube is an easy solution, but the only way to strip the vocals and see if the song sounded right with our singers and equipment was to make the lyric video and rehearse it... Chicken or Egg problem unless we can freely try it before purchasing... So using Pinchflat we can download songs in that playlist, I can edit the files, the team can try it out, and if we like it we can use it without hitting the budget any harder than necessary. > Pinchflat syncs that playlist to a folder on my NAS that another process > watches and performs the audio stripping workflow ## Use Case 2 I just wrote about [[if-you-want-something-make-it-so-song-style]] which is the second use case - cutting mashups of my favorite songs on youtube and then hosting them myself for easy listening # The Setup I run it in docker compose like practically all my other services ```yml services: pinchflat: container_name: "pinchflat" image: "ghcr.io/kieraneglin/pinchflat:latest" ports: - "8945:8945" volumes: - "/tank/encrypted/docker/the-pit-zfs/pinchflat/config:/config:rw" - "/tank/encrypted/nas/media/downloads/pinchflat/downloads:/downloads:rw" environment: TZ: "America/Chicago" PUID: "1000" PGID: "1000" restart: unless-stopped deploy: resources: limits: memory: "4g" ``` # The GUI The WebUI is easy to navigate - you organize your downloads however you want with "sources", and here's how I setup the source for these mashups that I want to edit... I add them to my playlist and Pinchflat grabs them for me automatically ![20250614144201_fa16e7ac.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250614144201_fa16e7ac.png) ![20250614144307_bdf12ee0.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250614144307_bdf12ee0.png) That's basically it - then as you can see in the compose file, I get the files in a folder on my NAS and from there I can edit them as I need. ## Example The church app that does the audio processing is in a compose stack with the pinchflat download directory on the host mounted in - this is how containers in different stacks can share data, by sharing a filesystem bind mount ```yml services: olivet_watcher_app: image: pypeaday/vocal-remover-watcher container_name: olivet-youtube-watcher volumes: - /tank/encrypted/nas/media/downloads/pinchflat/downloads:/input:ro - /tank/encrypted/docker/vocal_remover_app/watcher-app/processed_mp3s:/output - /tank/encrypted/docker/vocal_remover_app/watcher-app/db:/data working_dir: /app ```

If You Want Something Make It So - Song Style

2025-06-14T08:44:14Z

In grad school a professor of mine, Mark Hunacek, said one of the most profound things I've ever heard to the tune "if you want it, then make it so"... It's a great benefit to mathematicians at the bar to start a conversation with "hey let's assume X because we can"... This phrase has affected my work greatly to the work I do and how I go about it- it's literally changed how I design some of my code. But I also want this mantra to flow into other parts of my life... > So here's a post about how I use Pinchflat and Davinci Resolve to tickle my > brain with bespoke versions of beautifully crafted mashups In this example I took a Goobsie mashup, cut the part I didn't like quite as much (though admittedly it's still beautiful) and spliced it with itself to give me a longer running version of a musical piece that takes me to another world... [OG Youtube Mashup](https://www.youtube.com/watch?v=7v7nreDnmDg) ## Acquiring the Media Pinchflat isn't really necessary here, anyone can use `youtubedl` or another tool to download from YouTube. I do use Pinchflat to subscribe to Youtube playlists to automate this process for me. See more about that in [[homelab-pinchflat]] ## Creating the Mashup So Goobsie makes actually good mashups, and I'm not anything even remotely close to an audio engineer or whatever, but there's a segment of this song that just strikes my whole body in a unique way and I want to sometimes just listen to it conveniently... So enter Davinci Resolve... Basically I took the downloaded file, added it twice to a timeline, and then faded in and out until it sounded ok to me. ![20250614135418_b75bc26a.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250614135418_b75bc26a.png) ## Listening to the Mashup So now how do I listen to this? For the moment, it's a favorite in my Jellyfin library and I listen through that client. It's a TODO item for me to set-up a more music-centric streaming service !!! info "Go Forth" So Go Forth - and if you want something, make it so

ZSA typing challenge issue

2025-06-13T08:07:41Z

ZSA

I love zsa and have been using a moonlander for a few years now.

They recently launched typ.ing as a way to train your typing skills - it’s a lot like monkeytype.

I have a thought about it here

Today’s Challenge #

I went to do today’s daily challenge and have ran into an issue where part of the challenge is to type / the forward slash character but it’s actually a keybind on the website for navigation (well done btw!)

So when I try to type the challenge I actually bring up the nav menu

So for today I’ll take the error on the character to complete it and maybe ZSA can look into this?

Double Check Your DNS Records

2025-06-13T06:53:50Z

My Blog

I host my blog on cloudflare and manage my infra with terraform. I got a 522 when I went to https://www.pype.dev recently and I wondered why since https://pype.dev worked fine…

Terraform #

I had this right in the terraform:

resource "cloudflare_pages_domain" "cf_domain" {
  account_id   = local.account_id
  project_name = cloudflare_pages_project.build_config.name
  domain       = "pype.dev"
}

But turns out the www record was commented out for some reason in my terraform code:

resource "cloudflare_pages_domain" "cf_domain_www" {
  account_id   = local.account_id
  project_name = cloudflare_pages_project.build_config.name
  domain       = "www.pype.dev"
}

A quick terraform apply -auto-approve (careful with this) and we’re up and running with the www record now

💭 qwen3

2025-06-11T02:23:23Z

Here's my thought on qwen3 --- I really don't know much about every self-hostable LLM out there... so much as happened. I am blessed at home with a pretty beefy GPU and so although I don't know all the deets, I can run a lot of the reasonably large "open source" models. The one I've been playing with lately is qwen3. The conversational flow has felt more natural with it compared to llama3.2 or dolphin3. Granted I might need to update something and my experience is limited. I've also had good experience with gemma3 as a noteworthy contender for my go-to LLM right now in open-webui --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

💭 News - Apps - App Store - Nextcloud

2025-06-09T11:37:39Z

Here's my thought on News - Apps - App Store - Nextcloud --- I just started using FreshRSS to get familiar with subscribing to RSS feeds and leaning into that as opposed to social media... Twitter has become basically just a news outlet for me, not a place to connect with devs (except for 2 or 3 who are active). RSS seems like such cool and ancient technology that is protected from the corporate tentacles that hold up every social media outlet save for nostr... So as I look for an RSS reader and/or feed aggregator this Nextcloud plugin seems like a good choice to look at since I'm already hosting NC.. although do I really want to lean into it for more and more? --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Example For Andrew

2025-06-08T14:48:53Z

I built a simple app for adding images to my blog

2025-06-07T13:30:28Z

# Quick Deets I built a simple fastAPI app called "shotput" that I run locally inside a git repo where I save images for my blog. The app is simple: 1. upload image from clipboard 2. app commits image to repo (path is configurable) 3. app pushes the commit (auto-push configurable) 4. app returns a markdown link I can paste in my blog that uses statically.io (see [[statically-io-to-help-me-out]]) to serve the image from the repo to my blog See the image repo with app details at [images.pype.dev](https://github.com/pypeaday/images.pype.dev) # Usage Anyone could copy the script and config to their own git repo to put images in and run this at home - instructions are in the readme but for simple running on localhost it's just `uv run` # Example ![20250607130609_dadd33eb.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250607130609_dadd33eb.png) pasting an image let's you copy the markdown link for easy pasting ```markdown ![20250607130609_dadd33eb.png](https://cdn.statically.io/gh/pypeaday/images.pype.dev/main/blog-media/20250607130609_dadd33eb.png) ```

Double check your CIDR blocks!

2025-06-07T07:27:31Z

tl;dr

Copy paste - don’t rely on typing correctly…

The Issue

I use tailscale at home and generally love it. One thing I use it for is whitelisting - so I have some apps publically available and my homelab ones have DNS with SSL certs by LE but I whitelist to only my private IPs so that I can use https at home basically. However I have had a few issues with a couple apps, and I kind of chaulked it up to “network protocol this” “need to figure out that header that” but ultimately I was being lazy and the issues weren’t catastrophic… it was things like:

couldn’t get to my NVR by DNS, so I’d have to use tailscale_ip:port
couldn’t login with my user in jellyfin cause I lock me down to LAN only since I dont’ use a password
gotify wouldn’t connect using DNS

These things aren’t huge but they are annoying and today I decided to figure it out.

Typo! #

I’m not an expert yet but I think the main problem was the CIDR block I was using in my traefik config.

This’ll shock some of you but 10.64.0.0/10 and 100.64.0.0/10 are not the same blocks!

Check your CIDRs

It’s always DNS, and even when it’s not, it’s still probably networking

homelab-journey-part-1

2025-06-05T08:19:57Z

# Introduction I want to start writing about my homelab and this first post can be a short introduction to how I got into homelabbing and a review of my current hardware. I'll hopefully write about the evolution of my homelab to get to this current point and then start writing about updates as new things hit the lab. # The Journey ## Applications | Pi-Hole I started with a pi-hole appliance running on a pi-0 I had lying around. So, a little introduction into SSH and foreign operating systems (I was loosely familiar with Linux and so Raspbian wasn't a huge deal to start with) and then you could just [install pi-hole](https://docs.pi-hole.net/main/basic-install/). After installing it I got to learn a little bit about networking, DNS, DHCP, etc. all from just exploring and trying out the features of pi-hole. ## Hardware From there I happened to come into a few computers kind of for free that a church was recycling and so now with new-to-me x86 hardware and armed with a tiny bit of docker experience at work - I thought "well I wonder if I could run pi-hole in docker on a Linux machine?" and that opened the door to running other things in docker. # Applications | Free-NAS I quickly decided I wanted a NAS after talking with a friend at work, and one of those computers that I inherited was the perfect target for Freenas by iX systems (Freenas has since morphed into a few things - primarily TrueNAS Core for the homelabber). That got me somewhat familiar with jails (which is a BSD concept as FreeNAS was based on FreeBSD, not Linux) and running containers in different ways and then I decided I wanted Linux so I rebuilt with Ubuntu. !!! note "" there's been several hardware changes and such in there but that does more or less get me to arrive at where I'm at today which is I have a few different boxes running pretty standard operating systems for their purposes and they are as follows. # Current Homelab Setup !!! note "Find updated information at [[the-homelab]]" ## Ad blocking | Pi-Hole Hardware: pi-3 I have a Raspberry Pi 3 hardwired in that is running pi-hole so that's been a part of my infrastructure now for a long time. ## Router | OPNSense !!! note "stats" CPU: i5-8700 Memory: 8GB of whatever was in the box Storage: 1TB SSD My router is an old Optiplex SFF 5060 I got from Amazon for less than $100. So I'm running a x86 system for my router with OpenSense on it. Just seemed it to be another home lab standard and I'd like to someday bring pi-hole over there or use AdGuard or integrate those two things maybe a little bit more completely but right now they are mostly separate. ## Networking | VLANs There's a managed switch in there for some VLAN tomfoolery for my cameras and an IOT network type of a thing. ## Networking | VPN | Tailscale They're on tailscale as well so my DNS happens over tailscale. ## My Desktop !!! note "stats" CPU: Ryzen 7 5700X GPU: Nvidia 3090 Memory: 64 GB Storage: 4TB SSD + 2 TB HDD for zfs backup OS: Universal Blue Aurora My daily-driver desktop is somewhat apart of my homelab now as I have a 3090 in there with 20 GB of VRAM so I can run some heavier LLMs for self-hosted AI workloads. Everything in that space runs in a container and then I manage docker containers with compose stacks and portainer for visibility. ### Applications Some of the apps I run here are ollama, open-webui, automatic1111's stable diffusion webui, whisper-webui ## NAS / Server !!! note "stats" CPU: Ryzen 7 5700G GPU: None right now Memory: 128 GB Storage: 2x12 TB ZFS Mirror + another 2x12 TB ZFS Mirror + 500GB disk for NVR OS: Ubuntu Server 24.04 At the moment I am managing containers using Docker Compose stacks. Many applications have their own stack but then I also have stacks of many applications. The Compose files are all organized by host and they're in git and I use GitHub as my main get back and remote storage. ### Applications There are currently 74 containers running on my server, I will not list them all but here's some highlights: - traefik - portainer - nextcloud - jellyfin - \*arr stack - code-server - kanboard I regularly spin up compose stacks to try new things as I hear about them in newsletters and such - I will be making an attempt to write about them more frequently in a homelabbing series

Backups interrupted by full disk usage

2025-06-03T21:08:50Z

I just got a message from HCIO that my primary backup script is late… This happens every now and then but I decided to check on it… Quickly ssh in and I notice it takes a long time to connect, but nothing errors out… I’ve been in the game long enough to think this kind of latency is often disk IO. After waiting a bit and haulting the zshrc sourcing, I hit a sudo dust / -H and bam… There’s a container logfile that’s about 80GB sitting in /var…

  75G   │ │ │ │   ┌── 57409a7f8d3168b368e0230babd08ec2e20aabb288806aa13652db1e653a0eee-json.log
  75G   │ │ │ │ ┌─┴ 57409a7f8d3168b368e0230babd08ec2e20aabb288806aa13652db1e653a0eee           
  76G   │ │ │ ├─┴ containers                                                                   
  83G   │ │ ├─┴ docker                                                                         
  85G   │ ├─┴ lib                                                                              
  90G   ├─┴ var                                                                                
 117G ┌─┴ /

Easy enough to remove, but I need to setup some alerts on disk usage, and identify the container to limit the log file size

Useful Notifications

I use healthchecks.io to monitor several scripts. You can self-host it but I use the hosted service since he has simple Signal integration

homelab-computer-vision-pipelines

2025-05-31T08:18:55Z

Done in 11 seconds! Subtitle and audio files are in the outputs folder.

I wanted to talk through an idea I have for some computer vision pipelines at home. I heard of a tool called Roboflow, which is like a no-code computer vision pipeline training and inference tool. It seems very cool. I also use Frigate at home, and Frigate supports lots of detection on the NVR side. And so deploying a model in Frigate or using Roboflow, depending on what kind of customization you would need, you can have these detections potentially be as accurate as you want, at least with Roboflow. The thing is, I don’t know about Frigate, but my ideas are if I have a model at home that knows me and my wife and my children by name, then when monitored areas issue detections, for example, say I’m in the backyard and the temperature is above 80 degrees, have a home assistant kick on a sprinkler system or something like that. And if my wife is back there and it’s 80 degrees, don’t do that because she wouldn’t want that sprinkler on or whatever. I think that something like that would be very feasible once you have the model. So you would need a way to train the model, which would require gathering data and labeling it. Then you could train a model in lots of ways, but you need to have the deployment in mind. If the deployment was Frigate, you would need to be able to compile that model into some TensorFlow Lite thing that can run on a TPU. That sounds very possible. Otherwise, the Roboflow would probably be the platform for training, and then they make the deployment kind of a one-click thing. From there, I guess either way, Frigate is running the inference, and so the home assistant integration would be about the same. Frigate sends notifications over MQTT, and based on the Frigate blueprint that I’m using for home assistant anyway, based on the object and whatever other conditions you want to set in an automation, you do whatever you want to do in home assistant. That sounds neat.

Backups are dope

2025-05-27T19:42:27Z

I accidently chown'd -R an app directory and it totally screwed up the database folder. Luckily I zfs replicate my docker volumes to another drive even on the same host, so a quick [rsync] from /harbor/encrypted/docker/manyfold to /tank/encrypted/docker/manyfold got me back up and running since I hadn't replicated the messed up permission set yet Q: How to link to my rsync like a pro post? A: Wikilinks `[[rsync-like-a-pro]]` [[rsync-like-a-pro]]

SearXNG

2025-05-25T08:33:10Z

I heard about SearXNG on a couple podcasts and saw it trending on GitHub several times before I finally decided to stand it up. I used it transparently when trying out khoj, a self-hosted AI LLM agent playground kind of a thing, but I’ve also been messing around with Open-WebUI to have a self-hosted ChatGPT-like experience. I don’t know if SearchXNG used to be harder to set up, but it was pretty simple with a Docker Compose up and a couple configuration options given my personal homelab setup. Using it feels very nice.

services:
  redis:
    container_name: redis
    image: docker.io/valkey/valkey:8-alpine
    command: valkey-server --save 30 1 --loglevel warning
    restart: unless-stopped
    volumes:
      - ./data/redis:/data
    logging:
      driver: "json-file"
      options:
        max-size: "1m"
        max-file: "1"

  searxng:
    container_name: searxng
    image: docker.io/searxng/searxng:latest
    restart: unless-stopped
    ports:
      - "8080:8080"
    volumes:
      - ./data/searxng:/etc/searxng:rw
    env_file: .env
    environment:
      # - SEARXNG_BASE_URL=https://${SEARXNG_HOSTNAME:-localhost}/
      - UWSGI_WORKERS=${SEARXNG_UWSGI_WORKERS:-4}
      - UWSGI_THREADS=${SEARXNG_UWSGI_THREADS:-4}
    logging:
      driver: "json-file"
      options:
        max-size: "1m"
        max-file: "1"

I love the self-hosted aspect. I love not seeing any ads on my search results. I did a few searches where I know what results to expect and it did okay. It is good right now at filtering out garbage in my results.

So I look forward to tweaking it and using it as a search backend with open web UI. Next on my list is having enough resources to run Ollama and a stable diffusion generator at the same time and have image generation working through open web UI.

Grafana + Gotify for push notifications

2025-04-07T10:12:51Z

It’s easy to configure push notifications through Gotify from Grafana by setting up a generic Webhook Alerting Contact Point…

The parameters are:

HTTP Method: POST
URL: http:///message
Authentication is done with an app token generated from Gotify, and you put it in Authorization Header: Credentials in the Grafana configuration

See image

💭 audiobookshelf

2024-12-29T20:53:19Z

Here's my thought on audiobookshelf --- Audiobookshelf looks like a solid competitor for booksonic. I've liked the simple subsonic api driven nature of booksonic but my wife would like some more appropriate and easy-to-use organization... booksonic is not a great "library"... I'm looking forward to putting this into my homelab soon --- !!! note This is one of [[ my-thoughts ]]. I picked this up from [Waylon Walker](https://waylonwalker.com)(https://thoughts.waylonwalker.com). It's a short note that I make about someone else's content online. Learn more about the process [[ thoughts ]]

Add a healthcheck to your FastAPI app

2024-12-15T15:41:36Z

I’m building a few FastAPI apps to throw in docker and run on my homelab… I wanted to add healthchecks and here’s a simple way to do it

Make sure to install curl in the dockerfile (near the top for effeciency)

# Install curl with minimal dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Then I recommend making compose files even for single image deployments

services:
  app:
    build: .
    volumes:
      - type: bind
        source: .
        target: /app
    environment:
      - PYTHONPATH=/app
      - DOCKER_ENV=true
      - UV_VIRTUALENV=/opt/app-env
    user: "1000:1000"
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 10s

Then finally you’ll need a /health endpoint


@app.get("/health", response_class=HTMLResponse)
async def health_check():
    """
    A health check endpoint that returns a status message.
    """
    return "<html><body><h1>Service is healthy</h1></body></html>"

Jellyfin container updates for HWE + AMD

2024-12-14T11:27:15Z

I use LSIO Jelyfin container for the easy addon they provide for AMD GPUs but I couldn’t get trickplay to work with HWE…

There was almost NOTHING on the internet about the error, and all the threads were about BSD systems…

Thankfully someone posted on the formum here but the only answer was to literally upgrade stuff in the container…

Someday maybe I’ll build off of LSIO to add this, but until then I shell’d in and homelab’d the hell out of it

THIS IS INSIDE THE CONTAINER - I use Portianer to make it easy


apt update && apt install -y curl gpg

mkdir -p /etc/apt/keyrings

curl -fsSL https://repo.radeon.com/rocm/rocm.gpg.key | gpg --dearmor -o /etc/apt/keyrings/rocm.gpg

cat <<EOF | tee /etc/apt/sources.list.d/rocm.sources

Types: deb

URIs: https://repo.radeon.com/rocm/apt/latest

Suites: ubuntu

Components: main

Architectures: amd64

Signed-By: /etc/apt/keyrings/rocm.gpg

EOF

apt update && apt install -y rocm-opencl-runtime

hostnamectl to easily change hostname

2024-12-06T07:25:59Z

hostnamectl is apparently a linux utility for easily changing your hostname in a variety of ways


❯ hostnamectl --help
hostnamectl [OPTIONS...] COMMAND ...

Query or change system hostname.

Commands:
  status                 Show current hostname settings
  hostname [NAME]        Get/set system hostname
  icon-name [NAME]       Get/set icon name for host
  chassis [NAME]         Get/set chassis type for host
  deployment [NAME]      Get/set deployment environment for host
  location [NAME]        Get/set location for host

Options:
  -h --help              Show this help
     --version           Show package version
     --no-ask-password   Do not prompt for password
  -H --host=[USER@]HOST  Operate on remote host
  -M --machine=CONTAINER Operate on local container
     --transient         Only set transient hostname
     --static            Only set static hostname
     --pretty            Only set pretty hostname
     --json=pretty|short|off
                         Generate JSON output

See the hostnamectl(1) man page for details.

I learned there’s transient and static hostnames, so that’s cool…

The thing I needed was hostnamectl --static hostname babyblue-aurora

pretty sweet tool

DNS Broke After Reboot - Ubuntu 22.04

2024-11-22T08:08:40Z

I rebooted by server and DNS broke randomly. I have no idea if it was from a kernel update or what but that’s the issue with Ubuntu I guess…

After much toil and none of the other options working for me (sorry to not have those documented here) this is what got me the vic from this SO Post

sudo mkdir /etc/systemd/resolved.conf.d/ sudo $EDITOR /etc/systemd/resolved.conf.d/dns_servers.conf

Most folks probably are good with google (8.8.8.8) and cloudflare (1.1.1.1)

[Resolve]
DNS=8.8.8.8 1.1.1.1

But I decided to use tailscale

[Resolve]
DNS=100.100.100.100

Then restart systemd-resolved

sudo systemctl restart systemd-resolved

OPNSense Bootstrap Recovery

2024-11-07T08:40:19Z

enabling DHCP WAN port (dhclient <iface>)- running the bootstrap script - sh /usr/local/sbin/opnsense-bootstrap

Recovering OPNSense

2024-11-06T00:00:00Z

I woke up to faulty internet and after some troubleshooting it turns out the root zfs dataset that OPNSense boots from got corrupted…

PRO-TIP - Auto backup your OPNSense config to Google Drive, git, or nextcloud… But if you won’t then at least back up your OPNSense config somewhere everytime you update it.

It’s too much to recount every issue, so here’s a bullet list what worked.

On a fresh drive install OPNSense
Plug in the old drive through a USB enclosure - now I’m not sure what would happen if you plugged it in along with the new drive and then booted up. Because both drives will have a zfs pool zroot and the boot dataset is automounted at /zroot/ROOT/default. My old zroot pool was SUSPENDED so it didn’t automount
Because the old zoot/ROOT/default was corrupted I did this to mount it RO: zpool import -d <path to zfs partition - /dev/stuff> -N zroot zrootrecovery

-d is the zfs flag to import the pool by disk id, -N it to not mount any of the datasets (we need to change mountpoints) and the zroot zrootrecovery imports the zroot pool with a new name

Change the mountpoints for all the zrootrecovery datasets to somewhere like /mnt/zrootrecovery
Depending on the mount point you set you’ll find a config directory around /mnt/zrootrecovery/ROOT/default/config - copy the file you want to another machine via scp or whatever
Go to OPNSense webui and recover from that config!

All in all this process took me around 8 hours but I did run into about ever issue under the sun (several bad disks in the mix, a laptop that wouldn’t live boot into a BSD system, etc.)

docker-remote-add

2024-09-17T06:19:00Z

Add from url??

ADD http://example.com/cars.csv /tmp/cars.csv

Unpack automatically!? (.tar, .tar.gz, .tgz, .bz2, .tbz2, .txz, .zip)

ADD myapp.tar.gz /opt/myapp/

Docker copy and chown

2024-09-17T06:18:09Z

COPY –chown=myuser:mygroup source-file target-file

Interesting IPs between Jellyfin clients and server depending on tailscale and server address

2024-07-12T06:24:57Z

When connecting from my phone to jellyfin I’m seeing some interesting patterns.

Scenarios #

Tailscale IP of phone is listed as local network to jellyfin #

Wifi: off Tailscale: on Use exit node: on LAN access: on Jellyfin: LAN IP

Jellyfin sees 192.168.1.1, my router address

Wifi: off tailscale: on Use exit node: on LAN access: on Jellyfin: Tailscale magic DNS

Jellyfin sees the docker bridge network

Q: This might be because of traefik somehow

Wifi: off tailsacale: on Use exit node: on LAN access: off Jellyfin: LAN IP

Jellyfin sees the 192.168.1.1

Q: Why did this work even work?

Wifi: off tailsacale: on Use exit node: on LAN access: off Jellyfin: Tailscale magic DNS

Jellyfin sees the docker bridge network

Wifi: off tailsacale: on Use exit node: off LAN access: off Jellyfin: LAN IP

Jellyfin sees the 192.168.1.1

Q: Why did this work?

Wifi: off tailsacale: on Use exit node: off LAN access: off Jellyfin: Tailscale magic DNS

Jellyfin sees the docker bridge network

Wifi: on tailsacale: of Use exit node: off LAN access: off Jellyfin: LAN IP (via pihole DNS)

Jellyfin sees the IP of my phone

Upgrading your kernel can F you up… whoops

2024-06-26T10:21:59Z

https://forum.proxmox.com/threads/update-error-with-coral-tpu-drivers.136888/#post-608975 I lost TPU for frigate, this saved me ass

Customize K9s

2024-05-06T11:57:07Z

To customize k9s use the skins from catppuccin or the ones k9s supplies

OUT="${XDG_CONFIG_HOME:-$HOME/.config}/k9s/skins"
mkdir -p "$OUT"
curl -L https://github.com/catppuccin/k9s/archive/main.tar.gz | tar xz -C "$OUT" --strip-components=2 k9s-main/dist

Then edit your k9s config

# ~/.config/k9s/config.yml
k9s:
  ui:
    skin: catppuccin-mocha
    # ...or another flavor:
    # skin: catppuccin-macchiato
    # skin: catppuccin-frappe
    # skin: catppuccin-latte

    # ...or the transparent variants:
    # skin: catppuccin-mocha-transparent
    # skin: catppuccin-macchiato-transparent
    # skin: catppuccin-frappe-transparent
    # skin: catppuccin-latte-transparent

Other k9s skins are available here

Kanboard to keep me focused on my own ideas

2024-04-20T18:27:07Z

TL;DR #

I’ve been using kanboard as a self-hosted kanban board. It’s keeping me focused on Digital Harbor when I’d rather be doing something less productive.

My TODOs #

Here’s the thing about my TODOs… they’re everywhere. I’ve tried a crazy amount of different organizational tactics for my todo items. I’ve tried post-it notes, a journal, TODO.md in specific repos, a master todo repo, todo CLIs, etc…

The problem for me was consistency… I would regularly forget what I was using for TODOs at that moment in my life… was there a todo file in this project? Do I have a todo repo cloned on this computer? Is this stuff in my journal or on post-it notes?

And what do I do with new ideas? Do I organize them centrally or use a repo for the idea I have?

It was getting out of hand to a debilitating degree… For a while I just gave up on being organized at all… Things would get done as necessary and if I got some motivation to work on something it was immediately smothered with the anxiety of how I was going to organize my work…

At some point it became too much… Now, I have some experience with Azure DevOps/Jira for project management and then I came across Kanboard…

Kanboard #

Kanboard

Kanboard is just a self-hosted kanban style todo app. I know there’s a ton of these so the TL;DR of my lesson is I picked an app that I would just use based on simplicity of managing and hosting.

I have had kanboard running in my homelab for a long time, but I only barely use it intermittenly. And at that rate I didn’t spend any real time organizing my tickets, so I wasn’t akshuallly using it - it was just a post-it note replacement.

But then I had an idea for a genuine business idea, and if I was going to ever have a hope of making it a reality, I needed to stay organized. This was when I decided to give kanboard a little more effort… I knew I could always remember that I’d chosen an app as my TODO solution (given all the time I spent questioning what I was using at any given point in time). I also knew I could host kanboard so that I could get to it from wherever was necessary because my homelab is relatively easy to add another public service to.

So once I just decided to lean into this thing, I would take advantage of any moments of motivation and just jot down ideas for things that had to get done… Simple stuff like explore an infra management option, add one feature to a config, or migrate one website to another stack… And I would just write these things downs until I have enough time free to crack down on a task. The beautiful thing is, when I am struck with just enough time to do something, and the motivation to do something, I don’t waste any time deciding what to do - past-me did present-me a favor and decided what was important already… So as long as I do myself the favor, when I’m ready to go I am never beaten by the anxiety of not knowing what I ant to do or how I’ll track what I’m doing… I chose kanboard and even though it’s not as fast as a terminal TUI, it is reliable, simple, and keeps me focused on what I need to do.

The biggest benefit #

Not only is just having one app as the solution nice because it’s centrally managed and accessible from wherever I need to get it, but the TOP feature of kanboard I use is comments on tickets… I get to continue to do future-me favors, my jotting down where I’m at in a task, what’s left to do, what I’m trying, etc… and future-me is in a great mood, because when I have that free few minutes, I can just read my own past thoughts and get back up to speed without wasting time trying to remember things I never would’ve remembered if I hadn’t written them down!

DHCP Restart to Save Ubuntu 22.04 Server Networking

2023-12-31T20:26:50Z

I moved a computer to a remote location for an off-site backup but when it was powered on it wouldn’t show up on any networks. A solution that got me back in was a friend restarting the dhcp client for me:

sudo dhclient -r -v <interface> && sudo dhclient -v <interface>

Simple Port Forwarding OPNSense

2023-10-17T10:26:34Z

https://forum.opnsense.org/index.php?topic=8783.0

Refresh Nextcloud Groupfolders after messing around on the filesystem

2023-09-23T12:45:06Z

Exec in as www-data and run ./occ groupfolders:scan folder_id -v (the -v to see what it’s doing)

lsof to find what’s using your filesystem

2023-04-09T13:32:38Z

lsof | grep /tank/nas shows me what is using my nas at any time!

Changing ZFS key for child datasets of encrypted dataset after migration

2023-04-08T20:12:22Z

➜ pihole sudo zfs load-key -L file:///path/to/.zfs.tank.key tank/encrypted/vms/arch-sandbox ➜ pihole sudo zfs change-key -o keylocation=file:///path/to/.zfs.tank.key -o keyformat=raw tank/encrypted/vms/arch-sandbox Need to load-key for each individual dataset, then change key location to be a file instead of the prompt

Stable Diffusion Notes

2023-01-28T14:15:11Z

ChatGPT Prompt: #

Stable Diffusion is an AI art generation model similar to DALLE-2. Here are some prompts for generating art with Stable Diffusion.

Example:

A ghostly apparition drifting through a haunted mansion’s grand ballroom, illuminated by flickering candlelight. Eerie, ethereal, moody lighting.
portait of a homer simpson archer shooting arrow at forest monster, front game card, drark, marvel comics, dark, smooth
pirate, deep focus, fantasy, matte, sharp focus
red dead redemption 2, cinematic view, epic sky, detailed, low angle, high detail, warm lighting, volumetric, godrays, vivid, beautiful
a fantasy style portrait painting of rachel lane / alison brie hybrid in the style of francois boucher oil painting, rpg portrait
athena, greek goddess, claudia black, bronze greek armor, owl crown, d & d, fantasy, portrait, headshot, sharp focus
closeup portrait shot of a large strong female biomechanic woman in a scenic scifi environment, elegant, smooth, sharp focus, warframe
ultra realistic illustration of steve urkle as the hulk, elegant, smooth, sharp focus
portrait of beautiful happy young ana de armas, ethereal, realistic anime, clean lines, sharp lines, crisp lines, vibrant color scheme
A highly detailed and hyper realistic portrait of a gorgeous young ana de armas, lisa frank, butterflies, floral, sharp focus
lots of delicious tropical fruits with drops of moisture on table, floating colorful water, mysterious expression, in a modern and abstract setting, with bold and colorful abstract art, blurred background, bright lighting
1girl, The most beautiful form of chaos, Fauvist design, Flowing colors, Vivid colors, dynamic angle, fantasy world
solo, sitting, close-up, girl in the hourglass, Sand is spilling out of the broken hourglass, flowing sand, huge hourglass art, hologram, particles, nebula, magic circle
geometric abstract background, 1girl, depth of field, zentangle, mandala, tangle, entangle, beautiful and aesthetic, dynamic angle, glowing skin, floating colorful sparkles the most beautiful form of chaos, elegant, a brutalist designed, vivid colours, romanticism

The prompt should adhere to and include all of the following rules:

Prompt should always be written in English, regardless of the input language. Please provide the prompts in English.
Each prompt should consist of a description of the scene followed by modifiers divided by commas.
When generating descriptions, focus on portraying the visual elements rather than delving into abstract psychological and emotional aspects. Provide clear and concise details that vividly depict the scene and its composition, capturing the tangible elements that make up the setting.
The modifiers should alter the mood, style, lighting, and other aspects of the scene.
Multiple modifiers can be used to provide more specific details.

I want you to write me a list of detailed prompts exactly about the IDEA follow the rule at least 6 every time.

Ready for my idea?

ChatGPT Prompt for RPG V4: #

Stable Diffusion is an AI art generation model similar to DALLE-2. Here are some pairs of positive and negative prompts for generating art with Stable Diffusion.

Example:

Positive: A (full body:1.3) shot at 8k resolution, splash art, fantastic comic book style, photorealistic, intense look, anatomical photorealistic digital painting portrait of a (old male:1.3) human (warrior:1.3) in black and gold intricate (heavy armor:1.3) in a (dark and moody universe:1.3), light particle, very detailed skin,samurai, very detailed eyes, (elden ring style:1.3), (warhammer style:1.1), concept artist, global illumination, depth of field, splash art, art by artgerm and greg rutkowski and viktoria gavrilenko Negative: (symmetry:1.2), facial marking, crown, horn, (helmet:1.3), (hoodie:1.1), clock, Female, visible hand, asian, two face, big hair, open mouth, cartoon, high contrast, poorly drawn, Scribbles, Low quality, Low rated, Mediocre, Screenshot, Software, UI, watermark, text, overlay, getty images, cropped, low quality

Positive: close-up head, facing camera, beautiful satanic female necromancer blood queen, ritualistic (neck tattoo:1.3), (insanely detailed:1.5), ((solo)), (highest quality, Alessandro Casagrande, Greg Rutkowski, Sally Mann, concept art, 4k), (colourful), (high sharpness), ((detailed pupils)), ((painting:1.1)), (digital painting:1.1), detailed face and eyes,Masterpiece, best quality, highly detailed photo:1, 8k, detailed face,photorealistic, (black long Hair:1.1),(young woman),By jeremy mann, by sandra chevrier, by maciej kuciara, smoke and shadow, night, sharp, ((perfect body)), realistic, real shadow, 3d, ((full body)), ((dark and gloomy universe)), (by Michelangelo) Negative: crown, (facial marking:1.2), cloth gem, jewel, jewelry, (flower:1.3), (cloak:1.1), (bad art, low detail, pencil drawing, old, mature:1.6), (grainy, low quality, mutated hands and fingers:1.5), (watermark, thin lines:1.3), (deformed, signature:1.2), (big nipples, blurry, ugly, bad anatomy, extra limbs, undersaturated, low resolution), disfigured, deformations, out of frame, amputee, bad proportions, extra limb, missing limbs, distortion, floating limbs, out of frame, poorly drawn face, poorly drawn hands, text, malformed, error, missing fingers, cropped, jpeg artifacts, teeth, unsharp

Positive: (Painting:1.3) of (Detailed illustration:1.3) A (full body:1.3) shot at 8k resolution, splash art, fantastic comic book style, photorealistic, intense look, anatomical photorealistic digital painting portrait of a (old male:1.3) human (warrior:1.3) in black and gold intricate (heavy armor:1.3) in a (dark and moody universe:1.3), light particle, very detailed skin,samurai, very detailed eyes, (elden ring style:1.3), (warhammer style:1.1), concept artist, global illumination, depth of field, splash art, art by artgerm and greg rutkowski and viktoria gavrilenko Negative: (symmetry:1.2), facial marking, crown, (horn:1.1), (helmet:1.3), (hoodie:1.1), clock, Female, visible hand, asian, two face, big hair, open mouth, cartoon, high contrast, poorly drawn, Scribbles, Low quality, Low rated, Mediocre, Screenshot, Software, UI, watermark, text, overlay, getty images, cropped, low qualityLow quality,Bad composition,Faded,(Photo:1.5),(Frame:1.3),watermark,signa ture

Positive: sci-fi, space rogue thief girl, (smirk:1.1), short hair, black hood, light armor, big grey eyes, beautiful detailed eyes, drawn by Greg Rutkowski, Yoji Shinkawa:0.6 Negative: open mouth, facial marking, flower, bad-hands-5, lipstick, text, watermark

The prompt should adhere to and include all of the following rules:

Each prompt should consist of a description of the scene followed by modifiers divided by commas.
When generating descriptions, focus on portraying the visual elements rather than delving into abstract psychological and emotional aspects. Provide clear and concise details that vividly depict the scene and its composition, capturing the tangible elements that make up the setting.
The modifiers should alter the mood, style, lighting, and other aspects of the scene.
Multiple modifiers can be used to provide more specific details.

I want you to write me a list of detailed prompts exactly about the IDEA follow the rule at least 6 every time.

Ready for my idea?

IDEAS #

IDEA: A slice of pie that is filled with a futuristic city

Artists I like #

Examples

J. M. W. Turner
Jean-Baptiste Monge
John Lasseter
John Martin
Raymond Swanland

Anime styles #

Krenz Cushart
Kyoto Animation
Yoshiyuki Sadamoto
Yuumei

Scenic #

Ted Nasmith
Terry Redlin
Thomas Kinkade
Thomas Moran
Thomas W Schaller

Cityscapes #

Marc Simonetti
Wadim Kashin

Abstract #

Scott Naismith

Settings notes #

I tend to like cfg at least on the higher end

Good prompts #

Angel of death, shadows and mist, skelletol feathered wings, highly detailed, amazing details, intriciate lines, purple haze, styles Yuumei and Krenz Cushart, UHD, HDR, upscale, 8K photo, concept art

Unicorns #

positive: unicorn, lora:Unicorns:0.9, portrait, professional photography, 35mm, highly detailed, incredibly realistic, best quality, high quality, highres, purple, lora:mossbeast:0.3

negative: (two tails:1.2),FastNegativeV2,(bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,lowres, bad anatomy, bad hands, ((text)), (watermark), error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, ((username)), blurry, (extra limbs), bad-artist-anime, badhandv4, EasyNegative, ng_deepnegative_v1_75t, verybadimagenegative_v1.3, BadDream,(three hands:1.1),(three legs:1.1),(more than two hands:1.4),(more than two legs,:1.2),

FFMPEG 10-bit videos to 8-bit

2023-01-16T13:15:53Z

ffmpeg -i input.mp4 -map 0 -c:v libx264 -vf format=yuv420p -c:a copy output.mp4

Reminder about ssh-copy-id for SSH and Ansible

2022-12-28T13:33:07Z

ssh-copy-id -i my.key.pub <hostname probably from tailscale> this makes sure I can run ansible from my desktop against VMs on my server easily if they have tailscale for the hostname - otherwise use the IP

Nextcloud Docker Upgrade Error

2022-12-28T09:39:27Z

https://nicolasbouliane.com/blog/nextcloud-docker-upgrade-error

Systemd timer for syncoid

2022-12-21T11:38:27Z

I have a bash script called syncoid-job which boils down to a barebones -

#!/bin/bash

syncoid --no-sync-snap --sendoptions=w --no-privilege-elevation $SYNOIC_USER@$SERVER:tank/encrypted/nas tank/encrypted/nas

I want to run this script hourly but as my user (notice the no-privilege-elevation flag)

First - create a systemd unit file at /etc/systemd/system/syncoid-replication.service

[Unit]
Description=ZFS Replication With Syncoid

[Service]
Type=oneshot
ExecStart=/$HOME/dotfiles/syncoid-job
User=$USER
Group=$GROUP

[Install]
WantedBy=multi-user.target

Then we save the unit file, enable the service, and then start it

systemctl enable syncoid-replication.service
systemctl start syncoid-replication.service

Note this will run that script… so be ready for syncoid to do its thing

Now for the timer… We create /etc/systemd/system/syncoid-replication.timer

[Unit]
Description=Run syncoid-replication every hour

[Timer]
OnCalendar=hourly

[Install]
WantedBy=timers.target

Hit it with a systemctl enable syncoid-replication.timer and you’re in business!

Cron for Nextcloud in Docker

2022-12-13T06:43:45Z

AJAX wasn’t cutting it, traditional crontab in containers doesn’t make much sense to me, webcron is recommended but I don’t want to register with anything outside my LAN… Turns out you can just spin up an identical container with a different entrypoint to /cron.sh that does what you need!

Note that this is a task in an Ansible playbook - but the docker-compose is straight forward

So the only thing you need to make sure of is that all the configuration options - data volumes, user permissions, etc. are identical between the containers running the cron job and the one actually hosting NextCloud. This ensures that the container running cron has proper access to the database and filesystem - or at least the same access as NextCloud proper.

- name: Nextcloud Cron Docker Container
  docker_container:
    name: nextcloud-cron
    image: "{{ nextcloud_image }}"
    pull: true
    links:
      - nextcloud-mysql:mysql
    entrypoint: /cron.sh
    volumes:
      - "{{ nextcloud_data_directory }}/nextcloud:/var/www/html:rw"
    env:
      MYSQL_HOST: "mysql"
      MYSQL_DATABASE: "nextcloud"
      MYSQL_USER: "{{ nextcloud_sql_user }}"
      MYSQL_PASSWORD: "{{ nextcloud_sql_password }}"
      NEXTCLOUD_TRUSTED_DOMAINS: "{{ nextcloud_hostname }}.{{ ansible_nas_domain }}"
      PUID: "{{ nextcloud_user_id }}"
      PGID: "{{ nextcloud_group_id }}"
      TZ: "{{ ansible_nas_timezone }}"
    restart_policy: unless-stopped
    memory: "{{ nextcloud_memory }}"

You can embed gifs?!

2022-10-09T13:39:39Z

I learned you can embed gifs and stuch in things other than GH personal pages!

So here I’ll keep ones I like and might use later

via GIPHY

Don’t forget to load XMP!

2022-10-09T07:17:43Z

Bought some DDR4-3600 speed RAM but only seeing 2666? Load up the BIOS, find DRAM config or something similar, and make sure to load the XMP profile to get that advertised RAM speed!

My PassMark Scores

2022-10-08T18:43:29Z

Babyblue v2

Ryzen 5700x + 32 GB 3200 CL16 RAM

Venus

Ryzen 3200G + 16GB 3600 CL16 RAM

Hogwarts

Dual Xeon X5650 @2.67GHz | 90 GB 1883 DDR3 ECC RAM

Server 2 ... Change the name

Ryzen 5700G + 128GB 3200 CL18 RAM

Server 2 with XMP enabled and undervolted from 1.35 to 1.2

Ryzen 5700G + 128GB 3200 CL18 RAM

Undervolted to 1.2V (from 1.35) XMP enabled

Quick setup of ZFS encrytped datasets with sane permissions

2022-10-06T19:58:01Z

Assuming you have a pool called tank…

And assuming you have an encrypted dataset (See Jim Saltar’s short intro)

Create a group for permissions - in my case I have one called home
Then if there’s anything in /tank/encrypted his it with chgrp -R home /tank/encrypted to give the home group ownership
Next we need to make sure that the members of home can do the writing… so chmod 775 -R /tank/encrypted will do the trick
Finally we want to make sure that all data created inside our dataset has the same set of permissions with chmod g+s /tank/encrypted and chmod g+w /tank/encrypted

Trick to login to web server on another box on lan

2022-09-12T18:48:45Z

TODO: change title - add –listen or –host or something as 0.0.0.0… that way it listens not on localhost or 127.0.0.1, but on all addresses!

Paperless-NGX filtering on IDs instead of values

2022-08-09T08:28:51Z

To filter for saved views from the admin console you have to use the id of the tag you are using to filter on…

Ex: filtering for tags “Inbox” doesn’t work, but for tag 18 does (the id fo the Inbox tag). I can find that ID by clicking on the tag and looking at the url bar

Mounting exFAT USB in Linux

2022-07-31T07:38:43Z

Steps

sudo fdisk -l

then look for the device and partition

get the Type column

mount

Example #


dumbledore in /media  NO PYTHON VENV SET 
❯ sudo fdisk -l

...

Device     Boot    Start      End  Sectors  Size Id Type
/dev/sdk1  *        2048 60371951 60369904 28.8G  7 HPFS/NTFS/exFAT
/dev/sdk2       60371952 60437487    65536   32M ef EFI (FAT-12/16/32)


dumbledore in /media  NO PYTHON VENV SET 
❯ sudo mount -t ntfs /dev/sdk1 /media/ventoy-usb -o uid=1000                       
NTFS signature is missing.
Failed to mount '/dev/sdk1': Invalid argument
The device '/dev/sdk1' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?

dumbledore in /media  NO PYTHON VENV SET 
✗ sudo mount -t exfat /dev/sdk1 /media/ventoy-usb -o uid=1000

Xrdp-Authentication-Required-To-Create-Managed-Color-Device

2022-07-18T00:00:00Z

I just need to RDP into an Ubuntu box via Remmina and everytime I login I have to authenticate to create a color managed device… which I don’t even know what that is!

To fix it?

vim /etc/polkit-1/localauthority.conf.d/02-allow-colord.conf

polkit.addRule(function(action, subject) {
 if ((action.id == "org.freedesktop.color-manager.create-device" ||
 action.id == "org.freedesktop.color-manager.create-profile" ||
 action.id == "org.freedesktop.color-manager.delete-device" ||
 action.id == "org.freedesktop.color-manager.delete-profile" ||
 action.id == "org.freedesktop.color-manager.modify-device" ||
 action.id == "org.freedesktop.color-manager.modify-profile") &&
 subject.isInGroup("{users}")) {
 return polkit.Result.YES;
 }
 });

Give’r the :x, logout, and you’re good to go

ZFS Permissions for Sanoid/Syncoid

2022-07-08T15:54:58Z

zfs allow -u $USER clone,load-key,create,destroy,mount,mountpoint,receive,send,rollback,compression,snapshot,hold,keylocation,bookmark tank

load-key only needed if using encrypted datasets

Setup KVM to boot from local PXE server

2022-07-06T12:30:50Z

Assuming you have a PXE server running you only need small edits to the virt-manager config

Self-hosted Docker registry with proxy pull through

2022-07-06T06:44:35Z

I decided that I want to self-host all my docker images for the purposes of regularly rebuilding and security scanning. The first step is to set up a registry, which coincidently enough you can do with a Docker container 😛!

Instructions for setting up the proxy are on the offidical docker docs here

Deployment

I chose to deploy with Portainer because I already use it for monitoring all my docker containers. I deploy most container with Ansible but use Portainer to setup a few more adhoc or non-git driven applications.

kvm-network-interface-via-nat-ubuntu-20

2022-06-25T07:20:41Z

I have started using VMs more and more in my development workflow and it’s impossible to work in a VM without an internet connection for me most of the time. Setting up the KVM networking is kind of confusing to me and I’ve done it two different ways. Here is how I set it up on my home desktop using NAT.

Credit

First thing’s first: credit to this post

Commands

There was a default network already made by virt-manager but my VM couldn’t connect over it at all…

These commands got me up and running without even turning the VM off

I went full on sudo -i for this just to make it easier - be careful

Dump an existint network config #

# as root

virsh net-dumpxml default > br1.xml

vim br1.xml

Edit it #

I was unsure what the ip range should be so I just stuck with the original blog. The default network had the CIDR block defined as 192.168.122.0/24 which is different from my home network so I guess it’s fine?

<network>
  <name>br1</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='br1' stp='on' delay='0'/>
  <ip address='192.168.10.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.10.10' end='192.168.10.100'/>
    </dhcp>
  </ip>
</network>

Define a network #

virsh net-define br1.xml
virsh net-autostart br1

Then to check…

virsh net-list --all

 Name      State    Autostart   Persistent
--------------------------------------------
 br1       active   yes         yes
 default   active   yes         yes

UUID #

virsh net-uuid br1

Magic #

virsh attach-interface --domain <NAME OF VM> --type bridge --source br1 --model virtio --config --live

My VM, ubuntu20.04 was running and immediately connected to the newly attached device!

Credit again

Visit the original post for more details - this serves more as as a quicker set of notes for future me

Reset SSH key passphrase

2022-06-13T06:55:32Z

I got into a pickle where I encrypted the ssh keys I use for my SSH connections on LAN, but then I couldn’t run my ansible playbook on my server! ssh-keygen -p and leave the new passphrase blank saved my day (although password protected key files are safer!)

TL;DR - just reset it to nothing

ssh-keygen -p will let you reset the passphrase on your ssh keys (good for you! yay security!)

But I needed to remove the passphrase to quickly deploy an ansible playbook 🤓

Samba on Ubuntu 22 needs “inherit permissions” set

2022-05-31T06:30:42Z

see samba config on hogwarts

Reindex Nextcloud After Adding Data via CLI

2022-05-29T15:34:33Z

My Nextcloud woes

I wrote here about setting up www-data as the owner of any directories you want nextcloud to manage. However, I regularly struggle wtih permissions issues on my NAS because of the external storage app anyways so I’ve decided to just put our photos in the spot Nextcloud would otherwise put them, and use this as healthy pressure on our family to organize our photos and put the ones we care about with the rest of our family media.

Migration

Because I had a ton of photos on the NAS anyways that I wanted moved over to Nextcloud I just rsync’d the photos directory on my NAS to the user’s photos directory in nextcloud but they weren’t showing up in the web UI!

The Fix?

As www-data I needed to php /var/www/nextcloud/occ files:scan --all inside my nextcloud docker container AFTER moving a bunch of photos off my “NAS” into the folder mounted to the nextcloud container as its data folder! Before I did this they weren’t showing up in the web UI/

arr client config

2022-05-28T13:29:28Z

TIL that when setting up download clients for radarr/sonarr/lidarr/readarr/bazarr/prowlarr that you can utilize internal DNS and instead of hardcoding an IP address of your download client server, can use just the CNAME record (ie. instead of 172.10.14.13 I can use transmission.mydomain.com… notice the lact of http(s)://… adding that won’t allow the connection to work/

Furthermore, you can use internal DNS to lookup the domain, not the subdomain, and expose the port, like mydomain.com:7878 for sonarr. This was simpler to maintain because I don’t change which ports an application exposes or utilizes hardly ever, plus I don’t need to maintain CNAME records for every service!

Add space to your LVM on Ubuntu

2022-05-26T21:33:55Z

I ran out of space on the SSD in my server when doing some file transfers but only 100GB was used of a 256 GB SSD?

LVM

When installing Ubuntu live server the default option for how to partition the disk (in my experience) has been to setup an LVM group that defaults to less than the available space. Most recently I put Ubuntu server on a 256 GB SSD but the main partition was formatted as an LVM group with 100GB of storage… I didn’t think anything of this even though I’m mostly used to EXT4.

I think the reason for LVMs is performance, but in hindsight, I don’t really care much about the performance differences, I really just want all my storage that’s fast enough

Extending the LVM

A moment of googling brought me to Ubuntu’s wiki and I learned that I can expand my LVM to the space I need…

sudo lvdisplay and sudo pvdisplay show detailed views of the logical volumes and physical volumes respectively.

Take a look at those and find the volume you need to extend. For me I found this:

  --- Logical volume ---
  LV Path                /dev/ubuntu-vg/ubuntu-lv
  LV Name                ubuntu-lv
  VG Name                ubuntu-vg
  LV Write Access        read/write
  LV Status              available
  ...

There’s more that you’ll see but this is what’s relevant - I need to extend the ubuntu-lv logical volume in the ubuntu-vg volume group.

sudo lvextend -L +50g ubuntu-vg/ubuntu-lv gives me 50 more GB of storage which should be enough for at least tonight 🤓

RTFM

Tdarr worker nodes share the cache!

2022-05-25T06:36:59Z

When working with tdarr remote nodes, they need to have access not only to the same libraries but also the same transcode cache as the server otherwise the transcodes will fail…

Network Setup

To explain I’ll give a brief overview of my home setup

I have an old Dell PowerEdge R610 as my main server running a live server distribution of Ubuntu. I use ZFS for my NAS file system, and most of my datasets are accesible over my home network. I have a Tdarr server running in a docker container on the R610.

In my office I dailyi drive a gaming desktop with an Nvidia 2060 Super running Ubuntu as well. On that desktop I am running a Tdarr node in a docker container. The container has access to the network folders with my media.

Initial Magic

When I spun up a tdarr node on my desktop, the tdarr server running on my R610 automatically registered the node, which was freaking amazing. That magic though spoiled me and I thought that I didn’t need to read the rest of the docs…

Initial Fail

I setup a transcode cache directory on the R610 locally and a separate transcode cache on my desktop that the remote tdarr node would use. Having them separated led to 2 main issues:

Transcodes were not being migrated back to my library properly
I was running out of disk space on my desktop because tdarr wasn’t deleting the completed cache files properly.

The Fix

Make a transcode cache directory accessible on my network, and give access to that directory to the docker container running my remote tdarr node.

Local DNS with Pi-hole

2022-05-23T09:40:59Z

Spoilers #

Tailscale is way easier than this… I was doing this local DNS overwrite in Pi-hole before running tailscale and I haven’t just totally “kicked the habit” yet, so for anyone NOT running tailcale, but wanting local HTTPS and using Pi-Hole this method would work fine

Intro #

I use pihole as my DNS server at home. I run unbound as well and have a pretty standard setup from their docs.

Pi-hole does 2 primary things for me:

dns sink-hold, the primary use case I believe
local SSL for all my self-hosted apps

Caveat #

I know for sure there are better ways to do this, but there’s also worse ones… so for now this has been my pattern, and it’s only bitten me in the butt when I’ve forgotten to add the final CNAME record… which’ll make sense in a sec.

Process #

It’s really quite simple - in pihole I have a DNS record for my domain pointing to my primary server

Then for each service that I want to keep everything resolved locally for I add a CNAME

With DNS resolving this way for clients using my pihole, the networking all stays local and I still get HTTPS for my wildcard cert in cloudflare

Why? #

The reasons I do this are simple:

I’ve been doing it since I started homelabbing and it started out of misunderstanding of how networking works at all
I use a home dashboard with https:// links, and it’s nice to just use that same dashbaord publically or at home. With local DNS resolution then I can whitelist some services, but conveniently access over the public url with https but routing such that my whitelist let’s me in when it wouldn’t if I tried to access the service from an external client
it’s my homelab - I can do what I want

It’s too complicated #

Honestly, as simple as this is, it is tedious and kind of complicated… There’s options to make it better…

script any service deployment to update the pihole /etc/host file
just use tailscale…

Configure bridge network on Ubuntu 22.04 with Netplan

2022-05-22T14:55:28Z

See 02-….yaml in ansible-nas

Netplan change from Focal to Jammy

2022-05-22T13:23:17Z

I am revamping my home server and bumped myself early up to Jammy Jellyfish… however to my peril I reused my netplan config and after hitting my server with the ‘ol netplan apply I lost connection… DNS still seemed to kinda work externally, but internally nothing was up…

Turns out Netplan got a little change in how to express the gateway key in the netplan config!

Old Ubuntu 20.04 way

network:
  version: 2
  ethernets:
    enp0s4:
      addresses: [192.168.1.{Static IP}/24]
      gateway4: 192.168.1.1  # <-- This changes!
      nameservers:
        addresses: [192.168.1.1, 1.1.1.1]

New jammin way for Jammy Jellyfish (at least that worked for me)

network:
  version: 2
  ethernets:
    enp0s4:
      addresses: [192.168.1.{Static IP}/24]
      routes:
        - to: default
          via: 192.168.1.1 
      nameservers:
        addresses: [192.168.1.1, 1.1.1.1]

How I use Nextcloud for safe central storage

2022-05-19T21:17:42Z

Setup admin
External Storage extension
Add my nas zfs dataset
chown -R www-data:www-data on anything nextcloud uploads to.

Nextcloud permissions with ZFS and Ansible-NAS

2022-05-19T13:55:20Z

TL;DR

As the nextcloud docs say… if you want to write to an external volume that location has to be writeable by the user/group www-data on the host system… so if that makes sense to you then this TIL probably isn’t a ton of value.. if not however, read on :)

Case Study

You want to self-host your own cloud and use a smart file system for convenience… Nextcloud and ZFS are pretty common goto answers for each of those problems.

My home NAS is built on ZFS and among other things I have a zpool named tank and nested in there is a tank/nas dataset with several child zfs datasets under that.

I want to use nextcloud mainly for auto-uploading photos from my wife’s and my phones for automatic backups. The issue is that the nextcloud application (I run in Docker) is fixed as the www-data user and so any volume/folder that you want nextcloud to write to needs to be permissioned such that www-data owns it… but I don’t want www-data to own everything in my NAS… so what’s a girl to do?

Solution

Well, one way to go is to just utilize docker volumes, write the data in the container to /var/www/html and let that be the place your data backsup to.

I still wanted nextcloud to automatically write right to my NAS so I created a nextcloud-upload directory inside of tank/nas/media/photos (photos cause that’s all that gets automatically uploaded)

Then I chown -R www-data:www-data /tank/nas/media/photos/nextcloud-upload so that just that sub-folder is owned by www-data. Now everyone’s happy!

See ZFS snapshot disk usage

2022-05-19T06:05:23Z

As I was cleaning up my NAS recently I noticed that I ran out of storage even though my disk usage looked pretty low… turns out I was keeping a mega-ton of ZFS snapshots and due to my own ignorance at the time didn’t realize the storage cost of this!

zfs list-o space tank/home will show the disk usage of the dataset and snapshots in tank/home

Remove ZFS Dataset Specific Snapshots

2022-05-19T05:49:16Z

I started my homelab journey being super naive about ZFS and how to manage the filesystem… that bit me in the butt when transfering a ton of files out of folders and into datasets because ZFS is copy on write so I was essentially duplicating my storage until I got a hair smarter about removing files after they’re moved (rsync –remove-source-file ftw). But I had a ton of snapshots of child datasets with a ton of data that I just never will need, so I learned zfs list -H -o name -t snapshot tank/dataset1/dataset2 will list just the snapshots for dataset2 and if you pipe that into xargs -n1 zfs destroy then you have a way to clear out some snapshots you don’t need!

Jellyfin-Media-Players

2022-04-17T00:00:00Z

I use Jellyfin at home for serving up most of our media - movies and shows etc.

My dream is to have a GPU capable of transcoding any and all of our media for smooth playback on any device… Now, I thought I’d have that by now with my Nvidia Quadro P400 however I have issues left and right with 4k content.

What can I do to still use Jellyfin but get smooth playback?

THe first answer is figure out why I suck with GPUs, but pausing that there’s shorter solutions -> namely, use a media player that’s compatabile with the encoded content!

VLC #

I’ll keep this one short - VLC is great and if you don’t need a netflix like experience, I’d recommend just using it to browse your network drives and play whatever you have

Jellyfin Web Player #

This is the reason I’m writing this post… the web player is great but not everything is supported on all devices

Jellyfin MPV Shim #

This cross-platform cast client is my answer now. You can find the project here

The installation instrauctions are super straightforward for Windows, Mac OS, or Linux.

I’m on Linux and so my install went like this:


sudo apt update
sudo apt install mpv 

pipx install jellyfin-mpv-shim
pipx inject jellyfin-mpv-shim pystray

#profit

I used pipx to install the player as I prefer it for stand alone utilities over pip installing anything globally.

Afer that I just start the player at the terminal with jellyfin-mpv-shim Then in the web browser I can cast my content to the player and bypass the web player (and thus solve much of my transcoding issues) trivially!

self-hosted-media

2022-03-24T00:00:00Z

Self-hosting 1 or several media servers is another common homelab use-case. Getting content for your media servers is up to you, but I’ll show a few ways here to get content somewhat easily!

YouTube Disclaimer at Bottom

you-get #

you-get is a nice cli for grabbing media content off the web.

Installation #

pip install you-get or use ad-hoc with pipx run you-get <url>

Usage #

For example if I wanted to catch up on ancient Chinese military tactics I may go for The Art of War off the Internet Archive…

sandbox  🌱 main 🗑️  ×3🛤️  ×6via 🐍 v3.8.11 (sandbox)  took 15s
❯ you-get https://archive.org/details/art_of_war_librivox -i
Site:       Archive.org
Title:      The Art of War : Sun Tzu : Free Download, Borrow, and Streaming : Internet Archive
Type:       MP3 (audio/mpeg)
Size:       3.87 MiB (4055167 Bytes)

the -i is showing me the info of what would be downloaded without the flag (it’s like a dry run)

sandbox  🌱 main 🗑️  ×3🛤️  ×6via 🐍 v3.8.11 (sandbox)
❯ you-get https://archive.org/details/art_of_war_librivox
Site:       Archive.org
Title:      The Art of War : Sun Tzu : Free Download, Borrow, and Streaming : Internet Archive
Type:       MP3 (audio/mpeg)
Size:       3.87 MiB (4055167 Bytes)

Downloading The Art of War : Sun Tzu : Free Download, Borrow, and Streaming : Internet Archi.mp3 ...
 100% (  3.9/  3.9MB) ├████████████████████████████████████████████████████████████┤[1/1]  917 kB/s

Now I can toss that mp3 onto my booksonic server and study for world domination while I do the dishes!

pytube #

pytube is a python implementation of a youtube downloader that works at the command line or in python!

Installation #

docs

pip install pytube

Usage #

pytube has a lot of functionality, but a quick one would be the --list so you can see what qualities are available

sandbox   main ️  ×3️  ×7via  v3.8.11 (sandbox)  took 2m49s
❯ pytube https://www.youtube.com/watch\?v\=LDU_Txk06tM  --list
Loading video...
<Stream: itag="17" mime_type="video/3gpp" res="144p" fps="8fps" vcodec="mp4v.20.3" acodec="mp4a.40.2" progressive="True" type="video">
<Stream: itag="18" mime_type="video/mp4" res="360p" fps="30fps" vcodec="avc1.42001E" acodec="mp4a.40.2" progressive="True" type="video">
<Stream: itag="22" mime_type="video/mp4" res="720p" fps="30fps" vcodec="avc1.64001F" acodec="mp4a.40.2" progressive="True" type="video">
<Stream: itag="313" mime_type="video/webm" res="2160p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="401" mime_type="video/mp4" res="2160p" fps="30fps" vcodec="av01.0.12M.08" progressive="False" type="video">
<Stream: itag="271" mime_type="video/webm" res="1440p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="400" mime_type="video/mp4" res="1440p" fps="30fps" vcodec="av01.0.12M.08" progressive="False" type="video">
<Stream: itag="137" mime_type="video/mp4" res="1080p" fps="30fps" vcodec="avc1.640028" progressive="False" type="video">
<Stream: itag="248" mime_type="video/webm" res="1080p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="399" mime_type="video/mp4" res="1080p" fps="30fps" vcodec="av01.0.08M.08" progressive="False" type="video">
<Stream: itag="136" mime_type="video/mp4" res="720p" fps="30fps" vcodec="avc1.4d401f" progressive="False" type="video">
<Stream: itag="247" mime_type="video/webm" res="720p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="398" mime_type="video/mp4" res="720p" fps="30fps" vcodec="av01.0.05M.08" progressive="False" type="video">
<Stream: itag="135" mime_type="video/mp4" res="480p" fps="30fps" vcodec="avc1.4d401f" progressive="False" type="video">
<Stream: itag="244" mime_type="video/webm" res="480p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="397" mime_type="video/mp4" res="480p" fps="30fps" vcodec="av01.0.04M.08" progressive="False" type="video">
<Stream: itag="134" mime_type="video/mp4" res="360p" fps="30fps" vcodec="avc1.4d401e" progressive="False" type="video">
<Stream: itag="243" mime_type="video/webm" res="360p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="396" mime_type="video/mp4" res="360p" fps="30fps" vcodec="av01.0.01M.08" progressive="False" type="video">
<Stream: itag="133" mime_type="video/mp4" res="240p" fps="30fps" vcodec="avc1.4d4015" progressive="False" type="video">
<Stream: itag="242" mime_type="video/webm" res="240p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="395" mime_type="video/mp4" res="240p" fps="30fps" vcodec="av01.0.00M.08" progressive="False" type="video">
<Stream: itag="160" mime_type="video/mp4" res="144p" fps="30fps" vcodec="avc1.4d400c" progressive="False" type="video">
<Stream: itag="278" mime_type="video/webm" res="144p" fps="30fps" vcodec="vp9" progressive="False" type="video">
<Stream: itag="394" mime_type="video/mp4" res="144p" fps="30fps" vcodec="av01.0.00M.08" progressive="False" type="video">
<Stream: itag="139" mime_type="audio/mp4" abr="48kbps" acodec="mp4a.40.5" progressive="False" type="audio">
<Stream: itag="140" mime_type="audio/mp4" abr="128kbps" acodec="mp4a.40.2" progressive="False" type="audio">
<Stream: itag="249" mime_type="audio/webm" abr="50kbps" acodec="opus" progressive="False" type="audio">
<Stream: itag="250" mime_type="audio/webm" abr="70kbps" acodec="opus" progressive="False" type="audio">
<Stream: itag="251" mime_type="audio/webm" abr="160kbps" acodec="opus" progressive="False" type="audio">

pytube <url> --itag <> will download the specific itag from the list.

Notice that some itags are videos and others audio - so you can download just the music of a YT video.

pytube also works in python…

sandbox ↪ main v3.8.11 ipython
❯ from pytube import YouTube

sandbox ↪ main v3.8.11 ipython
❯ [x for x in YouTube("https://www.youtube.com/watch?v=LDU_Txk06tM").streams]

[
    <Stream: itag="17" mime_type="video/3gpp" res="144p" fps="8fps" vcodec="mp4v.20.3" acodec="mp4a.40.2" progressive="True" type="video">,
    <Stream: itag="18" mime_type="video/mp4" res="360p" fps="30fps" vcodec="avc1.42001E" acodec="mp4a.40.2" progressive="True" type="video">,
    <Stream: itag="22" mime_type="video/mp4" res="720p" fps="30fps" vcodec="avc1.64001F" acodec="mp4a.40.2" progressive="True" type="video">,
    <Stream: itag="313" mime_type="video/webm" res="2160p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="401" mime_type="video/mp4" res="2160p" fps="30fps" vcodec="av01.0.12M.08" progressive="False" type="video">,
    <Stream: itag="271" mime_type="video/webm" res="1440p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="400" mime_type="video/mp4" res="1440p" fps="30fps" vcodec="av01.0.12M.08" progressive="False" type="video">,
    <Stream: itag="137" mime_type="video/mp4" res="1080p" fps="30fps" vcodec="avc1.640028" progressive="False" type="video">,
    <Stream: itag="248" mime_type="video/webm" res="1080p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="399" mime_type="video/mp4" res="1080p" fps="30fps" vcodec="av01.0.08M.08" progressive="False" type="video">,
    <Stream: itag="136" mime_type="video/mp4" res="720p" fps="30fps" vcodec="avc1.4d401f" progressive="False" type="video">,
    <Stream: itag="247" mime_type="video/webm" res="720p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="398" mime_type="video/mp4" res="720p" fps="30fps" vcodec="av01.0.05M.08" progressive="False" type="video">,
    <Stream: itag="135" mime_type="video/mp4" res="480p" fps="30fps" vcodec="avc1.4d401f" progressive="False" type="video">,
    <Stream: itag="244" mime_type="video/webm" res="480p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="397" mime_type="video/mp4" res="480p" fps="30fps" vcodec="av01.0.04M.08" progressive="False" type="video">,
    <Stream: itag="134" mime_type="video/mp4" res="360p" fps="30fps" vcodec="avc1.4d401e" progressive="False" type="video">,
    <Stream: itag="243" mime_type="video/webm" res="360p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="396" mime_type="video/mp4" res="360p" fps="30fps" vcodec="av01.0.01M.08" progressive="False" type="video">,
    <Stream: itag="133" mime_type="video/mp4" res="240p" fps="30fps" vcodec="avc1.4d4015" progressive="False" type="video">,
    <Stream: itag="242" mime_type="video/webm" res="240p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="395" mime_type="video/mp4" res="240p" fps="30fps" vcodec="av01.0.00M.08" progressive="False" type="video">,
    <Stream: itag="160" mime_type="video/mp4" res="144p" fps="30fps" vcodec="avc1.4d400c" progressive="False" type="video">,
    <Stream: itag="278" mime_type="video/webm" res="144p" fps="30fps" vcodec="vp9" progressive="False" type="video">,
    <Stream: itag="394" mime_type="video/mp4" res="144p" fps="30fps" vcodec="av01.0.00M.08" progressive="False" type="video">,
    <Stream: itag="139" mime_type="audio/mp4" abr="48kbps" acodec="mp4a.40.5" progressive="False" type="audio">,
    <Stream: itag="140" mime_type="audio/mp4" abr="128kbps" acodec="mp4a.40.2" progressive="False" type="audio">,
    <Stream: itag="249" mime_type="audio/webm" abr="50kbps" acodec="opus" progressive="False" type="audio">,
    <Stream: itag="250" mime_type="audio/webm" abr="70kbps" acodec="opus" progressive="False" type="audio">,
    <Stream: itag="251" mime_type="audio/webm" abr="160kbps" acodec="opus" progressive="False" type="audio">
]

YouTube Frontends #

There’s 2 really good options for self-hosting a YT front-end…

Tube Archivist

YouTubeDL-Material

They have their pros and cons. You can also build yourself with the above utilities and use Plex or Jellyfin to serve up videos…

Your self-hosting journey is up to you!

YouTube #

Downloading YouTube videos is a bit of a sore topic… Mainly you don’t to hurt creators who rely on YT ad revenue for their livlihood.

Then again, maybe you’re a vigilante who knows that YT also monetizes videos for their own gain and that the creators don’t see that money either!

The solution is pretty easy and is 2-fold…

Download YT videos
Personally support the content creators you follow via paypall, patreon, or whatever else they might have set-up…. even a buck or two a month is more than they’d get from your ad revenue explicitly plus it all goes to them!

Truenas-And-Wireguard

2022-03-23T00:00:00Z

NAS #

One of the most common use cases for self-hosting anything is a file share system. I have been a fan of TrueNAS for a while. I currently use TrueNAS Core at home, and plan to consider transitioning to TrueNAS Scale soon.

Blog post forthcoming on that!

VPN #

I don’t write a ton about homelabbing yet but one of the first things to set up whether you have a massive homelab or a little raspberry pi would be a self-hosted VPN. I have notes on wireguard here.

I finally have a need to put my TrueNAS box on my wireguard network in order to transfer files to other devices that are outside my LAN.

There is a handy tutorial on setting this up via the GUI here. They walk you through setting up 2 tunables wireguard. One to enable the the connection and one to setup the network interface. Next you create a Post Init script which will check that the right directories exist and will copy the wireguard config that hasn’t been made yet to the proper location and finally starts wireguard.

The above is just copy/paste from the tutorial but the final step, although not super tricky, isn’t the same for everyone as it depends on your wireguard config and network setup.

The final step is for you create the relevant wireguard config (see my post but I just use pivpn -a) and send that config over to your TrueNAS box!

For me this final work flow looked like this:


ssh user@vpn-server

pivpn -a

<follow prompts>

scp ~/configs/truenas.conf root@<truenas ip>:/root/wg0.conf

Bug? #

The script in the tutorial for starting the wireguard service is straight forward enough however my TrueNAS box didn’t get the wireguard interface up and running on reboot.

Easy enough solution:

ssh root@<truenas ip>
/usr/local/etc/rc.d/wireguard start

We can check that the interface is now working with ifconfig and should see something like the following:

wg0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1420
        options=80000<LINKSTATE>
        inet x.x.x.x --> x.x.x.x netmask 0xffffff00
        groups: tun
        nd6 options=101<PERFORMNUD,NO_DAD>
        Opened by PID 1325

Gotcha! #

Here’s another thing I had to navigate when setting this up.

My pivpn configuration sets the endpoint for my wireguard clients to paynepride.com:<port forwarded to wireguard server>

What this means is that when I check the wireguard config for TrueNAS which is on my home network it resolves paynepride.com to the server I have running my reverse proxy. However! My reverse proxy is not responsible for my vpn traffic and so the traffic was just getting dropped - instead I needed to change the wireguard config just for my truenas box to piont to the local address of my vpn server. This really threw me for a loop today but is just another reminder that if you have network problems it’s probably DNS…

Give her the ‘ol reboot and now if I check wg I should see some traffic on my wireguard tunnel!

Wireguard

2022-03-12T00:00:00Z

VPN #

Virtual Private Networks are a big deal, and this shouldn’t be considered anything even close to a guide on using them. Here are just my notes and some setup for how I use wireguard at home.

Wireguard #

Wireguard is an awesome peer-to-peer VPN tunnel that makes it really easy for me to get into my home network when I’m out and about. My main reasons for this are 1. I don’t trust public wi-fi and 2. I want to use pi-hole for ad blocking when I’m not at home

Wireguard can be configured as a “peer-to-site” VPN tunnel as well. My vpn setup let’s me jump to various machines on my network from anywhere!

I use pivpn in a VM that’s already running pi-hole to host my wireguard server. It’s super easy to setup just by following the instructions on the pivpn site.

The reason I like it is that I have a nice cli for managing wireguard configs.

dumbledore@pihole-vpn:~$ pivpn
::: Control all PiVPN specific functions!
:::
::: Usage: pivpn <command> [option]
:::
::: Commands:
:::    -a, add              Create a client conf profile
:::    -c, clients          List any connected clients to the server
:::    -d, debug            Start a debugging session if having trouble
:::    -l, list             List all clients
:::   -qr, qrcode           Show the qrcode of a client for use with the mobile app
:::    -r, remove           Remove a client
:::  -off, off              Disable a user
:::   -on, on               Enable a user
:::    -h, help             Show this help dialog
:::    -u, uninstall        Uninstall pivpn from your system!
:::   -up, update           Updates PiVPN Scripts
:::   -bk, backup           Backup VPN configs and user profiles

When I’m ready to add a new client to my wg network, it’s as easy as pivpn add and follow the instructions. The easiest part here is that you’ll be given a QR code in the terminal that you can just scan with the client (like a smart phone) and you’ll have your wireguard config handled by the app (oh right, download the wireguard app) in no time!

Adblock-Coverage

2022-03-07T00:00:00Z

I run pi-hole at home for ad blocking and some internal DNS/DHCP handling.

pi hole posts on the way

One thing I’ve never put too much thought in is asking “how well am I doing at blocking?” There’s lots of ways to measure that depending on what you care about but I just learned of adblock tester. It’s awesome and gave me a quick glimpse into how my pi-hole is performing on keeping my webpages clean and my DNS history private!

Credits to d3ward for the awesome tool!

Webservers-And-Indexes

2022-03-06T00:00:00Z

I host a lot of services in my homelab, but they’re mostly dockerized applications so I have never had to care much about how content gets served up. Today I had several little concepts click into place regarding webservers, and it was a similar experience to when I started homelabing and didn’t know what a “server” was in the first place.

Servers

A “server” can have a lot of different meanings but specifically in my world it was a physical server, like my PowerEdge R610 which acts as my main “home server”. But then on my server, I have other servers… Jellyfin is my main media server - but that’s obviously not a hardware thing, that’s software. This is certainly not a groundbreaking thing but it was a tiny piece to the puzzle that I was missing… that “server” is highly contextual.

Webservers

Something that confused the heck out of me when I first started down the road of having a server was what a webserver even was… I always thought the “webserver” was just “a server that hosts a website”… and yes, that’s true, but also it wasn’t true in how I understood “server”. It turns out that across my 40-odd dockerized services I have at home that I must have about 40-odd web servers running, each docker container is spinning up its own!

So something I have wanted to do for a long time is put my theology notes online for my small group to access whenever they might want… it doesn’t need to be fancy or anything. My issue was not knowing what to even Google. I tried “How to serve up static html” but that kind of search is for people who know what a “static” site is - I am not one of those people. I kept running across nginx and apache things, wordpress and other website building tools, etc. In fact I only recently learned that JavaScript assets cann still be considered static so I am a complete baby in the web-dev space.

What I really wanted was just a simple landing page with a link to each of my “posts” which are in the form of a single html file each that I can easily export from my tiddlywiki (I have a post about tiddlywiki here)

The first win python -m http.server right in the directory I kept my html files in and that got me what I wanted functionally. But then I wanted just a hair more organization… I started looking for a way to dynamically generate an index for a directory of html files but again the verbiage of that Google search just wasn’t helping me - I didn’t want anything complicated and I knew that what I wanted had to be easy…

The Index

Luckily I randomly came across a SO that mentioned a Linux utility called tree which does exactly what I wanted!

See my TIL on tree here

So now it goes like this:

Take notes on X in my tiddlywiki
Export that tiddler to a html file
Put that html file into a notes folder in my github repo for small group notes
Use tree to generate an index.html of each of those files in the notes directory
Use python -m http.server to start a web server that lands me at the index.html and now I can click through to any post!

It’s not fancy but it’s functional… This site/blog is built with markdown and markata and I wanted way more functionality in my tech notes. But for this simple use case I learned a ton about how content gets served up on a webpage and my small group benefits from the easy access as well!

Traefik

2022-03-06T00:00:00Z

Traefik

If you don’t know about traefik and you need a reverse-proxy then you might want to check it out. I used to use nginx for my reverse proxy but the config was over my head, and once it was working I was afraid to touch it. Traefik brings a lot to the table, my main uses are reverse proxy and ip whitelisting, but it’s doing even more under the hood that I don’t have a full-grasp of yet.

I like Traefik a lot because once I get some basic config up it’s incredibly easy to add services into my homelab whether they run on my primary server or not. This will not be exhaustive but I’ll outline my simple setup process of traefik and how I add services whether they are in docker or not.

Docker

In 2022 I’m still a docker fan-boy and I run my traefik instance in a docker container. This isn’t necessary but I love the portability since my homelab is very dynamic at the moment. And even if it wasn’t I’d still want to keep traefik in docker because deployment and updating are just so flipping easy

A simple docker-compose file for traefik might look like this:

name: traefik
image: "traefik:v2.4"
network_mode: host
volumes:
  - "docker-data/traefik/traefik.toml:/etc/traefik/traefik.toml:ro"
  - "docker-data/traefik/config.yml:/etc/traefik/config.yml:ro"
  - "docker-data/traefik/letsencrypt:/letsencrypt:rw"
  - "/var/run/docker.sock:/var/run/docker.sock:ro"  # for auto-discovery
env: "{{ traefik_environment_variables }}"
restart_policy: unless-stopped
memory: "1g"

Ansible deployment

I plan to have more on my homelab and Ansible on this site eventually…

I use Ansible to deploy most of my services at home, including traefik. My main homelab repo is here which is a fork of Ansible NAS.

If you want my stuff then be sure to go to the user/nic branch on my fork

You can see the ansible stuff for traefik here

Config

I use a traefik.toml as the main config and it looks something like this. With ansible a lot of this is done through template variables but this is the general idea. This config tells traefik what ports to listen and forward on, and gives the names to be referenced by docker labels (down below).

Traefik also has a handy web ui that with this config you can find on port 8080. There is a providers section - which is one of the biggest selling points of traefik for me. I have a docker provider configured and a static file.

The docker provider lets traefik auto-discover new services that I deploy and automatically handle the routing! The static file lets me easily add non-dockerized service routing, or routing to dockerized services on another host (I think traefik has an easier way to do this automatically but I don’t do it often enough to need that kind of automation). Then at the bottom is the SSL cert stuff. Using Let’s Encrypt is pretty easy and I use Cloudflare as my DNS provider


[entryPoints]
[entryPoints.web]
address = ":80"

[entryPoints.web.http.redirections.entryPoint]
to = "websecure"

[entryPoints.websecure]
address = ":443"

[entryPoints.websecure.http.tls]
certResolver = "letsencrypt"

[entryPoints.websecure.http.tls.domains]
main = "example.com"
sans = [
"*.example.com"
]

[entryPoints.traefik]
address = ":8080"

[providers]
providersThrottleDuration = "1s"
[providers.docker]
exposedbydefault = false
[providers.file]
filename = "/etc/traefik/config.yml"

[api]
insecure = true
dashboard = true

[log]
level = "INFO"

[ping]
terminatingStatusCode = 0

[certificatesResolvers]
[certificatesResolvers.letsencrypt]
[certificatesResolvers.letsencrypt.acme]
email = "my_email@example.com"
storage = "/letsencrypt/acme.json"
caserver = "https://acme-staging-v02.api.letsencrypt.org/directory"  # le staging, not prod

[certificatesResolvers.letsencrypt.acme.dnsChallenge]
provider = "cloudflare"

Providers.file

To my knowledge there isn’t much to configure on the docker provider side of things until you deploy a service. But the provider config file should get a little screen time here.

The file defines a traefik http router for each service you define, in this case just pihole.

Here I am adding my pihole instance which is not run inside docker but is inside a VM on another host. I want the entryPoints to be set to websecure which is configured above in the http redirects. I want some middlewares, addprefix-pihole and default-headers, which I’ll explain below. I set letsencrypt as the cert certResolver. Finally I name the service pihole.

Then in the services section I configure where pihole is located by just giving the internal IP for traefik to route to. Finally I define my middlewares. To get to the pihole homepage you need to use the route /admin so I want that added automatically when I go to pihole.example.com so I come to pihole.example.com/admin. And I wanted to restrict access to just my internal network and my wireguard network - this is done with the default-whitelist. The last thing is to configure a chain of middlewares that I called secured which is just easier for the docker labels later on.

With this config in play though, traefik will know about the route pihole.example.com and handle the ip whitelisting and load balancing for me.

http:
 #region routers 
  routers:
    pihole:
      entryPoints:
        - "websecure"
      rule: "Host(`pihole.example.com`)"
      middlewares:
        # - default-headers
        - addprefix-pihole
        - default-whitelist
      tls: 
        certResolver: letsencrypt
      service: pihole
  #region services
  services:
    pihole:
      loadBalancer:
        servers:
          - url: "http://192.168.1.3:80"
        passHostHeader: true
  #endregion
  middlewares:
    addprefix-pihole:
      addPrefix:
        prefix: "/admin"
    https-redirect:
      redirectScheme:
        scheme: https

    default-headers:
      headers:
        frameDeny: true
        sslRedirect: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN

    default-whitelist:
      ipWhiteList:
        sourceRange:
        - "10.6.0.0/24"  # wg
        - "192.168.1.0/24"  # lan
        - "172.17.0.0/16"  # docker

    secured:
      chain:
        middlewares:
        - default-whitelist
        - default-headers

Docker labels

Now the real magic is with Docker. Here is an example docker-compose file for spinning up a jellyfin server that you want to expose to the world, or at least access at home with jellyfin.example.com instead of http://192.168.1.N:8096…

I left some of the ansible variable stuff in here, but the main part to be concerned with is the labels section…

We define just a few labels to throw onto this docker container which let’s traefik discover it automatically and apply any settings necessary (like my ipWhiteList).

traefik.enable is either True or False.
traefik.http.router.jellyfin.rule defines an http router called jellyfin and sets the url to jellyfin.example.com (if example.com was my ansible_nas_domain)
traefik.http.routers.jellyfin.tls.certresolver is set to letsencrypt since I use LE for my wildcard certs.
traefik.http.routers.jellyfin.tls.domains[0].main will just be example.com -> and this should remind you of the toml file above
traefik.http.routers.jellyfin.tls.domains[0].sans is set to *.example.com
traefik.http.services.jellyfin.loadbalancer.server.port is set to jellyfin’s default http port of 8096, which tells traefik which port to point to for this service.

name: jellyfin
image: linuxserver/jellyfin
volumes:
  - "{{ jellyfin_config_directory }}:/config:rw"
  - "{{ jellyfin_movies_directory }}:/movies:{{ jellyfin_movies_permissions }}"
  - "{{ jellyfin_music_directory }}:/music:{{ jellyfin_music_permissions }}"
  - "{{ jellyfin_photos_directory }}:/photos:{{ jellyfin_photos_permissions }}"
  - "{{ jellyfin_tv_directory }}:/tv:{{ jellyfin_tv_permissions }}"
  - "{{ jellyfin_books_directory }}:/books:{{ jellyfin_books_permissions }}"
  - "{{ jellyfin_audiobooks_directory }}:/audiobooks:{{ jellyfin_audiobooks_permissions }}"
ports:
  - "{{ jellyfin_port_http }}:8096"
  - "{{ jellyfin_port_https }}:8920"
env:
  TZ: "{{ ansible_nas_timezone }}"
  PUID: "{{ jellyfin_user_id }}"
  PGID: "{{ jellyfin_group_id }}"
restart_policy: unless-stopped
memory: 1g
labels:
  traefik.enable: "{{ jellyfin_available_externally }}"
  traefik.http.routers.jellyfin.rule: "Host(`jellyfin.{{ ansible_nas_domain }}`)"
  traefik.http.routers.jellyfin.tls.certresolver: "letsencrypt"
  traefik.http.routers.jellyfin.tls.domains[0].main: "{{ ansible_nas_domain }}"
  traefik.http.routers.jellyfin.tls.domains[0].sans: "*.{{ ansible_nas_domain }}"
  traefik.http.services.jellyfin.loadbalancer.server.port: "8096"

And just like that traefik will automagically find your jellyfin container and route jellyfin.example.com to it!

Posts tagged: homelab Archive

The Faulted Disk: harbor Replacement Writeup

What happened #

The difference from last time #

Diagnosis #

The swap (the annoying part) #

The replace #

The monitoring that made this possible #

Remaining homework #

💭 Docker | Atuin Docs

Homelabbing Realization - Configs and Git

Realization #

RepoFlow Robot User

Proxy Pull Docker Images From Self-Hosted Container Registry Through Self-Hosted Repoflow

PyPi #

Docker #

💭 My Ultimate Self-hosting Setup

Using restic to backup my home directory

Jellyfin

Ghost

The Homelab

💭 Docker Brings Compose to the AI Agent Era | Docker

💭 restic · Backups done right!

Thoughts To Nostr

💭 HomeBox

💭 Self-Hosting A Cluster On Old Phones | Hackaday

Github Star Sorter

temporal

Traefik and gRPC for Temporal at home

Nextcloud PHP Opache Memory Consumption

Intro

In the container #

In the config #

Quit issueing 200s for 500s

Stop this nonsense

💭 Control Your n8n Instance Remotely with Telegram Bot Commands …

Thinking In Diagrams

💭 pypeaday/whose-turn-is-it: Simple uv webapp to determine order…

Homelab Youtube Podcasts to AudioBookShelf Pipeline

source_up for nested .envrc

TLDR

Context

Problem #

Fix #

Stable Diffusion QR Codes for Josh

After my blunder

Final Candidates

Some favorites

Codes and prompts

Top Contendor #

U-Blue Aurora Upgrade Broke HWA Docker

Related #

Homelab Pinchflat

If You Want Something Make It So - Song Style

ZSA typing challenge issue

ZSA

Today’s Challenge #

Double Check Your DNS Records

My Blog

Terraform #

💭 qwen3

💭 News - Apps - App Store - Nextcloud

Example For Andrew

I built a simple app for adding images to my blog

Double check your CIDR blocks!

tl;dr

The Issue

Typo! #

homelab-journey-part-1

Backups interrupted by full disk usage

homelab-computer-vision-pipelines

Backups are dope

SearXNG

Grafana + Gotify for push notifications

💭 audiobookshelf

Add a healthcheck to your FastAPI app

Jellyfin container updates for HWE + AMD

hostnamectl to easily change hostname

DNS Broke After Reboot - Ubuntu 22.04

OPNSense Bootstrap Recovery