Posts tagged: cli

source_up for nested .envrc

2025-06-17T09:52:17Z

TLDR

I have a use case for nesting directories with cascading .envrc files. and ran into an issue where the parent environment was being reset by the child environment. The fix is source_up

Context

I manage some infrastructure at home with a folder like this:

.
├── homelab
│   ├── .envrc
│   └── stack1
│       ├── .envrc

I do this so that the “homelab-wide” terraform variables can be exported with TF_VAR_the_variable in the homelab/.envrc and then the stack specific variables in stack1/.envrc get exported like TF_VAR_stack_specific_var.

For example, my cloudflare account id is needed in every stack, but the cloudflare zone id is different per stack - so instead of copying the account id var to every stack (however the vars are managed), I can set it in a parent .envrc and share it in nested stack directories

Problem #

# homelab/.envrc
#!/bin/bash
export VAR_FOO=BAR

So when I cd homelab I get

nic in /tmp/homelab   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ direnv allow
direnv: loading /tmp/homelab/.envrc
direnv: export +VAR_FOO

nic in /tmp/homelab   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ env | grep VAR
VAR_FOO=BAR

You can see VAR_FOO is set… but now I’ll cd stack1

nic in /tmp/homelab   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ cd stack1
direnv: error /tmp/homelab/stack1/.envrc is blocked. Run `direnv allow` to approve its content

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ direnv allow
direnv: loading /tmp/homelab/stack1/.envrc
direnv: export +VAR_BAZ

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ env | grep VAR
VAR_BAZ=BAM

As you can see VAR_FOO was unset, and VAR_BAZ is set… so homelab/.envrc was blown away!

Fix #

Edit stack1/.envrc with a source_up command

# stack1/.envrc # NEW
#!/bin/bash
source_up  # source the parent environment
export VAR_BAZ=BAM

After the edit

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ vim .envrc
direnv: error /tmp/homelab/stack1/.envrc is blocked. Run `direnv allow` to approve its content

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET  took 5s
⬢ [devbox] ❯ direnv allow
direnv: loading /tmp/homelab/stack1/.envrc
direnv: loading /tmp/homelab/.envrc
direnv: export +VAR_BAZ +VAR_FOO

nic in /tmp/homelab/stack1   (dev) 󰒄 󱔎 NO PYTHON ENVIORNMENT SET
⬢ [devbox] ❯ env | grep VAR
VAR_FOO=BAR
VAR_BAZ=BAM

Both vars are set!

I was wrecked by a weird combo of » and -e

2025-04-23T08:44:59Z

TL;DR #

If state matters then check it in the beginning or handle it on a failure… Let me explain

I ran into some trouble recently almost losing some encrypted data… Now this would be the second time I’ve had that happen, so I’m going to write a little bit about it now that I figured it out

The Stage #

Here’s the scenario - I use ansible-vault to keep some sensitive data in a few public repos, and I keep the keys I use in Bitwarden Secrets Manager.

I use just as a command runner and have common just encrypt and just decrypt recipes throughout many justfiles. It might look something like this


get-vault-key:
    bws secret get $HOMELAB_BOT_VAULT_KEY_ID  | jq -r '.value'

encrypt:
    #!/bin/bash
    set +e
    just get-vault-key > key
    ansible-vault encrypt ./secret-file --vault-password-file key
    ansible-vault encrypt ./secret-file2 --vault-password-file key
    rm key

decrypt:
    #!/bin/bash
    set +e
    just get-vault-key > key
    ansible-vault decrypt ./secret-file --vault-password-file key
    ansible-vault decrypt ./secret-file2 --vault-password-file key
    rm key

Spoiler #

Above is a good example of the 2 recipes, but prior to this morning they looked like this


get-vault-key:
    bws secret get $HOMELAB_BOT_VAULT_KEY_ID  | jq -r '.value'

# BAD EXAMPLES! DO NOT DO THIS

encrypt:
    set -e
    just get-vault-key >> key
    ansible-vault encrypt ./secret-file --vault-password-file key
    ansible-vault encrypt ./secret-file2 --vault-password-file key
    rm key

decrypt:
    set -e
    just get-vault-key >> key
    ansible-vault decrypt ./secret-file --vault-password-file key
    ansible-vault decrypt ./secret-file2 --vault-password-file key
    rm key

Problem #

So here’s the story - in real life I started using distrobox when I ran into this and I thought it was an issue with different versions of ansible installed in distrobox and on my desktop. After a few more odd deployment workflows I think I’ve determined that the real problem is that >> appends to a file (see the issue yet?), but if my recipes fail (say I try to encrypt an already encrypted file) then the command exits but what’s left on my disk? key is still there… and the next time I go to run a command I’ll echo the real key to the key file again and if I called encrypt when the first or more files in the list were decrypted then they’ll be encrypted not with my key, but with my key repeated as many times as I ran a command with just get-vault-key >> key before ever removing the key file!


nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ cat secret-file
Danger, Will Robinson

nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ just encrypt
bws secret get $HOMELAB_BOT_VAULT_KEY_ID  | jq -r '.value'
Encryption successful

nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ cat secret-file
$ANSIBLE_VAULT;1.1;AES256
39616332393662346639633030633766666336323939346138633238626239363733316431333737
6463663763366434376437303335643431663431326135300a636261663636336139323033316232
37666439383131393631333332353731633661396431633834326432363936613331623135666565
6364363039663933620a326563323931643632323031396664646363613636613562366166386439
63653661653037393538356461323239396630643338393231306163343964623866

nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ just decrypt
bws secret get $HOMELAB_BOT_VAULT_KEY_ID  | jq -r '.value'
Decryption successful

nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ cat secret-file
Danger, Will Robinson

So as you can see, I have secret-file here, and I can encrypt and decrypt just fine. But if I have that >> in the recipe, add a second file, and a failure in a command - well then I’ll start getting confused…

Follow along with the workflow of adding a file that I’ll need encrypted

Only secret-file is in the just recipe here - I add secret-file2 after

nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ just decrypt
bws secret get $HOMELAB_BOT_VAULT_KEY_ID  | jq -r '.value'
ERROR! input is not vault encrypted data. /var/home/nic/projects/personal/pype.dev/secret-file is not a vault encrypted file for /var/home/nic/projects/personal/pype.dev/secret-file
error: Recipe `decrypt` failed with exit code 1

nic in pype.dev   main   ×2  ×3  ×8 via   v3.11.10(pype-dev)   (dev) 󰒄 
✗ echo "Good - secret-file is already decrypted! So lets add another file"
Good - secret-file is already decrypted! So lets add another file

nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ echo "Foo Bar Bam Baz" > secret-file2                                    

nic in pype.dev   main   ×2  ×3  ×8 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ echo "Added secret-file2 to my Just recipes"
Added secret-file2 to my Just recipes

nic in pype.dev   main   ×2  ×3  ×8 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ just encrypt
bws secret get $HOMELAB_BOT_VAULT_KEY_ID  | jq -r '.value'
Encryption successful
Encryption successful

nic in pype.dev   main   ×2  ×3  ×7 via   v3.11.10(pype-dev)   (dev) 󰒄 
❯ just decrypt
bws secret get $HOMELAB_BOT_VAULT_KEY_ID  | jq -r '.value'
ERROR! Decryption failed (no vault secrets were found that could decrypt) on /var/home/nic/projects/personal/pype.dev/secret-file for /var/home/nic/projects/personal/pype.dev/secret-file
error: Recipe `decrypt` failed with exit code 1

nic in pype.dev   main   ×2  ×3  ×8 via   v3.11.10(pype-dev)   (dev) 󰒄 
✗ echo "Oh no!"

So what happened?

Recall that my recipes originally had set -e at the top and used >> to cat my secret to a file key. Well, when the fir| decryption recipe failed the key file was left behind. Then when I went to encrypt both of my files the recipe got the key again and made a keyfile like this:

secretKey
secretKey

The encryption recipe encrypted the files with this key, removed the key file, and then the next decryption recipe failed because when it got the key and made the file it looked like

secretKey

So the solution in the end is to remake the key file every time (> instead of >>), or check if it exists, or handle the error when the recipe fails, or just keep going (set +e)

Fin #

And that’s how I lost some secrets in my homelab once, and how I almost lost a second time if I hadn’t noticed the key file in my file tree

Monolith - download and save site as html

2025-03-16T16:03:31Z

TIL: Monolith #

I needed to save a website earlier and this was just what I needed…

https://github.com/Y2Z/monolith

rsync like a pro

2024-12-11T10:52:23Z

I am moving a hefty amount of data to a new ZFS pool due to some corruption and I want to avoid using zfs send/recv for this just to make sure I don’t propagate any corrupted data to my new pool.

I’ve used rsync for simple things before but I needed this to be a little smarter and I wanted to see simple progress without flooding my terminal with a billion filenames.

TLDR #

TLDR: rsync -aHAX --chmod=Da+s --info=progress2 --inplace --exclude='encrypted/docker/frigate-media' /tank/ /harbor/

Explanation #

-aHAX: Preserves attributes (archive mode, hard links, ACLs, extended attributes). –chmod=Da+s: Ensures the setgid bit is applied to directories. –info=progress2: Provides detailed progress information, including overall data transfer stats. –inplace: Writes directly to the destination file, avoiding temporary files (useful for large files). –exclude=‘encrypted/docker/frigate-media’: Excludes the specified path (relative to the /tank root). /tank/ /harbor/: Ensures the contents of /tank are copied directly into /harbor.

hostnamectl to easily change hostname

2024-12-06T07:25:59Z

hostnamectl is apparently a linux utility for easily changing your hostname in a variety of ways


❯ hostnamectl --help
hostnamectl [OPTIONS...] COMMAND ...

Query or change system hostname.

Commands:
  status                 Show current hostname settings
  hostname [NAME]        Get/set system hostname
  icon-name [NAME]       Get/set icon name for host
  chassis [NAME]         Get/set chassis type for host
  deployment [NAME]      Get/set deployment environment for host
  location [NAME]        Get/set location for host

Options:
  -h --help              Show this help
     --version           Show package version
     --no-ask-password   Do not prompt for password
  -H --host=[USER@]HOST  Operate on remote host
  -M --machine=CONTAINER Operate on local container
     --transient         Only set transient hostname
     --static            Only set static hostname
     --pretty            Only set pretty hostname
     --json=pretty|short|off
                         Generate JSON output

See the hostnamectl(1) man page for details.

I learned there’s transient and static hostnames, so that’s cool…

The thing I needed was hostnamectl --static hostname babyblue-aurora

pretty sweet tool

Restart KDE Plasma

2024-11-08T15:53:52Z

Plasma shits the bed a little too often on Fedora for me right now but I finally have a quick fix…


sudo killall plasmashell

kstart plasmashell

Customize K9s

2024-05-06T11:57:07Z

To customize k9s use the skins from catppuccin or the ones k9s supplies

OUT="${XDG_CONFIG_HOME:-$HOME/.config}/k9s/skins"
mkdir -p "$OUT"
curl -L https://github.com/catppuccin/k9s/archive/main.tar.gz | tar xz -C "$OUT" --strip-components=2 k9s-main/dist

Then edit your k9s config

# ~/.config/k9s/config.yml
k9s:
  ui:
    skin: catppuccin-mocha
    # ...or another flavor:
    # skin: catppuccin-macchiato
    # skin: catppuccin-frappe
    # skin: catppuccin-latte

    # ...or the transparent variants:
    # skin: catppuccin-mocha-transparent
    # skin: catppuccin-macchiato-transparent
    # skin: catppuccin-frappe-transparent
    # skin: catppuccin-latte-transparent

Other k9s skins are available here

Git fetch failing - check your config

2024-04-18T08:32:33Z

I started deploying a website to Cloudflare on a branch called pages. Similar to one of the GH Pages deployment patterns. But when my CI was pushing the branch I couldn’t see it locally…

git fetch -a wasn’t pulling any new branches, and git branch -a was only showing my development and main branches at the remote… so what gives?

I checked my git config, and to this moment I have no idea how this happened but check out my fetch config:

git config --get remote.origin.fetch
+refs/tags/*:refs/tags/*

So to fix this:

git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'

Now git fetch -a works again

> git fetch -a

From github.com:DigitalHarbor7/DigitalHarbor
   357a28a..969b027  develop    -> origin/develop
   c052ac9..6d40210  main       -> origin/main
 * [new branch]      pages      -> origin/pages

Git repo specific SSH Key!

2024-01-04T15:42:15Z

git config –add –local core.sshCommand ‘ssh -i «<PATH_TO_SSH_KEY»>’

Convert Word Doc to PDF with Headless Libreoffice

2023-03-09T06:48:38Z

I’ve been using paperless-ngx to manage all my documents, but every once in a while I’ll get a .docx file to deal with…

Turns out Libreoffice has a headless mode a pdf converter built-in!

libreoffice --headless --convert-to pdf /path/to/file.docx --outdir /path/to/output/directory

Note that --outdir is in fact a directory, not the path to a file

FFMPEG 10-bit videos to 8-bit

2023-01-16T13:15:53Z

ffmpeg -i input.mp4 -map 0 -c:v libx264 -vf format=yuv420p -c:a copy output.mp4

Use non-standard named ssh keys with github

2023-01-03T08:34:50Z

I was getting (publickey denied) when trying to push to GH using ssh. When I tested the connection I saw that a bunch of keys in ``~/.ssh/ were being attempted

✗ ssh git@github.com -vv

...

debug1: Will attempt key: /home/nic/.ssh/id_rsa 
debug1: Will attempt key: /home/nic/.ssh/id_ecdsa 
debug1: Will attempt key: /home/nic/.ssh/id_ecdsa_sk 
debug1: Will attempt key: /home/nic/.ssh/id_ed25519 
debug1: Will attempt key: /home/nic/.ssh/id_ed25519_sk 
debug1: Will attempt key: /home/nic/.ssh/id_xmss 
debug1: Will attempt key: /home/nic/.ssh/id_dsa 

...

debug1: No more authentication methods to try.
git@github.com: Permission denied (publickey).

None of those were the key I setup with GH. So I added an entry into ~/.ssh/config:

Host
github.com
User git
Port 22
Hostname github.com
IdentityFile ~/.ssh/my_custom_github_key
TCPKeepAlive yes
IdentitiesOnly yes

Modal Labs

2022-12-28T21:01:52Z

Playing around with Modal Labs

One of the first things I tried was a regular cron job…

@stub.function(
    schedule=modal.Period(minutes=59), secret=modal.Secret.from_name("my-dummy-secret")
)
def say_hi():
    now = time.ctime()
    secret = os.environ.get("dummy-secret")
    print(f"Hello {os.environ.get('USER', 'Rodney')} at {now}")
    print(f"{secret=}")

This can get deployed with modal deploy --name <app name> <path to .py file with the stub and function defined in it>

This function gets deployed as an app that I conveniently call say_hi (as far as I can tell the app name can be anything - as I add functions to this same app and deploy with the same name to get a new version)

Notice that this also is an example of giving access to a secret - defined in the Modal Labs dashboard

We can take a look at the apps running at https://modal.com/apps

I then added another function to experiment with custom container images and saw then that Modal will just slap a new version on anything provisioned with the same name (intuitive enough for sure) so when I add functions to my .py script and run modal deploy --name say_hi myscript.py over and over, the app called say_hi in the Modal apps dashboard just gets a new version

This means I can spin up several instances of functionally the same app but with different names/versions etc… Q: Maybe there’s gitops or policy stuff builtin to app names then?

I needed to take down an app I deployed as a duplicate but you don’t stop apps by name, you stop them by an id… see below


modal-sandbox/modal_sandbox   main   ×1  ×9 via   v3.10.6(modal-sandbox)
✗ modal app stop --help

 Usage: modal app stop [OPTIONS] APP_ID

 Stop an app.

╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *    app_id      TEXT  [default: None] [required]                                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                                                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯


modal-sandbox/modal_sandbox   main   ×1  ×9 via   v3.10.6(modal-sandbox)
❯ modal app list
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ App ID                    ┃ Description         ┃ State    ┃ Creation time             ┃ Stop time                 ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ap-lzy1AAuVy7POFkUcDKRxpQ │ print_info          │ deployed │ 2022-12-28 20:59:07-06:00 │                           │
│ ap-qYjE45dciqgT3C3CpNp3RL │ say_hi              │ deployed │ 2022-12-28 19:49:22-06:00 │                           │
│ ap-X7FYneUeYV5IKHcyirSb87 │ link-scraper        │ stopped  │ 2022-12-28 15:39:02-06:00 │ 2022-12-28 15:39:04-06:00 │
│ ap-UOXTUU4uSRx2UZypJOcAsk │ example-get-started │ stopped  │ 2022-12-28 15:17:47-06:00 │ 2022-12-28 15:17:49-06:00 │
└───────────────────────────┴─────────────────────┴──────────┴───────────────────────────┴───────────────────────────┘

modal-sandbox/modal_sandbox   main   ×1  ×9 via   v3.10.6(modal-sandbox)
❯ modal app stop ap-lzy1AAuVy7POFkUcDKRxpQ

Git warning!

I ran modal deploy ... after comitting some stuff I wanted to try BUT I had changes in my file I didn’t want to deploy… some git safety would be nice for deployment!

git stash && modal deploy .. && git stash pop

Question for Modal team - in my modal sandbox repo at commit:

aab6162 (HEAD -> main) HEAD@{1}: commit: print base version of my own image to prove it to me
 1 file changed, 2 insertions(+)

An environment variable, BASE_VERSION that I expect to be in my base image was not available to the python function in my Modal app… hopefully the log is still here

GitHub Actions #

I think it’d be easy to deploy from GH actions… just need an API Key…

Deployment

Without gitops it’s already ahrd to manage my versions… need to get that built in either to github or locally sooooon

Nomad?? :D

Pipe to a pager to preserve console output in SSH session

2022-12-18T15:04:02Z

I’m playing with my ansible playbook in a remote tmux session, and I’m no wiz so I don’t know the ins and outs, but I can’t scroll up to get any console log output that’s not already visible on my screen. So I’m starting to end my commands with | less so I can page through the console output!

ansible-playbook plays.yml -v --tags mytag | less

Call basicConfig to get Python log messages in iPython

2022-12-10T14:04:23Z

Logging instead of printing

I am trying to adopt logger.debug instead of print but ran into a confusing thing in ipython during Advent of Code… I riddled by script with logger.debug (yes after setting logging.setLevel('DEBUG')) but in ipython none of my log messages showed up!

import logging

logger = logging.getLogger(__name__)
logger.setLevel("DEBUG")

Turns out what I was missing was a call to basicConfig

import logging

# forget this and your messages are in the ether! or at least not seen in ipython...
logging.basicConfig()

logger = logging.getLogger(__name__)
logger.setLevel("DEBUG")

Bonus

Want your new messages to show up while iterating on something without killing the ipython kernel?

from importlib import reload
reload(logging) # to make sure you get new log messages you add while developing!

Make a series of directories fast!

2022-11-10T15:27:50Z

mkdir s{1..10} will make directories s1, s2, … s10 in one command!

Limit zfs list to avoid docker vomit

2022-10-20T06:39:18Z

zfs list has a flag -r, but if you use zfs driver for docker then you’ll get flooded with every docker volume in the world. zfs list -r -d N will limit the dept of the print out, so zfs list -r -d 2 gives me tank, tank/encrypted, tank/encrypted/docker -> but then I don’t see all the continer volumes

Use the right lspsage, you dope!

2022-10-14T19:41:44Z

If you think glepnir/lspsaga.nvim config will work with tali5/lspsaga.nvim installed… you’re in for a rude awakening. Double check your forks folks!

Suddenly SSH requires a password

2022-10-07T11:14:37Z

ssh -v -i ~/.ssh/id_rsa nic@hogwarts

THen we can look at print outs

cat /var/log/auth.log also showed me that I had too wide permissions on files in ~/.ssh -> probably changed from an rsync job

Check Your BIOS Version On Ubuntu

2022-10-02T18:21:58Z

sudo dmidecode -s bios-version