Posts tagged: dataops Archivehttps://pype.dev/tags/dataops/archive/atom.xml2026-05-07T06:33:03ZAll posts with the tag "dataops"Nic Paynemarkata-goDataOpshttps://pype.dev/dataops/2026-05-07T06:33:03Z2026-05-07T06:33:03Zold: using-restic-to-backup-my-home-directory<p>old: <a href="/using-restic-to-backup-my-home-directory/" class="wikilink" data-title="Using restic to backup my home directory" data-description="I need to backup my personal $HOME to my NAS cause there's a lot in there, and mostly my git projects with .env files all over. Plus some docker data" data-date="2025-07-16" data-preview="I need to backup my personal $HOME to my NAS cause there's a lot in there, and mostly my git projects with .env files all over. Plus some docker data">Using restic to backup my home directory</a></p>
<p>NOTE: zfs-ops is now in my <a href="/forgejo/" class="glossary-term" title="self-hosted code forge - codeberg - fork of gittea" data-hover-title="forgejo">forgejo</a> instance. ghost, ghost-vault, and my desktop clone from forgejo
NOTE: zfs-ops and homelab-mono/dataops are separate, they could be combined</p>
<h2 id="things-to-backup"><span class="heading-wear-glyph">Things to Backup</span> <a href="#things-to-backup" class="heading-anchor">#</a></h2>
<ul>
<li>Desktop
<ul>
<li>projects + workspaces
sans .venv and other stuff</li>
<li>home directory</li>
</ul>
</li>
<li>Server
<ul>
<li>zfs datasets in tank (nas, <a href="/docker/" class="glossary-term" title="docker - container runtime" data-hover-title="docker">docker</a>, etc.)</li>
</ul>
</li>
</ul>
<h2 id="aurora-desktop-disk-layout"><span class="heading-wear-glyph">Aurora (Desktop) Disk Layout</span> <a href="#aurora-desktop-disk-layout" class="heading-anchor">#</a></h2>
<table>
<thead>
<tr>
<th>Device</th>
<th>Size</th>
<th>Pool</th>
<th>Datasets</th>
<th>Mounts</th>
<th>Role</th>
</tr>
</thead>
<tbody>
<tr>
<td>Samsung 970 EVO NVMe</td>
<td>500 GB</td>
<td>—</td>
<td>—</td>
<td><code>/</code>, <code>/home</code>, <code>/boot</code></td>
<td>Boot + OS</td>
</tr>
<tr>
<td>Samsung 870 EVO SATA</td>
<td>1 TB</td>
<td>—</td>
<td>—</td>
<td>—</td>
<td>VMs / scratch / future</td>
</tr>
<tr>
<td>Crucial BX500 SATA</td>
<td>2 TB</td>
<td><code>docker-storage-zfs</code> 🡒 <code>orbit</code> (planned)</td>
<td><code>volumes/</code>, <code>projects/</code></td>
<td><code>/var/docker-storage-zfs</code>, <code>~/projects</code></td>
<td>Primary — Docker volumes + projects</td>
</tr>
<tr>
<td>Seagate IronWolf HDD</td>
<td>4 TB</td>
<td><code>depot</code></td>
<td><code>sanoid/</code>, <code>restic/</code>, <code>nvme-rsync/</code></td>
<td>—</td>
<td>Backup target</td>
</tr>
</tbody>
</table>
<h2 id="how"><span class="heading-wear-glyph">How</span> <a href="#how" class="heading-anchor">#</a></h2>
<ul>
<li>Desktop
<ul>
<li>Will use 2TB disk, renaming docker-storage-zfs to orbin for kicks, I knew I’d regret naming it docker-storage-zfs anyways from <a href="/desktop-crash-2026/" class="wikilink" data-title="Desktop Crash 2026" data-description="Desktop crashed days ago, apparently my primary drive has been going bad for a while and eventually it just died." data-date="2026-04-22" data-preview="Desktop crashed days ago, apparently my primary drive has been going bad for a while and eventually it just died.">Desktop Crash 2026</a></li>
<li><code>projects</code>
<ul>
<li>zfs dataset mounted to ~/projects</li>
<li><input disabled="" type="checkbox"> will need to ignore it in the $HOME backup solution</li>
</ul>
</li>
<li><code>docker-volumes</code> will just be named <code>volumes</code>
<ul>
<li>mounted to <code>/var/docker-storage-zfs/</code>, might change mountpoint, only relevant to my compose files</li>
</ul>
</li>
<li><code>$HOME</code> directory is NOT zfs
<ul>
<li>rsync to a zfs dataset on an external SSD, same one that projects is on but the difference is that projects is mounted from there and the home directory is just backed up to it.</li>
<li>from there backup to NAS just like projects… so no matter what we’ll go from the 2 or 4 TB disk in my desktop to ghost:/tank/… and then tank replicates to to the harbor zpool, and that’s all local in my house across 2 machines (desktop and NAS, NAS has tank and harbor)</li>
</ul>
</li>
<li>Local backup to 4TB disk on device, pool will be <code>depot</code> and I can setup <code>depot</code> -> <code>tank</code> if I find it relevant…
<ul>
<li>the redundancy to my NAS would give me only redundancy, maybe for off-site set something up from <code>depot</code> -> <code>ghost-vault:/tank</code></li>
<li><input disabled="" type="checkbox"> ?? $HOME</li>
<li><input disabled="" type="checkbox"> ?? projects</li>
<li><input disabled="" type="checkbox"> ??docker volumes
<ul>
<li>specific volumes maybe?</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li>Server
<ul>
<li>tank -> harbor is local redundancy</li>
<li>sanoid sends configured zfs datasets from tank -> ghost-vault</li>
</ul>
</li>
</ul>
<h2 id="todos"><span class="heading-wear-glyph">TODOs</span> <a href="#todos" class="heading-anchor">#</a></h2>
<ul>
<li><input checked="" disabled="" type="checkbox"> rename <code>docker-storage-zfs</code> to <code>orbit</code></li>
<li><input checked="" disabled="" type="checkbox"> create <code>projects</code> zfs dataset in <code>orbit</code></li>
<li><input disabled="" type="checkbox"> setup zfs dataset for home directory backup on <code>orbit</code></li>
<li><input checked="" disabled="" type="checkbox"> set mountpoint for projects</li>
<li><input checked="" disabled="" type="checkbox"> set mountpoint for docker volumes dataset</li>
<li><input checked="" disabled="" type="checkbox"> restart the ai stack containers after validating compose file volume path is correct
<ul>
<li>the “…/volume/” part will probably be wrong</li>
<li>just kept the mountpoint as /var/docker-storage-zfs/<em>for orbit/volumes/</em></li>
</ul>
</li>
<li><input disabled="" type="checkbox"> setup appropriate backup (rsync, restic, etc.) for home directory to <code>orbit</code>’s backup dataset</li>
<li><input disabled="" type="checkbox"> setup sanoid snapshot schedule in zfs-ops</li>
<li><input disabled="" type="checkbox"> setup syncoid script/systemd for <code>orbit</code> to <code>depot</code></li>
<li><input disabled="" type="checkbox"> recover the rest of my projects from emergency or home-restore</li>
<li><input disabled="" type="checkbox"> remove orbit/emergency</li>
<li><input disabled="" type="checkbox"> remove orgit/home-restore</li>
<li><input disabled="" type="checkbox"> remove orbit/volumes/mcphub</li>
<li><input disabled="" type="checkbox"> remove orbit/volumes/tdarr maybe?</li>
</ul>
Nic Paynehttps://pype.dev