<rksh/>

An Always-On AI Agent on a Raspberry Pi

Run Hermes Agent on a Pi 4 that boots from a USB pendrive: flash tuning, a gateway that survives reboots, and backups that stay small.

· Rakesh Gautam · 15 min read

I wanted an agent that keeps working when my laptop is closed. Something I can message from my phone, that remembers the previous conversation, and that can run a job at 8am whether or not I am awake.

A Raspberry Pi 4 and a spare 16 GB USB pendrive were enough. Most of the effort went into the disk rather than the agent, and the failures that cost the most time were the ones that only appear after a reboot.

Here is the whole build, in the order I did it, including the four things that broke.

What You End Up With

After a few days of use it sits at 8.2 GB used of 14 GB, around 530 MB of RAM, and 51°C with no throttling. The Pi handles scheduling, memory and tool calls. Nothing about the model changes because the machine is small: the inference happens elsewhere on an API key.

You need a Pi 4 or newer, a 16 GB or larger USB stick, an Ethernet cable, your SSH public key, and an API key for a model provider.

0. Prepare the Pendrive

Raspberry Pi Imager writes Raspberry Pi OS (other) → Raspberry Pi OS Lite (64-bit) straight to the stick. Write to the parent device, not to a partition.

A Pi 4 only boots from USB if its bootloader is recent enough. If you get a rainbow screen instead of a boot, flash Misc utility images → Bootloader (Pi 4 family) → USB Boot first, boot that until the screen goes green, then reflash the OS.

Before you hit Write, open the Settings dialog. It handles the first boot for you:

Use Ethernet for the first boot anyway. The Pi 4’s USB 3.0 ports interfere with 2.4 GHz WiFi, and you want one less variable while bringing the machine up.

What Imager Is Actually Writing

Worth knowing, because it changes what you can edit later. Trixie images no longer use the old firstrun.sh mechanism. They ship cloud-init, and the settings land in user-data on the FAT boot partition:

#cloud-config
hostname: keshu
manage_etc_hosts: true
timezone: Asia/Kolkata
users:
  - name: pi
    groups: users,adm,dialout,audio,netdev,video,plugdev,cdrom,games,input,gpio,spi,i2c,render,sudo
    shell: /bin/bash
    lock_passwd: false
    passwd: "$6$..."                # salted SHA-512, never the plain password
    ssh_authorized_keys:
      - ssh-ed25519 AAAA... you@laptop
enable_ssh: true
ssh_pwauth: false                   # key-only logins

The dialog writes that file for you. If you would rather do it by hand, mount the FAT partition and create three files next to config.txt: user-data, network-config and meta-data. Generate the hash with openssl passwd -6, and leave network-config as shipped if you are on Ethernet. Edit meta-data at your peril: it holds the instance_id, and changing it makes cloud-init treat the boot as a brand new machine.

1. First Boot, Then Clean Up After cloud-init

The first boot takes a couple of minutes. cloud-init creates your user, sets the hostname, and the resize argument in cmdline.txt grows the root filesystem from 2.5 GB to the full stick.

ssh pi@keshu.local
df -h /                        # should show the full card, around 14G

Then remove the password hash, because it is sitting on a FAT partition that anyone holding the stick can read:

sudo passwd pi                 # change it if the hash was ever exposed
sudo tee /boot/firmware/user-data >/dev/null <<'EOF'
#cloud-config
# Applied on first boot; contents cleared for security.
EOF
sudo touch /etc/cloud/cloud-init.disabled

The last line matters more than it looks. Trixie’s cloud-init keeps some modules enabled after the first boot, with growpart and resizefs set to run on every boot. Disabling it stops the machine reconfiguring itself behind your back. Keep the file valid YAML rather than deleting it.

2. Make Flash Survive Being Written To

Every guide about running a Pi from an SD card tells you the card will die. A USB pendrive is no better, and the stock layout makes it worse than necessary: Debian puts a swapfile on the same flash you are trying to protect, and writes to it constantly.

# 1. Confirm swap is in RAM, not on the stick.
#    Trixie already provides zram swap (rpi-swap + systemd-zram-setup@zram0).
#    Do NOT also install zram-tools. The two fight over /dev/zram0 and one of
#    them fails; purge it if you installed it before reading this.
swapon --show

# 2. Logs in RAM, with a cap so a chatty service cannot eat memory.
sudo apt-get install -y log2ram
sudo mkdir -p /etc/systemd/journald.conf.d
printf '[Journal]\nSystemMaxUse=200M\nRuntimeMaxUse=64M\n' |
  sudo tee /etc/systemd/journald.conf.d/00-cap.conf
sudo systemctl restart systemd-journald

# 3. /tmp in RAM.
sudo systemctl enable --now tmp.mount

# 4. No access-time writes, and fewer commits. Edit /etc/fstab and add these
#    two options to the root line, which is already "defaults":
#        defaults,noatime,commit=120
sudo mount -o remount /

The commit=120 trades up to two minutes of file writes on a power cut for far fewer commits. The filesystem stays consistent, and a WAL-mode SQLite database is unaffected, so a power loss costs you the last few seconds of a session rather than the disk. Remove that option if you move to an SSD.

Also enable the weekly TRIM that Debian ships, which extends the life of anything that supports it:

sudo systemctl enable --now fstrim.timer
sudo fstrim -v /

There is a price for putting logs in RAM: journalctl -b -1 will be empty after a reboot, because the journal never touched the disk. The agent writes its own logs to ~/.hermes/logs/, which do survive, so debug a bad boot from those.

3. Give the Agent Its Own User

The agent runs shell commands. Do not let it run them as your admin user.

sudo useradd -m -s /bin/bash hermes

That is deliberately all. The useradd default puts hermes in its own group and nowhere else, so it cannot sudo. That is the security boundary for everything below: a bad tool call can damage the agent’s own files, not the machine.

Now install the agent as that user:

sudo -u hermes -H bash -lc \
  'curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash'

It clones the repository, downloads its own Python runtime, and takes about twenty minutes on a Pi 4. Then configure it, still as hermes:

sudo -u hermes -i
hermes setup            # provider and API key
hermes setup gateway    # Telegram bot token and your user ID allowlist

For Telegram you need a bot token from @BotFather and your numeric user ID from @userinfobot. Put your own ID in the allowlist. Anything else that can message the bot can also ask it to run shell commands.

One thing to know before you test: the gateway ignores /start. It treats it as a platform ping. Send actual text.

4. Make the Gateway Survive a Reboot

This is the failure that cost me the most time, because it produces no error at all. The agent answers on Telegram, works perfectly, and then never comes back after a reboot.

The gateway is a systemd user service, and setting it up is two steps that look like one:

sudo -u hermes -i
export XDG_RUNTIME_DIR=/run/user/1001     # to reach the user bus

hermes gateway install                    # installs and starts it
systemctl --user enable hermes-gateway    # makes it start at boot
systemctl --user is-enabled hermes-gateway

install starts the service but does not enable it. There is a separate --start-on-login flag for that, and without it the unit works until the first reboot.

Linger has to be on too, or the user service never starts without a login:

sudo loginctl enable-linger hermes

I found this by rebooting on purpose. The journal showed the user manager starting and no gateway line at all, which is what a service that is not enabled looks like. Reading the config would have told me nothing.

Two more traps in this area:

5. Back Up What Cannot Be Re-Downloaded

The agent keeps everything in ~/.hermes. Most of it is software you could fetch again, and a little of it is the only copy of what the agent has learned.

My first archive was 1.06 GB. The actual state was about 4 MB.

1.6G   tools/          browser and driver binaries
938M   hermes-agent/   the git checkout
285M   cache/          web and package caches
235M   installs/       Python runtime

My first version excluded those by name. That is the wrong shape of rule for a program that downloads things while you sleep, and it was surprised twice. A later run pulled in 6,330 files under lsp/ and a 2,943 file stt/venv, and the archive went back up to 105 MB.

So the script below is a whitelist. Everything absent from STATE_ITEMS is excluded, including anything the agent downloads in future. Save it as /usr/local/bin/hermes-backup.sh:

#!/usr/bin/env bash
# Back up the agent's state, then push it off the machine.
# Databases go through sqlite .backup; everything else is a whitelist.
set -euo pipefail

SRC="/home/hermes/.hermes"
DEST="/home/hermes/.hermes-backups"
TARGET_FILE="/home/hermes/.hermes-backup-target"
KEEP=14
TS="$(date +%Y%m%d-%H%M%S)"

mkdir -p "$DEST"

# A tar of a live WAL database can capture a torn file. .backup takes a
# coherent copy while the agent keeps writing, and integrity_check proves it.
shopt -s nullglob
for db in "$SRC"/*.db; do
    name=$(basename "$db" .db)
    sqlite3 "$db" ".backup '$DEST/$name-$TS.db'" 2>/dev/null || {
        echo "sqlite backup failed: $db" >&2; exit 1; }
    check=$(sqlite3 "$DEST/$name-$TS.db" 'pragma integrity_check;' 2>/dev/null)
    [[ "$check" == "ok" ]] || { echo "integrity_check FAILED on $name-$TS.db" >&2; exit 1; }
done
shopt -u nullglob

# The whitelist. config.yaml, .env and auth.json are deliberately absent
# because they hold credentials, and `hermes setup` can rebuild them.
STATE_ITEMS=(
    memories skills cron hooks platforms pairing pending_messages source-checks
    kanban scripts plugins state runtime sessions terminal-sessions
    SOUL.md channel_directory.json .hermes_history install_id
)

# Only pass members that exist. GNU tar's --ignore-failed-read would do this,
# but it is not in BSD tar, so the script would silently write nothing on macOS.
PRESENT=()
for item in "${STATE_ITEMS[@]}"; do
    [ -e "$SRC/$item" ] && PRESENT+=("$item")
done
[ ${#PRESENT[@]} -gt 0 ] || { echo "nothing to back up under $SRC" >&2; exit 1; }

tar -czf "$DEST/memory-$TS.tar.gz" -C "$SRC" \
    --exclude='*.db' --exclude='*.db-*' --exclude='*.sock' "${PRESENT[@]}"

# Read the archive back. A corrupt tar that cannot be listed looks exactly
# like a valid one if you only check that the file exists.
tar -tzf "$DEST/memory-$TS.tar.gz" >/dev/null || { echo "archive is unreadable" >&2; exit 1; }

# Tripwire. The real payload is about a megabyte, so anything past 20 MB means
# the whitelist is wrong and the archive is growing again unnoticed.
# wc -c, not stat -c%s: the GNU stat flag does not exist on macOS, and the
# failure is silent, which is the opposite of what a tripwire is for.
size=$(wc -c < "$DEST/memory-$TS.tar.gz" 2>/dev/null || echo 0)
if [ "$size" -gt 20971520 ]; then
    echo "WARNING: memory tarball is $((size/1048576)) MB (>20 MB), check the whitelist" >&2
fi

# Keep the newest N of each, pruned per name so a large database cannot evict
# another's history. Unquoted globs on purpose.
prune() { ls -1t $1 2>/dev/null | tail -n +$((KEEP+1)) | xargs -r rm -f || true; }
prune "$DEST/memory-*.tar.gz"
for name in $(cd "$DEST" && ls -1 *.db 2>/dev/null |
              sed -E 's/-[0-9]{8}-[0-9]{6}\.db$//' | sort -u); do
    prune "$DEST/$name-*.db"
done

# A backup on the same flash as the original is not a backup.
if [ -s "$TARGET_FILE" ]; then
    target="$(cat "$TARGET_FILE")"
    rsync -a -e "ssh -o BatchMode=yes -o ConnectTimeout=15" "$DEST/" "$target" \
        || { echo "off-box rsync to $target failed (asleep? the next hour retries)" >&2; exit 1; }
else
    echo "no off-box target set in $TARGET_FILE" >&2
fi

echo "backup complete: $(ls -1 "$DEST" | wc -l) files in $DEST"

sudo install -m 755 -o root -g root hermes-backup.sh /usr/local/bin/
sudo -u hermes hermes-backup.sh     # run it once by hand
ls -la /home/hermes/.hermes-backups

You should see a tarball around 900 KB plus one file per database, rather than anything measured in gigabytes.

6. Back Up to a Machine That Sleeps

Point ~/.hermes-backup-target at somewhere else, and give the hermes user a key for it:

sudo -u hermes -i
ssh-keygen -t ed25519 -N ''
ssh-copy-id you@your-mac
exit

echo 'you@your-mac:/Users/you/hermes-backups/' |
  sudo tee /home/hermes/.hermes-backup-target

Then the schedule, which is where my first attempt was wrong. I ran it nightly at 03:30 and it failed every night with:

rsync error: unexplained error (code 255)

Code 255 is an SSH failure, and the reason was my MacBook being asleep at 03:32. A nightly push to a machine that is asleep every night is not a backup.

Two changes fix it. The BatchMode and ConnectTimeout options are already in the script above, so a sleeping target fails in seconds instead of hanging. And the timer runs hourly:

# /etc/systemd/system/hermes-backup.timer
[Unit]
Description=Hermes state backup

[Timer]
OnCalendar=hourly
Persistent=true
RandomizedDelaySec=300

[Install]
WantedBy=timers.target

# /etc/systemd/system/hermes-backup.service
[Unit]
Description=Hermes state backup

[Service]
Type=oneshot
User=hermes
Group=hermes
ExecStart=/usr/local/bin/hermes-backup.sh

sudo systemctl daemon-reload
sudo systemctl enable --now hermes-backup.timer

A missed run now costs one attempt that exits immediately, and the next hour picks it up. The copy is at most an hour out of date, and the unit’s exit status tells the truth either way.

Verify It Actually Works

Each of these checks something that fails silently rather than loudly.

# 1. The gateway starts on its own, with nobody logged in. The only real test.
sudo reboot
#    wait two minutes, then message the bot BEFORE logging in anywhere

# 2. Is it running, and who owns it?
ps -eo user,args | grep "[g]ateway run"

# 3. Will the backup fire, and did the last one succeed?
systemctl list-timers hermes-backup.timer
systemctl status hermes-backup.service

# 4. Is the copy actually off the machine? Run this on the backup target.
ls -lt ~/hermes-backups | head

If the bot answers after step 1 without you logging in, the machine is genuinely always on.

Using It

The build is finished once the bot answers. What follows is how to live with it.

There are three ways in, and you will use all of them:

The Command Line

ssh pi@keshu.local
sudo -u hermes -i

hermes                     # start chatting
hermes --continue          # resume the last session
hermes sessions list       # what you have talked about

Tell It Who You Are

It starts out knowing nothing about you, and its memory files do not exist yet. Fix that first, because every later conversation gets better for it:

Remember this: I'm <name>, a developer in <place>. My laptop is a Mac.
This Pi is my always-on agent, and I work mostly in <stack>.

Then start a new session and ask what it remembers. If the answer comes from memory rather than from the transcript, MEMORY.md and USER.md now exist under ~/.hermes/memories/. That is the part of this setup that is hardest to replace, so it is worth confirming early.

Give It Something That Repeats

Every morning at 8am, send me a short briefing: anything new in <topic>,
and the weather in <city>. Keep it under 15 lines.

The gateway runs a cron scheduler next to the messaging platforms, so the job runs whether or not you are awake, and the result lands in the same chat. Watch the first one arrive before you rely on it.

Commands Worth Knowing

hermes doctor              # when something feels wrong
hermes tools               # enable or disable capabilities
hermes gateway status -l   # is the gateway healthy
hermes gateway restart     # apply configuration changes
hermes update              # update; kills in-flight subagents, so do not cron it
journalctl _UID=1001 -f    # live gateway logs, run from the pi account

What It Can and Cannot Reach

This part surprises people. The agent’s terminal is the Pi, running as the hermes user. It can read and write files there and run commands locally, and it cannot see your laptop at all.

So “fix this bug in my project” does nothing until the code is somewhere it can reach. Two options: clone the repository onto the Pi, or point the terminal backend at another machine, which gives it hands there while its memory stays here.

hermes setup terminal          # local (the Pi), or ssh to another machine

Keep that boundary in mind, because it is the same boundary that stops a bad tool call from touching the rest of your machine. hermes is deliberately not in the sudo group.

Where Everything Lives

~/.hermes/memories/    what it has learned about you
~/.hermes/skills/      skills it has written for itself
~/.hermes/cron/        scheduled jobs
~/.hermes/state.db     conversations, and the index it searches them with
~/.hermes/logs/        its own logs, which survive a reboot

What Is Still Fragile

The pendrive will fail eventually. Hourly backups to a second machine exist because of that, and the restore path is the part I have not tested end to end. I have confirmed the archive is readable, that the databases pass integrity_check, and that the right files are inside. I have not yet unpacked one and watched the agent come back.

The other dependency is the model API. The Pi is always on; the thing doing the thinking is not on the Pi, and no amount of local tuning changes that.

What this buys is worth the trouble. The agent remembers across sessions, runs jobs while I am asleep, and does not need my laptop to be open. A machine that draws a few watts and sits in a corner is a better home for that than a terminal window I keep forgetting to reopen.

#ai-agents #raspberry-pi #linux #self-hosting
Reply to this post by email ↗