Backups and Disaster Recovery

Backups and Disaster Recovery

Most of Lobster’s state lives in ~/.openclaw/ on a single Mac. That directory holds everything that isn’t human-authored configuration — session transcripts, compacted context summaries, vector memory embeddings, OAuth tokens, cron run history. Losing the Mac means losing all of it.

This guide covers the two-layer backup strategy Lobster uses today, the decision rationale behind it, and how to restore onto a fresh machine.

The two backup layers

Layer	What	Where	Cadence	Size
Slim (recovery)	Irreplaceable state only	iCloud Drive, one known-good copy	Daily at 3 AM PT via cron	~47 MB
Full (convenience)	Entire ~/.openclaw/	Local ~/lobster-backups/	Manual via openclaw-backup	~1.7 GB

They serve different purposes. The full backup makes a same-machine restore fast — if you corrupt a file and want to roll back an hour later, tar -xzf from the local tarball beats re-downloading Chromium and reinstalling plugins. The slim backup is the real disaster-recovery layer — it ships off-machine so a drive failure, fire, or lost Mac doesn’t take your memory with it.

What’s in the slim backup

Only the things that can’t be rebuilt by re-running setup.sh or reinstalling plugins:

agents/          sessions, auth profiles, workspaces (all 13 agents)
lcm.db / -wal    compacted long-conversation context
memory/          vector embeddings (semantic memory index)
credentials/     OAuth tokens, encrypted at rest
tasks/           task ledger
openclaw.json    main config
cron/            cron definitions + run history
secrets.json     SecretRef values, encrypted at rest
exec-approvals.json

Deliberately excluded:

Excluded	Reason
browser/ (1.7 GB)	Headless Chromium — openclaw browser start re-downloads
extensions/ (1.5 GB)	Plugin installs — reinstallable via openclaw plugins install
logs/ (300+ MB)	Forensic only; regenerates on every gateway run
media/ (230 MB)	Message attachments — text context survives without them

Cutting these four takes the backup from 1.7 GB → 47 MB, which matters because we overwrite a single file in iCloud rather than keeping dated archives.

Why one known-good copy, not a timeline

iCloud Drive is sized in single-digit GB on the free tier, and recovery doesn’t need history — it needs the most recent correct snapshot. The slim backup script writes to openclaw-slim-latest.tar.gz, overwriting yesterday’s copy each day.

The script is safe against interruption:

# Pseudo-code from openclaw-backup-slim
tar -czf "$ICLOUD/.openclaw-slim-latest.tmp.tar.gz" ...   # write to temp
SIZE=$(stat ... "$TMP")
if (( SIZE < 10_MB )); exit 1                              # sanity check
mv "$TMP" "$ICLOUD/openclaw-slim-latest.tar.gz"            # atomic rename

If the run dies mid-tar (e.g. iCloud hiccup, disk full, kernel panic), the temp file never replaces the last known-good copy. You lose a day of backup, not a machine’s worth of memory.

The known gap

If the state corrupts mid-day, the next 3 AM backup faithfully overwrites the known-good copy with the corrupted state. The iCloud slim backup is a snapshot of “now,” not a rollback point.

Mitigations if this matters to you:

• Time Machine on an external SSD — Apple’s built-in, keeps hourly/daily/weekly snapshots. Covers the rollback-to-yesterday case the iCloud slim backup can’t.
• Multiple iCloud files — extend the script to rotate openclaw-slim-{mon,tue,...}.tar.gz. Uses ~7× the iCloud budget but covers the most recent week.

Today’s setup deliberately trades rollback coverage for simplicity. Most disaster-recovery scenarios (disk failure, fire, theft, Mac loss) are fully covered. State-corruption rollback is a different problem we haven’t solved.

Restoring onto a fresh Mac

# 1. Install OpenClaw
npm install -g openclaw

# 2. Clone the Lobster repo (configuration-as-code lives here)
git clone https://github.com/omarshahine/Lobster ~/GitHub/lobster

# 3. Run full setup (deps, skill deployment, MCP servers, Messages keepalive)
bash ~/GitHub/lobster/setup/setup.sh

# 4. Restore state from the iCloud slim backup
#    (download openclaw-slim-latest.tar.gz via another iCloud-signed-in device)
mkdir -p ~/.openclaw
tar -xzf openclaw-slim-latest.tar.gz -C ~/.openclaw

# 5. Re-authenticate model providers (OAuth tokens don't survive machine moves)
openclaw models auth login --provider anthropic
openclaw models auth login --provider openai-codex
# ... any others you use

# 6. Re-pair channels
#    BlueBubbles: install app, re-pair to Messages.app, set webhook URL
#    WhatsApp: wacli login (scan QR), re-pair via Baileys

# 7. Re-approve paired devices (iPhone, dashboard, etc)
openclaw devices list              # shows pending requests
openclaw devices approve <id>

# 8. Start gateway
openclaw gateway start

Total time: under an hour once OAuth flows complete.

What’s not in the backup and where it lives

Thing	Recovery
Obsidian vault	Obsidian Sync (Obsidian’s own paid service) — separate backup pipeline
OpenClaw source checkouts (~/GitHub/*)	Git clone from GitHub
Repo secrets (~/.openclaw/secrets.json before OAuth)	Restored by the slim backup
Apple Mail archive	iCloud Mail serves it; local mail store regenerates
Messages.app database	Time Machine on external SSD, or iCloud Messages sync
BlueBubbles server config	Reinstall + re-pair — not snapshot-restorable
Tailscale device registration	Re-auth via admin console; old device can be revoked

Operational notes

Check the last backup:

ls -la "$HOME/Library/Mobile Documents/com~apple~CloudDocs/lobster-backups/"
# openclaw-slim-latest.tar.gz   (inspect the mtime)

Force a manual slim backup:

~/.local/bin/openclaw-backup-slim

Verify the archive is restorable without doing a full restore:

tar -tzf "$HOME/Library/Mobile Documents/com~apple~CloudDocs/lobster-backups/openclaw-slim-latest.tar.gz" | head -20

Cron job state:

openclaw cron list | grep -i "slim backup"
openclaw cron runs --id e471e2b2-e879-4bc9-8562-0808a6520659

The cron fires daily at 03:00 America/Los_Angeles, reports NO_REPLY on success, and iMessages Omar only when the script exits non-zero.

Test schedule

Untested backups are Schrödinger’s backups — both working and broken until you open the box.

Every quarter:

1. Check iCloud backup mtime is within the last 24 hours.
2. Extract to a scratch directory and confirm layout:

   mkdir -p /tmp/backup-test
   tar -xzf "$HOME/Library/Mobile Documents/com~apple~CloudDocs/lobster-backups/openclaw-slim-latest.tar.gz" -C /tmp/backup-test
   ls /tmp/backup-test  # expect: agents/ lcm.db memory/ ...
   rm -rf /tmp/backup-test

3. If anything is missing or the mtime is stale, investigate the cron run log.

Related

• scripts/local-bin/openclaw-backup-slim — the backup script itself
• scripts/local-bin/openclaw-backup — full local backup script
• config/cron-jobs.json — [Infra] Daily Slim Backup job definition
• Secrets Management — what’s inside secrets.json