Ralph Wiggum loop via crons feasibility

[tranquil urchin]

I have an autonomous code improvement system running on a VPS. I want you to evaluate whether this architecture is sound or if there are risks I'm missing.

Setup:

• Two cron jobs running 24/7 via OpenClaw (an AI agent orchestration platform)
• Both run as isolated sessions with free LLM models (Groq llama-3.3-70b, Cerebras gpt-oss-120b)
• Both deliver results to Telegram
Cron 1 — "Reviewer" (every 6 hours):

• Reads one Python source file from a list of 12 trading bot monitors
• Hunts for real bugs: crashes, unhandled exceptions, logic errors, values contradicting a config file (DECISIONS.json)
• Adds found bugs to a BACKLOG.json file with severity ratings
• Never modifies source code — read-only
• Rotates through files sequentially, loops back when all reviewed
Cron 2 — "Fixer" (every 2 hours):

• Picks the first unfixed bug from BACKLOG.json
• Uses sed/python one-liners to make surgical edits (never rewrites full files)
• Safety: backs up file before edit, checks line count after (must be within ±5), verifies Python import passes, auto-reverts from backup if any check fails
• Updates backlog and logs results
Rate limit protection:

• 3-model fallback chain (minimax → cerebras → groq) with exponential backoff cooldowns
• Jobs spaced well below API rate limits (16 requests/day vs 1000+/day allowed)
File safety:

• Git committed before loop started — full rollback available
• Backup-before-edit + auto-revert on failure
• No parallel agents editing the same file
My concerns:

1. Is the fixer prompt robust enough to prevent bad edits even with auto-revert?
2. Could the reviewer generate false positive bugs that the fixer then "fixes" incorrectly?
3. Am I missing any failure mode?
4. What would you change?

[wanton sorrel]

Your separation into Reviewer (read-only) → Fixer (writes) with rollback is the right shape, and the “no parallel editors” + “git committed before loop” are solid baseline controls.

The big gap is: your current safety checks mostly catch syntax/packaging failures, not semantic failures. For a trading system, semantic failures are the expensive ones.

1) “Is the fixer prompt robust enough?”

Prompts help, but don’t rely on them for safety. Assume the model will occasionally:

• misread intent,
• “fix” by changing behavior,
• issue a dangerous shell command,
• or apply a correct fix to the wrong location.

Your technical guardrails matter more than the prompt. Right now, line-count ±5 + “imports pass” are weak proxies for correctness.

Upgrades that actually harden this:

• Patch-only workflow: fixer generates a unified diff; you apply it with git apply (or a minimal patch applier), not raw sed strings.
• Allowlist edits: explicitly restrict what transformations are permitted (e.g., add missing try/except around one call; add None guard; fix incorrect constant). Everything else becomes “proposal-only”.
• Semantic checks: run at least one of:

• unit tests / integration smoke tests (even a small harness),
  • static analysis (ruff, pyright/mypy),
  • python -m compileall (better than import),
  • a deterministic “monitor dry-run” with sample inputs.
• AST-aware edits instead of regex: use libcst/bowler (or even a tiny Python script) to change exactly one node. Regex/sed is how you get “works, but subtly different”.

2) False positives from reviewer → incorrect “fixes”

Yes, this is a top risk. Two failure patterns:

• Hallucinated bug → fixer changes logic to match the hallucination.
• Ambiguous “smell” (style preference) gets treated as a defect.

How to reduce:

• Require every backlog item to include:
  • file, line_start/line_end,
  • code snippet + hash of snippet, and
  • commit hash of the code reviewed.
• Fixer must verify the snippet hash still matches before editing; otherwise mark as stale and send back to reviewer.

• Add a third step (can be cheap): Validator (could be the reviewer model, or a smaller one) that only sees the diff and answers: “Does this patch actually fix the stated bug? What could it break?”

Also consider policy: Reviewer must provide a minimal reproduction or a falsifiable claim (e.g., “this will throw KeyError if config lacks X”). If it can’t be falsified, don’t auto-fix it.

3) Missing failure modes to think about

The ones I’d worry about most:

State / concurrency

• Cron overlap: if a run takes >2h/6h, you can get overlapping runs unless you use a lock.
• Partial writes: reviewer could read while fixer is mid-write unless you do atomic write (temp + rename).
• BACKLOG.json corruption (partial write, truncation). Use atomic writes + validate JSON schema before accepting.

Backlog poisoning / repeated churn

• One unfixable bug at top → fixer retries forever. Add attempt_count, last_attempt, and an “auto-quarantine after N failures” state.

Command injection / unsafe shell

• Even well-intentioned models sometimes emit ;, pipes, redirections, subshells, etc.

• Run the fixer under a least-privilege user, with no secrets in env, ideally no network egress, and a hard allowlist of executable commands (or do edits purely in Python without shell).

Semantic regression with “passing checks”

• Imports pass but behavior changes (off-by-one, wrong threshold, inverted conditional).
• For trading/monitoring code, add at least one cheap runtime check that exercises key paths.

Config drift

• DECISIONS.json might change; reviewer flags “contradiction”, fixer “fixes” code, but decision file was wrong/outdated. Track decision version + require human confirmation for “policy contradictions”.

4) What I would change (high leverage)

If you only do a few things, I’d do these:

1. Do all fixes in a clean git worktree/branch, never in-place. Only fast-forward/merge back when checks pass.
2. Replace “sed one-liners” with diff generation + patch apply, and store the diff in the backlog item.
3. Add file locks + atomic writes for both source files and BACKLOG.json.
4. Add a validator gate on every patch (separate model or separate run) + require snippet/commit match before applying.
5. Restrict auto-fixing to a low-risk class of bugs; everything else becomes “propose + notify”.