lobster workflow failure

[copper plank]

I have a lobster workflow failing , can you help me debug it, how do I run this manually

name: sql_lead_drafts_email_output

steps:

• id: read_leads
  command: cat /home/admin/.openclaw/workspace-dba-scout/sql_leads.md

• id: build_llm_request
  command: node /home/admin/.openclaw/workspace-dba-scout/lobster-scripts/build-llm-request.js
  stdin: $read_leads.stdout

• id: debug_llm_request
  command: tee /home/admin/.openclaw/workspace-dba-scout/debug-llm-request.json
  stdin: $build_llm_request.stdout

• id: draft_email
  command: sh -c 'node /home/admin/.openclaw/workspace-dba-scout/lobster-scripts/invoke-llm-task.js 2> /home/admin/.openclaw/workspace-dba-scout/debug-draft-email-stderr.txt'
  stdin: $debug_llm_request.stdout

• id: debug_draft_email
  command: tee /home/admin/.openclaw/workspace-dba-scout/debug-draft-email.json
  stdin: $draft_email.stdout

• id: extract_llm_json
  command: node /home/admin/.openclaw/workspace-dba-scout/lobster-scripts/extract-llm-json.js
  stdin: $debug_draft_email.stdout

• id: debug_extracted_json
  command: tee /home/admin/.openclaw/workspace-dba-scout/debug-extracted-json.json
  stdin: $extract_llm_json.stdout

• id: render_email_html
  command: node /home/admin/.openclaw/workspace-dba-scout/lobster-scripts/render-email-html.js
  stdin: $debug_extracted_json.stdout

• id: debug_rendered_email
  command: tee /home/admin/.openclaw/workspace-dba-scout/debug-rendered-email.json
  stdin: $render_email_html.stdout

• id: write_email_html_file
  command: node /home/admin/.openclaw/workspace-dba-scout/lobster-scripts/write-email-html-file.js
  stdin: $debug_rendered_email.stdout

• id: final_output
  command: cat /home/admin/.openclaw/workspace-dba-scout/sql-leads-draft-output.html

[lilac harness]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[minor sonnet]

First grab the version too, since Lobster behavior has moved a bit between builds:

openclaw --version

To run that workflow manually, make the implicit stdin chain explicit. This is the same pipeline, but with debug files at each boundary:

set -euo pipefail

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

cat "$WORK/sql_leads.md" \
  | node "$WORK/lobster-scripts/build-llm-request.js" \

[minor sonnet]

  | tee "$WORK/debug-llm-request.json" \
  | sh -c 'node /home/admin/.openclaw/workspace-dba-scout/lobster-scripts/invoke-llm-task.js 2> /home/admin/.openclaw/workspace-dba-scout/debug-draft-email-stderr.txt' \
  | tee "$WORK/debug-draft-email.json" \
  | node "$WORK/lobster-scripts/extract-llm-json.js" \
  | tee "$WORK/debug-extracted-json.json" \
  | node "$WORK/lobster-scripts/render-email-html.js" \
  | tee "$WORK/debug-rendered-email.json" \
  | node "$WORK/lobster-scripts/write-email-html-file.js"

cat "$WORK/sql-leads-draft-output.html"

If it fails, rerun step-by-step so you can see exactly which file becomes invalid:

[minor sonnet]

set -euo pipefail
WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

cat sql_leads.md > /tmp/read_leads.out
node lobster-scripts/build-llm-request.js < /tmp/read_leads.out > debug-llm-request.json
node lobster-scripts/invoke-llm-task.js < debug-llm-request.json > debug-draft-email.json 2> debug-draft-email-stderr.txt
node lobster-scripts/extract-llm-json.js < debug-draft-email.json > debug-extracted-json.json
node lobster-scripts/render-email-html.js < debug-extracted-json.json > debug-rendered-email.json
node lobster-scripts/write-email-html-file.js < debug-rendered-email.json
cat sql-leads-draft-output.html

Then check the JSON handoffs:

[minor sonnet]

jq . debug-llm-request.json >/dev/null
jq . debug-draft-email.json >/dev/null
jq . debug-extracted-json.json >/dev/null
jq . debug-rendered-email.json >/dev/null
cat debug-draft-email-stderr.txt

Most likely failure points here are:

• invoke-llm-task.js is writing an auth/model/tool error to stderr
• the LLM response is not clean JSON, so extract-llm-json.js fails
• the rendered JSON shape does not match what write-email-html-file.js expects

Docs for the workflow runner are here: https://docs.openclaw.ai/tools/lobster. If you paste the failing command output plus openclaw --version, I can help pin down the exact step.

[copper plank]

I think invoke-llm-task.js is failing, how do I run it up to that point

[minor sonnet]

Run only the steps before invoke-llm-task.js, then feed that saved request into it directly.

set -euo pipefail

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

# 1. Recreate the input that invoke-llm-task.js receives
cat "$WORK/sql_leads.md" \
  | node "$WORK/lobster-scripts/build-llm-request.js" \
  | tee "$WORK/debug-llm-request.json" >/dev/null

# 2. Sanity-check the generated request
jq . "$WORK/debug-llm-request.json" >/dev/null
head -80 "$WORK/debug-llm-request.json"

[minor sonnet]

# 3. Run just the suspected failing step
node "$WORK/lobster-scripts/invoke-llm-task.js" \
  < "$WORK/debug-llm-request.json" \
  > "$WORK/debug-draft-email.json" \
  2> "$WORK/debug-draft-email-stderr.txt"

# 4. Inspect what happened
echo "STDERR:"
cat "$WORK/debug-draft-email-stderr.txt"

echo "STDOUT JSON check:"
jq . "$WORK/debug-draft-email.json"

If the command exits before the inspect section, rerun step 3 without set -e so you can capture the exit code:

[minor sonnet]

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

node "$WORK/lobster-scripts/invoke-llm-task.js" \
  < "$WORK/debug-llm-request.json" \
  > "$WORK/debug-draft-email.json" \
  2> "$WORK/debug-draft-email-stderr.txt"

echo "exit=$?"
cat "$WORK/debug-draft-email-stderr.txt"
head -120 "$WORK/debug-draft-email.json"

Also please include openclaw --version with the output. That helps avoid chasing a version-specific Lobster/plugin issue.

[copper plank]

I get this

exit=1
Gateway HTTP request failed
URL: http://127.0.0.1:18789/tools/invoke
Status: 401 Unauthorized
Raw response:
{"error":{"message":"Unauthorized","type":"unauthorized"}}

[minor sonnet]

That means the script is reaching the Gateway, but the HTTP call to /tools/invoke is missing the Gateway auth secret. So invoke-llm-task.js is probably not failing on the LLM yet; it is failing before the tool call is authorized.

First check auth mode and version:

openclaw --version
openclaw config get gateway.auth.mode
openclaw gateway status

If you are using token auth, run it like this:

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

[minor sonnet]

export OPENCLAW_GATEWAY_TOKEN="$(openclaw config get gateway.auth.token)"

node "$WORK/lobster-scripts/invoke-llm-task.js" \
  < "$WORK/debug-llm-request.json" \
  > "$WORK/debug-draft-email.json" \
  2> "$WORK/debug-draft-email-stderr.txt"

echo "exit=$?"
cat "$WORK/debug-draft-email-stderr.txt"
head -120 "$WORK/debug-draft-email.json"

If openclaw config get gateway.auth.token prints something like a SecretRef instead of the actual token, you need to export the resolved token from wherever you stored it:

[minor sonnet]

export OPENCLAW_GATEWAY_TOKEN="actual-token-here"

Don’t paste the token here.

If your gateway is password auth instead, the script may need to support OPENCLAW_GATEWAY_PASSWORD or an Authorization: Bearer <password> header. Quick way to confirm the gateway secret works:

TOKEN="$(openclaw config get gateway.auth.token)"
curl -sS -H "Authorization: Bearer $TOKEN" http://127.0.0.1:18789/v1/models | head

For the underlying rule: HTTP APIs like /tools/invoke require shared-secret auth by default. Docs: https://docs.openclaw.ai/gateway and https://docs.openclaw.ai/web/dashboard#if-you-see-unauthorized-1008.

[copper plank]

I fixed the token issue, I see this in the error logs

08:46:12
warn
agent/embedded
embedded run timeout: runId=llm-task-1778830990430 sessionId=llm-task-1778830990430 timeoutMs=180000
08:46:14
warn
agent/embedded
embedded abort settle timed out: runId=llm-task-1778830990430 sessionId=llm-task-1778830990430 timeoutMs=2000
08:46:14
warn
agent/embedded
embedded run failover decision
08:46:14
error
diagnostic
lane task error: lane=main durationMs=184300 error="FailoverError: LLM request timed out."
08:46:14
error
diagnostic
lane task error: lane=session:agent:main:explicit:llm-task-1778830990430 durationMs=184315 error="FailoverError: LLM request timed out."
08:46:14
warn
tools-invoke
tool execution failed: FailoverError: LLM request timed out.

[minor sonnet]

Good, that moves the failure: auth is fixed, and now llm-task is timing out after 180s.

I’d check two things: whether llm-task works at all with a tiny request, and whether your generated request is too large/slow.

Tiny health test:

TOKEN="$(openclaw config get gateway.auth.token)"

curl -sS http://127.0.0.1:18789/tools/invoke \
  -H "Authorization: Bearer $TOKEN" \
  -H "content-type: application/json" \
  --data '{
    "tool": "llm-task",
    "args": {
      "action": "json",

[minor sonnet]

      "prompt": "Return JSON only: {"ok":true}",
      "schema": {
        "type": "object",
        "properties": { "ok": { "type": "boolean" } },
        "required": ["ok"],
        "additionalProperties": false
      },
      "timeoutMs": 30000,
      "maxTokens": 50,
      "thinking": "low"
    }
  }' | jq .

Then inspect the real request size and shape:

[minor sonnet]

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

wc -c debug-llm-request.json
jq 'keys' debug-llm-request.json
jq '{promptChars:(.prompt|length), inputChars:(.input|tostring|length), maxTokens, timeoutMs, thinking, model, provider}' debug-llm-request.json

If the tiny test passes but your workflow times out, the fix is in build-llm-request.js / the request payload:

• reduce the lead input size for a test run
• set thinking: "low"
• set a smaller maxTokens
• add/increase timeoutMs in the llm-task args only if the job is legitimately long
• make sure it is using a reliable cloud model/provider; for current model recommendations, use #models

If the tiny test also times out, it’s probably provider/auth/model routing rather than your workflow. Run:

[minor sonnet]

openclaw models status
openclaw logs --follow

The relevant docs are https://docs.openclaw.ai/tools/llm-task and https://docs.openclaw.ai/tools/lobster.

[copper plank]

It was definiately timeout as I was watching lmstudio

[minor sonnet]

Yep, then the pipeline is working mechanically. The bottleneck is the LM Studio model not finishing before llm-task’s timeout.

For a quick confirm, bump the request timeout and keep output small:

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

jq '.timeoutMs = 600000 | .thinking = "low" | .maxTokens = 800' \
  debug-llm-request.json > /tmp/debug-llm-request-long-timeout.json

node "$WORK/lobster-scripts/invoke-llm-task.js" \
  < /tmp/debug-llm-request-long-timeout.json \
  > "$WORK/debug-draft-email.json" \
  2> "$WORK/debug-draft-email-stderr.txt"

[minor sonnet]

echo "exit=$?"
cat "$WORK/debug-draft-email-stderr.txt"
jq . "$WORK/debug-draft-email.json"

If that works, make the same changes in lobster-scripts/build-llm-request.js: add/increase timeoutMs, lower maxTokens, and use thinking: "low".

If it still times out on LM Studio, the practical fixes are:

• test with fewer leads first, then batch/chunk the workflow
• make the prompt/schema stricter and shorter
• reduce requested output size
• switch this step to a faster/cloud model for the drafting task

Local LM Studio/Ollama-style models can technically work here, but structured workflow steps are rough if generation is slow. The timeout log says OpenClaw waited 180s and aborted cleanly; it’s not a Lobster YAML issue anymore.

[copper plank]

how do I increase the timeout beyond 180 seconds

[minor sonnet]

There are two timeouts to care about:

1. llm-task timeout: the actual model run timeout.
2. Lobster/tool-call timeout: the outer workflow call timeout, if you are invoking the whole workflow through the lobster tool.

For your manual run, increase the llm-task timeout by putting timeoutMs in the JSON request that invoke-llm-task.js reads:

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

jq '.timeoutMs = 600000' debug-llm-request.json > debug-llm-request-10min.json

node "$WORK/lobster-scripts/invoke-llm-task.js" \
  < "$WORK/debug-llm-request-10min.json" \
  > "$WORK/debug-draft-email.json" \

[minor sonnet]

  2> "$WORK/debug-draft-email-stderr.txt"

600000 is 10 minutes. Use 300000 for 5 minutes, etc.

For the permanent fix, update /home/admin/.openclaw/workspace-dba-scout/lobster-scripts/build-llm-request.js so the object it outputs includes something like:

timeoutMs: 600000,
thinking: "low",
maxTokens: 800,

Or set a plugin default in OpenClaw config:

[minor sonnet]

{
  "plugins": {
    "entries": {
      "llm-task": {
        "enabled": true,
        "config": {
          "timeoutMs": 600000
        }
      }
    }
  }
}

Then restart the gateway after config changes.

[minor sonnet]

If you run the whole Lobster workflow through the lobster tool, also make sure the outer call timeout is longer than the LLM timeout, e.g. timeoutMs: 700000. Otherwise Lobster can kill the workflow before llm-task finishes.

Docs: https://docs.openclaw.ai/tools/llm-task and https://docs.openclaw.ai/tools/lobster.

[copper plank]

I think I aleadt have a timeout set in openclaw, what is the command to check

[minor sonnet]

Check the specific llm-task timeout first:

openclaw config get plugins.entries.llm-task.config.timeoutMs

Also check the broader agent timeout, because your log shows the embedded agent run timing out:

openclaw config get agents.defaults.timeoutSeconds

And if this is set per-agent:

openclaw config get agents.list

[minor sonnet]

For your case, also inspect the request itself, because debug-llm-request.json can override the config default:

jq '{timeoutMs, maxTokens, thinking, provider, model}' /home/admin/.openclaw/workspace-dba-scout/debug-llm-request.json

Precedence for llm-task is: request timeoutMs first, then plugins.entries.llm-task.config.timeoutMs, then the built-in default. If the log says timeoutMs=180000, something is setting 180 seconds either in the request or config.

[copper plank]

I get this

{
"timeoutMs": 180000,
"maxTokens": 8000,
"thinking": "low",
"provider": null,
"model": null
}

[minor sonnet]

That confirms it: the request itself is overriding the timeout to 180s.

For a one-off rerun, change the generated request file:

WORK=/home/admin/.openclaw/workspace-dba-scout
cd "$WORK"

jq '.timeoutMs = 600000 | .maxTokens = 1500' \
  debug-llm-request.json > debug-llm-request-10min.json

node "$WORK/lobster-scripts/invoke-llm-task.js" \
  < "$WORK/debug-llm-request-10min.json" \
  > "$WORK/debug-draft-email.json" \
  2> "$WORK/debug-draft-email-stderr.txt"

[minor sonnet]

For the real fix, edit this file:

/home/admin/.openclaw/workspace-dba-scout/lobster-scripts/build-llm-request.js

Find where it outputs timeoutMs: 180000 and change it to e.g.:

timeoutMs: 600000,
maxTokens: 1500,
thinking: "low",

I’d lower maxTokens while debugging. 8000 tokens on LM Studio can easily turn into a very long local generation, especially for structured JSON.

After changing build-llm-request.js, regenerate the request and confirm:

[minor sonnet]

cat "$WORK/sql_leads.md" \
  | node "$WORK/lobster-scripts/build-llm-request.js" \
  > "$WORK/debug-llm-request.json"

jq '{timeoutMs, maxTokens, thinking, provider, model}' "$WORK/debug-llm-request.json"

provider: null / model: null just means llm-task is using your OpenClaw default model/provider config.

[copper plank]

I found a timeout value in the value and changed it, it's now working

[minor sonnet]

Nice, that tracks with the timeoutMs: 180000 we saw in the generated request. So the workflow path was fine: token auth first, then the LLM step just needed a longer timeout for LM Studio.

If this fixed it, please right-click the solution message → Apps → ✅ Mark Solution.
That helps Answer Overflow index the fix for future users.