sacred elbow
Is it by design that when using local models (via llama.cpp) that token usage is not capture? running some curl to the endpoints, I see usage information returned.
claw@clawserv:~$ curl http://devbox:8080/v1/chat/completions -d '{"model":"qwen36-35b-a3b","messages":[{"role":"user","content":"hi"}],"stream":false}' | python3 -m json.tool | grep -A5 usage
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1697 100 1612 100 85 233 12 0:00:07 0:00:06 0:00:01 415
"usage": {
"completion_tokens": 254,
"prompt_tokens": 11,
"total_tokens": 265,
"prompt_tokens_details": {
"cached_tokens": 0
claw@clawserv:~$ curl http://llmserver:8080/v1/chat/completions -d '{"model":"qwen35-9b","messages":[{"role":"user","con
tent":"hi"}],"stream":false}' | python3 -m json.tool | grep -A5 usage
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1257 100 1177 100 80 521 35 0:00:02 0:00:02 --:--:-- 557
"usage": {
"completion_tokens": 119,
"prompt_tokens": 11,
"total_tokens": 130,
"prompt_tokens_details": {
"cached_tokens": 0