neon fulcrum
Urgent help needed: the o1preview api seems to stop generating content whenever there is a relatively long prompt (more than 1k token) being used.
Yet, it still charges for the upstream cost whilst does not generating / generating partially a response.
The error code is ‘null’ ‘network error’.
All these billings are for partially generated prompt with the ‘network error’ return code.
When I switched to Claude 3.5 with more context window, no issue.
But my prompt did not exceed the 128k o1 limit.
Could anybody from the team explain why ? Thanks
