how to log web search sources

[rancid oasis]

The docs ( https://openrouter.ai/announcements/introducing-web-search-via-the-api ) say that the web search plugin adds the explanation of the search results and the actual search results to the system message.

This happens on the OpenRouter server side, where it passes info to exa and then adds it before calling whatever model you're using.

Is there a flag or something that makes it possible to log the exact message that was added? I'm trying to debug a web search prompt, but without seeing what it's actually searching for it's difficult.

When I copy my original prompt into Exa and see the query / responses, they provide info that correctly answers the original prompt, so something wacky is going on.

You can "log" the result urls by trying to have it regurgitate it from the prompt directly "Append all of the web search context at the end of the response." but that doesn't have the queries used.

[small birch]

We don't currently have a way of exposing the query, AFAIK we just use exa's auto function for search

We know this is definitely not optimal and intend to make improvements in the future

[rancid oasis]

@small birch is it public which options you use for Exa Search? I'm like 90% sure after experimenting that you have the AI page summary turned on

[small birch]

it's not published but it's not sensitive info, let me see if I can find it

[small birch]

it's something like this:

      query,
      type: 'auto',
      useAutoprompt: true,
      numResults: this.maxResults,
      contents: {
        highlights: {
          numSentences: 5,
        },
      },```
and the search prompt is
```  A web search was conducted on ${date.toISOString()}. Incorporate the following web search results into your response.

  IMPORTANT: Cite them using markdown links named using the domain of the source.
  Example: [nytimes.com](https://nytimes.com/some-page).```

[rancid oasis]

Thanks! That's super useful, I'll just run my queries directly through Exa with this to debug

Long term it'd be cool if you could request from OpenRouter the system prompt for anything using web search (or any other future OpenRouter server-side plugins)

[small birch]

yeah I think the goal is also just to make it all customizable too

like you can pass your own sys prompt etc