#MCP visibility and discovery

1 messages · Page 1 of 1 (latest)

round quiver
#

I'm running into some problem with MCP tool visibility and discovery. I was trying to teach Hermes what MCP search tools to use for my project, and Hermes seems to be confused about the MCP tools that are available. I have quite a few MCP servers added to the Hermes config.yaml, among them, MiniMax, which provides a web_search tool, and Z.ai's web-search-prime, which provides a web_search_prime tool. (1st screenshot)

For some reason, Hermes seems to have the wrong idea of what the tools are when it writes the SKILL.md file. I suspect the web_search tool from MiniMax is getting shadowed by Hermes' built-in one. I also don't understand where Hermes got the idea that mcp_web_search_prime has to do with Perplexica/Pro. (See the 2nd screenshot, which is what Hermes wrote to the SKILL.md file.)

Furthermore, if I specifically ask Hermes to give me a list of the MCP servers it sees, many are simply missing. (3rd screenshot)

image.png image.png image.png
calm moat
#

I think I'm having a similar problem with Minimax 2.7. I sent it the Hermes website to look at the documentation, and it tried to use the MCP to browse the site and failed. Switched to Codex and it used the tools correctly.

round quiver
calm moat
#

That's the strange part, I don't have any. But I did do a hermes update, and all of a sudden Minimax 2.7 decided to output the following instead of actually analyzing a webpage:

[TOOL_CALL]
{tool => "mcp_web_extract", args => {
--urls ["https://hermes-agent.nousresearch.com/docs/reference/tools-reference"]
}}
[/TOOL_CALL]

I'm not sure it's totally related to your issue, but maybe it's a Minimax 2.7 with Hermes issue since I switched to Codex and did the same thing, and it didn't have the problem.

round quiver
#

I switched to GLM-5 and it still makes no sense.

Meanwhile, hermes doctor shows me the MCP tools.

image.png image.png image.png
round quiver
#

Seriously, what even is going on???

image.png
#

I asked Hermes to show me her system prompt verbatim. Lo and behold, no MCP server or tool mentioned.

round quiver
#

Very broken.

image.png image.png
open horizon
#

can you possibly send your kogs for this? ~/.hermes/logs

round quiver
#

sure

round quiver
#

Wait..., I just saw it couldn't find uvx and npx, how is that even possible, given that hermes mcp test worked for me?

#

for example

image.png image.png
#

Even so, suppose that all the stdio MCP servers that depend on either npx or uvx to launch failed, what about the ones that use HTTP streamable transport, requiring only a URL? web-search-prime and exa are such MCP servers and even they didn't appear in the system prompt.

round quiver
#

If I try to start a hermes chat session in the terminal, instead of going through Discord, I can see the process tree in htop, and the MCP server processes are clearly there. However, when I ask Hermes in the chat, it still says it can't see the MCP tools.

image.png image.png
round quiver
#

I suspect that part of the problem has to do with the PATH env var when the Hermes is run as a gateway. I have Hermes gateway installed as a systemd service, using hermes gateway install. And then I do hermes gateway start. If I check the env vars of the gateway process by doing sudo cat /proc/<proc ID>/environ | tr '\0' '\n' | sort, I can see that the PATH where uvx could be found is missing. However, it still makes no sense that the gateway process could not find the npx, it's clearly still within the PATH of the gateway process.

image.png image.png
lapis parrot
#

same issue here as of today, running CLI not gateway. have a few MCP servers configured, mix of HTTP streamable and stdio. hermes shows them as connected on startup, no errors.

but when I check what the model actually has access to 30 built-in tools, zero from any MCP server. none of them made it through. tried calling them by name, same thing, they just don't exist in the tool list.

this might not be a PATH/systemd thing since I'm seeing it on plain CLI too, and my HTTP-only servers are missing as well so maybe not transport-specific either. looks like discovery runs but the tools never get injected into the LLM context somehow?

open horizon
#

hey guys, ive started working on the fix. i'll keep you guys updated

#

Hermes’ MCP discovery is working, but the toolset layer is stripping MCP tools back out when CLI and configured platforms resolve “enabled toolsets” down to only the configurable built-in groups

#

MCP tools are injected into hermes-* umbrella toolsets and per-server toolsets, not those reduced built-in groups, so they never survive into get_tool_definitions() for the live agent

#

the actual bug was downstream: platform toolset resolution was collapsing hermes-cli / hermes-discord into only the configurable built-in toolsets, which dropped MCP server toolsets before get_tool_definitions() built the final API tool list

open horizon
# round quiver

So the HTTP MCP tools were real and connected, but never made it into the model-facing tools= payload. This also explains the bad names/descriptions in your skill file: I found no Perplexica/Pro mapping in the repo, so that text was the model guessing, not a canonical Hermes mapping.

#

thanks for the logs

#

they helped alot

#

raising a PR for this fix in 5mins

open horizon
#

sorry forgot to attach the PR here

#

do you guys want to try the PR and see if that solves it for you?

open horizon
#

this is merged to main now, thanks @shut granite

round quiver
# open horizon this is merged to main now, thanks <@387972437901312000>

Thanks! It's working better now!

However, there is still the remaining problem where the Hermes gateway, installed via hermes gateway install as a systemd service, somehow does not inherit the complete PATH of the user, and so it is unable to find some executables that would be found in hermes chat - in my case, it is the uvx executable that cannot be found, because ~/.local/bin is not in the PATH of the systemd service.
The 3 screenshots are, respectively:

  1. output of hermes chat when I ask it "Show me all of the MCP tools that you see"
  2. output of hermes gateway (Discord), same query
  3. the error log
image.png image.png image.png
#

I think the problem is in the generate_systemd_unit function in hermes_cli/gateway.py.

#

A quick "fix" would be to include "~/.local/bin" by default, if not simply cloning the os.environ['PATH'].
This is given that even the Hermes install script would install uv to ~/.local/bin by default.
What do you think?

round quiver
#

I'm creating a PR.

open horizon
#

done? @round quiver

round quiver
#

Just need to test, actually.

open horizon
#

alright alright

#

lmk if you need help

round quiver
open horizon
#

can you test the solution?

round quiver
#

Writing a test case for it would not be straightforward, because it depends on the (in)existence of user local paths, which is out of the control of any test harness (unless the test suite is completely run in a throwaway Docker container).

open horizon
#

solved now?

round quiver
open horizon
#

ggs

shut granite
#

Sweet!!

round quiver