A web application for testing and comparing different Language Model implementations.
Website: https://snappy.raulcarini.dev
Github: https://github.com/R4ULtv/llms-speed-test
#🦀 Snappy - LLMs Speed Test
digital herald
**Current Features: **
- Test a local LLM with 4 different prompts
- Only run models from Ollama
- Download results metrics in CSV, PNG
Future Features:
Custom URL for OllamaChange the difficulty of the testText StreamingTests History- Add AI SDK Support (for not self hosted model)
- Add LM Studio Support
- Multi model test (you can compere multiple model)
digital herald
Ollama allows cross-origin requests from 127.0.0.1 and 0.0.0.0 by default.
You need to set OLLAMA_ORIGINS=* so the website can access your local models.
Here's how you can set the env variables based on your system:
https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server
digital herald
digital herald
I just added the ability to change the host and the test difficulty in the settings.
digital herald
Added a small indicator at the top of the page where you can see which model is running and how much GPU is being used.
digital herald
Added Streaming Mode
cunning isle
Interesting
digital herald
Thanks