Model Information:
Model: CohereLabs/c4ai-command-r-plus
Parameters: 104 billion parameters
Model Size: 104B parameters (approximately 194GB in FP16)
Context Length: 128K tokens
Architecture: Auto-regressive transformer
License: CC-BY-NC-4.0
Current VPS Specifications:
RAM: 31GB
GPU: NVIDIA RTX A4000 (16GB VRAM)
Storage: 319GB SSD
OS: Linux
Issue Description:
We are trying to run the CohereLabs/c4ai-command-r-plus model locally but encountering memory limitations. The model requires approximately 194GB of memory, but our current VPS has only 31GB RAM and 16GB GPU VRAM, which is insufficient.
Technical Requirements:
Based on the model specifications from the Hugging Face page:
Model Size: 104B parameters = ~194GB in FP16
Memory Requirements: 200GB+ total memory needed
GPU Requirements: 32GB+ VRAM recommended for optimal performance
Storage: 500GB+ for model files and swap space
Questions:
What is the minimum VPS configuration that can run this model?
What is the recommended VPS configuration for optimal performance?
Are there any VPS providers that offer configurations suitable for 200GB+ memory?
What are the cost estimates for such configurations?
Are there any alternatives like model quantization or cloud solutions?
Thank you
#Please help - VPS Hardware Requirements for Running CohereLabs/c4ai-command-r-plus (104B Parameters)
gaunt stag
tribal sierra