Help deploying ChromaDB in production?

[deft pelican]

Hi everyone. I am new to this world. I'm an ML researcher, not a software engineer.

I'm trying to figure out how to deploy ChromaDB on Railway and then use it interactively from somewhere else. But no matter what I try with TCP networking, I get the following error:

ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

Code I'm using to connect on client side:

import chromadb
client = chromadb.HttpClient(
    host='HOST_NAME_PROVIDED_BY_RAILWAY',
    port='PORT_PROVIDED_BY_RAILWAY'
)
client.heartbeat()

I also don't really understand how this whole internal private networking thing works. I'd like to create a system that can do private networking once I'm ready to move the entire app into the cloud. But I want to prototype on my local machine, so that leaves me with TCP.

This is using the docker template: https://railway.app/template/tifygm

The project ID: a141d396-2ac4-44e2-820c-06fd15fa7c8b

Any help at all would be super appreciated. Thank you!

[bright solar]

Project ID: a141d396-2ac4-44e2-820c-06fd15fa7c8b

[daring swan]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[smoky timber]

chromadb communicates over http, not tcp.
add a service variable PORT set to 8000
remove the tcp proxy and generate a railway domain
for the client, the host is the railway domain and the port is 443

keep mind that this makes the database fully publicly accessible, there is no password protection

take a look at this section of this article i found
https://blog.amikos.tech/secure-your-chroma-db-instance-part-1-authentication-c2f1979e7c19#159c

[deft pelican]

thank you!

[smoky timber]

no problem

[deft pelican]

incredibly helpful

It works!

[smoky timber]

awsome

[deft pelican]

Okay, I spoke too soon.

import chromadb
from chromadb import Settings

client = chromadb.HttpClient(
    host='chroma-production-myspecialurl.up.railway.app',
    port='443',
    ssl=True,
    settings=Settings(allow_reset=True, anonymized_telemetry=False),
    headers={'authorization': 'PASS'}
)

from chromadb.utils import embedding_functions

embedding_fn = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name='BAAI/bge-base-zh-v1.5'
)

collection = client.get_or_create_collection(
    "demo",
    embedding_function=embedding_fn
)

This all works fine, and I'm able to do things like collection.count()

But when I try to add docs, it throws a 404.

collection.add(
    documents=documents[:10],
    metadatas=metadata[:10],
    ids=ids[:10],
)

Result:

HTTPError: 404 Client Error: Not Found for url: https://chroma-production-9aba.up.railway.app:443/api/v1/pre-flight-checks

...

Exception: {"detail":"Not Found"}

(not my actual password)

[smoky timber]

thats more so a question for chroma's docs

[deft pelican]

thanks again for superhuman response time

https://github.com/chroma-core/chroma/issues/1204

It was this. Client-server mismatch. I had to load a new docker image. The one-click deploy Chroma docker image that you have is 0.4.5. Latest version is 0.4.15.

I'd suggest changing it from a hardcoded version to the dockerhub chromadb, which is just chromadb/chroma. So that other people don't fall into this same issue.

[smoky timber]

id like to note i dont work for railway, and that the template is user submitted

[deft pelican]

well then, I very much appreciate all your uncompensated labor. I hope you get some joy out of it.

[smoky timber]

haha thank you

[autumn shore]

This is good feedback, thank you!

[smoky timber]

give conductors admin perms on templates 🙂

[autumn shore]

Will bring up with team