#Context Module behaviour

1 messages · Page 1 of 1 (latest)

vagrant patio
#

What is meant by “considered” here? What action is triggered to select these items, and for what purpose? Is it to establish context for the prompt that chat uses?

In chat, when you see "Read 13 Context Items" these are items that the chat model considered before generating its response.

rugged venture
#

Considered means exactly what you say -- they are included in the prompt for the chat model as information that may be relevant to help answering the user's question

#

As for what triggers this, every time you submit a chat message a model determines based on various factors whether the question you are asking would benefit from context from your local code environment, or not

#

It's not perfect obviously, but for example if you ask it something like "Where in our codebase do we ..." then the model identifies that you are probably asking about your codebase, and tries to find the parts of the codebase you are asking about

#

if you conversely ask "what is the syntax for logging errors in javascript" then it does not do this

vagrant patio
#

Does this initiate a embeddings-based code search if it determines it relates to the codebase?

rugged venture
#

if you have codeium search enabled, then yes

vagrant patio
rugged venture
#

I can't speak to the details of their implementation but embedding search generally works this way yes

#

The biggest difference vs Cody is that we have our own proprietary embedding model and we can run it locallyon your machine

#

So neither the source code nor the emebeddings have to leave your machine

vagrant patio
#

Thanks for the color. I believe they are currently transitioning away from 3rd party embeddings per a call I had with them and their white paper:

https://docs.google.com/document/d/1uU6g-NuHZJgoQ54o4bxd1aYUJLE01czc8sj7-F2tUcM/edit

And from https://about.sourcegraph.com/blog/cody-is-cheating

I also think that self-hosted embeddings are going to be a game-changer, and we’re working to make them available for Cody soon. You will no longer need to upload your code base to a cloud provider; all your IP assets can be embedded in your own secure environment.

Google Docs
Sourcegraph Cody: security and legal whitepaper

This document was last updated on June 6, 2023 and a work-in-progress and subject to change. Sourcegraph Cody ("Cody") is a state-of-the-art AI coding assistant designed to help software developers by answering code questions and generating code. Cody uses Large Language Models (LLMs), Sour...

Sourcegraph
Cody is Cheating

Stevey’s “Cheating With Cody” blog series, Episode 2

#

As an aside, I really like the breakdown that doc does to describe how all the major system components interact in the “Data Flow” section