Is it possible to run a model on a single/multiple server-like devices for other clients (for example simple notebooks) to acces the power of those stron server-like devices using something like an api key?
#Hosting the ai on a single device for other clients in network to acces
terse osprey
boreal dockBOT
Hi @terse osprey. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Yes, it is possible to run a model on a server and allow other clients to access it. This can be achieved by enabling the local server in GPT4All and allowing any application on your device to use GPT4All via an OpenAI-compatible GPT4All API. The local HTTP port for the local API server is 4891 by default, but it can be changed according to your needs source (https://docs.gpt4all.io/gpt4all_desktop/settings.html#settings).
However, as of now, GPT4All does not provide a web interface for remote clients. It is primarily a client-only application. But it can be installed and used on an Ubuntu system (server) and can be used by remote clients (in particular with Windows or Linux) by using virtual remote desktops source (https://github.com/nomic-ai/gpt4all/issues/1650).
There are also discussions in the community about the need for a remote server mode within the UI client, so that it is possible to run a server on the LAN remotely and connect with the UI source (https://github.com/nomic-ai/gpt4all/issues/820).
Moreover, there is a feature request for GPT4All to make requests to external services, specifically to query an external chat API service hosted on a server source (https://github.com/nomic-ai/gpt4all/issues/2553).
Please note that these features are currently under discussion and may not be available at the moment.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
terse osprey
@boreal dock there is a funtion in the gpt4all client to, instead of running a model on your machine, to input an api key, which lets you utilize another device's resources. how do I generate an api key on another one of my devices which works across the LAN network?
@boreal dock there is a funtion in the gpt4all client to, instead of running a model on your machine, to input an api key, which lets you utilize another device's resources. how do I generate an api key on another one of my devices which works across the LAN network?
boreal dockBOT
Let me check the knowledge base for you. One moment! :books:
Based on the provided knowledge sources, GPT4All allows you to add your API key for remote model providers. However, it does not specify how to generate an API key on another device for use across a LAN network source (https://docs.gpt4all.io/gpt4all_desktop/models.html#connect-model-apis).
For Nomic, you can obtain your API key at https://atlas.nomic.ai/cli-login. Make sure you are already signed into Atlas in your browser at https://atlas.nomic.ai/. Once you have your API key, you can log in to your Nomic account in your Python code (or your terminal) with your API token source (https://docs.nomic.ai/atlas/introduction/quick-start#uploading-your-unstructured-dataset).
However, please note that this API key is for accessing Nomic's services and not specifically for enabling LAN access to a GPT4All model running on another device.
If you want to run a model on a server and allow other clients to access it, you might need to set up a server application that exposes an API endpoint. This server application would handle requests from clients, use the model to generate responses, and send these responses back to the clients. This setup would require programming knowledge and might involve technologies such as Flask or Django for Python.
Please refer to the relevant documentation or seek advice from a network administrator or a software engineer to ensure a secure and efficient setup.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
terse osprey
@boreal dock "If you want to run a model on a server and allow other clients to access it, you might need to set up a server application that exposes an API endpoint. This server application would handle requests from clients, use the model to generate responses, and send these responses back to the clients. This setup would require programming knowledge and might involve technologies such as Flask or Django for Python." does an application like this already exist?
boreal dockBOT
Let me check the knowledge base for you. One moment! :books:
Based on the provided knowledge sources, there isn't a specific application mentioned that already provides this functionality for GPT4All. However, there are discussions in the GPT4All community about developing such features. For instance, there's a mention of a server based on the node.js bindings that is being worked on, which can handle concurrent inference source (https://github.com/nomic-ai/gpt4all/issues/1816).
For now, you might need to develop a custom server application that exposes an API endpoint, as mentioned earlier. This would involve setting up a server, loading the model on the server, and writing code to handle API requests and responses. Technologies such as Flask or Django for Python could be used for this purpose.
Please note that developing such an application would require programming knowledge and understanding of how to work with APIs, servers, and potentially machine learning models. It's recommended to refer to the relevant documentation or seek advice from a software engineer to ensure a secure and efficient setup.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
terse osprey
also, which document types can gpt4all access/analyze?
@boreal dock which document types can gpt4all acces/analyze? Is this dependent on the model that is currently used?
boreal dockBOT
Let me check the knowledge base for you. One moment! :books:
GPT4All's LocalDocs feature can access and analyze the following document types:
- Plain text files (
.txt) - Markdown files (
.md) - RestructuredText files (
.rst) - PDF files (
.pdf)
This is specified in the LocalDocs settings of GPT4All and is not dependent on the model that is currently used source (https://docs.gpt4all.io/gpt4all_desktop/settings.html#localdocs-settings).
However, there is a feature request to add support for parsing CSV, JSON, and XML files in the future source (https://github.com/nomic-ai/gpt4all/issues/2059).
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
terse osprey
So if i were to have a pdf depicting a table with informations on it, gpt4all could understand all of these informations?
@boreal dock So if i were to have a pdf depicting a table with informations on it, gpt4all could understand all of these informations?
boreal dockBOT
Let me check the knowledge base for you. One moment! :books:
GPT4All's LocalDocs feature can process and analyze PDF files. However, the ability to accurately extract and understand information from a PDF, especially complex structures like tables, can depend on several factors such as the quality of the PDF, the complexity of the table, and the presence of a text layer in the PDF.
There is a feature request for LocalDocs to use Optical Character Recognition (OCR) to read text in PDF files without a text layer, which would be particularly useful for scanned documents source (https://github.com/nomic-ai/gpt4all/issues/1255).
However, the provided knowledge sources do not specify how well GPT4All can handle tables within PDFs. It's possible that complex table structures might not be perfectly interpreted. For best results, the data within the PDF should be in a clear, readable format.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
terse osprey
@boreal dock what is meant with the "text layer"?
boreal dockBOT
Let me check the knowledge base for you. One moment! :books:
A "text layer" in a PDF refers to the layer of the document that contains searchable, selectable text. When a PDF is created from a text document, such as a Word document or a webpage, it usually includes a text layer. This means you can search for specific words, select text to copy and paste, and so on.
However, when a document is scanned and saved as a PDF, it's often saved as an image. This means there's no text layer - the PDF just contains an image of the text, not the text itself. In this case, you can't search or select the text in the PDF without using Optical Character Recognition (OCR) to add a text layer.
The feature request mentioned in the source (https://github.com/nomic-ai/gpt4all/issues/1255) is asking for LocalDocs to use OCR to read text in PDF files that don't have a text layer, such as scanned documents.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
terse osprey
@boreal dock So Localdocs currently does not have OCR?
boreal dockBOT
Let me check the knowledge base for you. One moment! :books:
Correct, based on the provided knowledge sources, LocalDocs does not currently have built-in Optical Character Recognition (OCR) capabilities. This means it may not be able to extract text from scanned PDF files or images that do not have a text layer source (https://github.com/nomic-ai/gpt4all/issues/1255). However, there is a feature request for adding OCR capabilities to LocalDocs, indicating that this might be a functionality added in the future.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
terse osprey
@boreal dock can localdocs understand everything that isn't text, that is going on in a document. Say for example a visual graphic that explains how a technical component works. Could it for example convert this graphic into a text explanation accesible for the user?
boreal dockBOT
Let me check the knowledge base for you. One moment! :books:
No, based on the provided knowledge sources, LocalDocs does not have the capability to interpret or convert visual graphics into text explanations. LocalDocs works by maintaining an index of all data in the directory your collection is linked to, and this data consists of small chunks of each document that the Language Learning Model (LLM) can receive as additional input when you ask it a question source (https://docs.gpt4all.io/old/gpt4all_chat.html#how-localdocs-works).
Currently, LocalDocs supports plain text files (.txt, .md, and .rst) and PDF files (.pdf). It does not have the ability to interpret images or graphics within these documents source (https://docs.gpt4all.io/gpt4all_desktop/settings.html#localdocs-settings).