I want to use ChatGPT to access a website that provides SOAP API documentation from an ERP system. It should help me to retrieve data from the ERP via the API and visualize it, for example. However, ChatGPT always reports that it cannot access this page because it contains JavaScript and the browser must support frames (which, for example, the Edge browser actually supports). I have tried the GPT-4 with Bing search, but also plugins like Link Reader, Webpilot etc. or GPTs like Scraper, Web Scraper etc.. None of these methods can read the website. This is also the case with many other websites.
Does anyone know of a solution that makes ChatGPT crawl/scrape websites that require JavaScript and frames to serve data for different use cases? Or are there intermediate solutions that can provide ChatGPT with such data?
#Chat with API of ERP
worldly compass
sterile muralBOT
Hi @worldly compass. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Based on the information provided in the extracts, it appears that ChatGPT uses a user agent called ChatGPT-User/1.0 to crawl web pages and retrieve content to answer user queries. However, the extracts do not provide specific details on how ChatGPT handles JavaScript or frames on web pages.
ChatGPT is trained to not repeat information verbatim from the data, but it will paraphrase, translate, summarize, and abstract information as requested by the user. The retrieved content is used to teach the models how to respond to a user request given this content, not to make the models better at creating responses. The content may be seen by OpenAI employees and trainers who are improving the models. The data is fetched on nearly every user request, using caching to reduce load, striving to show the freshest content to users. The data is stored for training purposes and so users can access the conversation history in their accounts source (https://platform.openai.com/docs/plugins/bot).
OpenAI is also beginning to extend ChatGPT’s capabilities by teaching it to use plugins. You can think of plugins as “tools” that the model can learn how to use. However, the extracts do not provide specific details on how these plugins handle JavaScript or frames on web pages source (https://help.openai.com/en/articles/7183286-how-to-access-plugins).
If you're facing issues with accessing certain web pages using ChatGPT, it might be due to limitations in the current implementation of the model or the plugins. It's recommended to reach out to OpenAI's support for more specific guidance on this issue.