#🔒 trying to extract <script> var data from website

13 messages · Page 1 of 1 (latest)

wispy oracle
#

i'm trying to get the "var item_details" data in my code. i've heard that when you see "script" or var aka variable, it's to do with javascript but i have no idea how to extract it. i appreciate any help thanks!

i just got basic request code atm:

import re
import json
import requests

url = 'https://www.rolimons.com/deals'
response = requests.get(url)
image.png
long vaultBOT
#

@wispy oracle

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
wispy oracle
wind drum
#

idk if python has javascript parser libraries that could help you with that, but you can get what you want by splitting on "var", then on " = ", and indexing correctly ig

#

then you can use json to load the string into a python dict

hazy sage
# 
import re
import json
import requests

url = 'https://www.rolimons.com/deals'
response = requests.get(url)
pattern = r'var item_details\s*=\s*(\{.*?\});'

match = re.search(pattern, response.text, re.DOTALL)
if match:
    json_text = match.group(1)
    try:
        data = json.loads(json_text)
        print(data)
    except json.JSONDecodeError as e:
        print("json decode error:", e)
wispy oracle
#

tysm for ur help guys i appreciate it

#

that worked a charm

hushed juniper
#

just wanted to add that using regex to parse html pages isn't really optimal

instead, using an xml or html parser is considered a way better solution. In the future you may want to take a look at beautifulsoup4.

wispy oracle
hushed juniper
#

no problem, happy to help

long vaultBOT
#
Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.