I have huge json file with many objects from Apify.com, each object is a business, the file is too big and when I request it in the browser in order to save the results to mysql it's waiting for hours to load the file. What should I do? I need to process the file before insert it to mysql.
#How to handle a huge Json file?
reef crown
severe kestrel
Are you downloading it or opening in new tab? Downloading expected to work even for huge files, if its too big to process consider to split by https://apify.com/lukaskrivka/dedup-datasets and process data by chunks
reef crown
I open it in a new tab, when I try to open it in vs code, vs code crashed each time. I will check this. thanks!
severe kestrel
New tab will be browser killer for sure, you need to dll file and then use notepad or sublime to open it, basically anything wo formatting or linting applied on opening, not vscode exactly because of post-processing, it works fine only for regular file sizes.