📝 If you want to review this again, abhi:
:one: Right-click (or long press with phone) on the message that contains the log
:two: Select: Copy Message Link
:three: Use the command: /logscan <message_link> or !logscan <message_link> and paste the value copied from the previous step where you see <message_link> 📝
#optimizing to run quicker
fathom horizonBOT
Welcome @toxic vector!
Someone from <@&938443185347244033> will assist when they're available.
Including the meta.log from the beginning is a huge help, type !logs for more information.
You can press the "Close Post" button above or type /close at any time to close this post.
fathom horizonBOT
You can press the "Close Post" button above or type `/close` at any time to clos...
Hello!
I am trying to figure out what I can do to make it so the overall script runs quicker. Right now, I see that the biggest slow down is when it goes through a list on mdblist which has about 18000 movies in it. Most of the movies aren’t in my library and it looks like it checks it every time. In the screenshot it says it’s a TMDb error but I don’t have it anywhere in my movie collections file? I attached my movies and config yml as well as the latest run log.
toxic vector
!logscan #1228422611365793872 message
jolly hearth
Right now, I see that the biggest slow down is when it goes through a list on mdblist which has about 18000 movies in it. Most of the movies aren’t in my library and it looks like it checks it every time
There's no way to avoid that. Every run PMM is going to have to check which of those 18000 movies are in Plex this time.
Options would be to run that less frequently or use a more narrowly-focused list.
This run only took 3 minutes, though:
Finished: 12:06:38 2024-04-12 Run Time: 0:03:14
Of course, nearly all of that was this one collection:
Finished Bollywood Collection
Collection Run Time: 0:02:42
But 2:42 to process 19000 movies seems pretty reasonable to me.
toxic vector
Oh hmm maybe I was just impatient. I did have another question.
I know IMDb tends to search the fastest. I was trying to make a search url work with imdb_search but I keep getting an error. I’ll send over more info as soon as I am free from work.
toxic vector
Here is the log file for whats happening
lament kraken
what's in movies.yml and tv.yml? can you share both?
if there isn't metadata, then you don't need that section of your config. it's throwing an error because there isn't anything metadata related in those files
jolly hearth
"Bollywood" and "Punjabi" are incorrectly formatted in the YAML.
[DEBUG] | Validating Method: imdb_search
[DEBUG] | Value: None
[DEBUG] | Traceback (most recent call last):
| File "//plex_meta_manager.py", line 747, in run_collection
| builder = CollectionBuilder(config, metadata, mapping_name, collection_attrs, library=library, extra=output_str)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/modules/builder.py", line 976, in __init__
| raise Failed(f"{self.Type} Error: {method_final} attribute is blank")
| modules.util.Failed: Collection Error: imdb_search attribute is blank
|
[ERROR] | Collection Error: imdb_search attribute is blank
We'll need to see the collection definition to say specifically how. Chances are it's incorrect indentation.
The error on the other two is fixed in the nightly branch.
toxic vector
Here is the movies.yml file. I dont have a tv.yml made because this is a fresh install of PMM and want to start with movies collection first before I move on to anything else. Thats also why metadata is not made yet!
I changed the docker to the nightly build
meisnate12/plex-meta-manager:nightly
toxic vector
Okay so an update. I was able to get the Marvel and Walt disney one to work, albeit the bollywood and punjabi one still doesnt, even though they share the same style of indentation
Here is the most up to date movies.yml collections file
Here is the latest log file
fathom horizonBOT
tough rampart
On the IMDB website, if you do an advanced search, they use "HI" for Hindi, and "PA" for Punjabi. You have "hin" and "pan" and I wonder if the extra characters are the issue.
Since the languages are the only difference between the ones that work and the ones that don't....
toxic vector
Wow that worked im silly. Missed that part in the PMM wiki
tough rampart
hahaha Glad it was that easy!
toxic vector
Well I re ran it looks like bollywood and walt disney worked, punjabi and marvel didnt work
Maybe I missed something again. Ill double check
tough rampart
Especially if you already had Marvel working
toxic vector
Interesting. I changed nothing this time but deleted the collections that got made from the last run and now the only one it wants to do is Marvel
Deleted the collections that got made in plex ^
Changed nothing in the movie.yml collection. Deleted the marvel collection that got made in plex ^ and ran it again. Now none of them are working again
tough rampart
post the log
fathom horizonBOT
toxic vector
Seems like sometimes it wants to work and then it just doesnt. Unless im missing something?
tough rampart
I'm not 100% sure but I think the Unknown Error: 'data' that you're getting may be an issue with IMDB and you may want to run the nightly build. Someone with more experience on that one would have to chime in when they get a minute.
toxic vector
I put it onto the nightly build earlier. I have it set to
meisnate12/plex-meta-manager:nightly
tough rampart
Oh yeah, so you are. Could still be an issue either with IMDB or with the code (IMDB may have changed something).
toxic vector
Is it possible there is a limit on how often IMDb allows you to run the command?
I havent ran it in a few minutes let me try it out now
tough rampart
I'm sure there's a limit but I don't think you could run it that fast
toxic vector
I ran it again and now the marvel and walt disney one works haha
tough rampart
I would expect those to work every time so it may just be an issue with IMDB or network. Maybe try again later?
toxic vector
Yeah I'll try again later and report back! For now here are the latest logs incase anyone wants to see them!
fathom horizonBOT
lament kraken
Here is the movies.yml file. I dont have a tv.yml made because this is a fresh i...
that is why you're getting some of the errors
toxic vector
that is why you're getting some of the errors
In relation to the collections being inconsistent?
I just ran it again and this time it only put 51 movies in the bollywood collection where once before it did 200 movies
lament kraken
In relation to the collections being inconsistent?
No. Some of the errors in the logs, saying metadata is missing or file is blank
toxic vector
Do they need to be there for the issue being discussed?
lament kraken
for the issue of "optimizing to run quicker" will clean up the file so it's not trying to run files that don't exist, or that do exist and do not have metadata; or in the case of tv, they just don't exist
toxic vector
I mentioned earlier that I’ll be making those files I just haven’t gotten around to it cause I wanted to fix this issue first
I appreciate the tip though. I do plan on making them just want to figure out why IMDb Search builder is being weird
jolly hearth
[INFO] | Processing IMDb Search:
[INFO] | limit: 100
[INFO] | type: ['movie']
[INFO] | country: ['IN']
[INFO] | language.primary: ['hi']
[ERROR] | Response: {'errors': [{'message': 'PersistedQueryNotFound', 'extensions': {'code': 'PERSISTED_QUERY_NOT_FOUND'}}]}
This looks like a problem inside IMDB, since IMDB is returning that error.
It could be that you are using a two-character language code [639-1] where the docs say a three-character [639-2] is needed.
jolly hearth
I can't reproduce this, and the docs appear to be wrong about the language code.
|================================ Running ahbi_test_hindi Collection
|
| Sync Mode: append
|
| Builder: imdb_search: {'limit': 100, 'type': ['movie'], 'country': ['IN'], 'language.primary': ['hi']} |
|
| Processing IMDb Search:
| limit: 100
| type: ['movie']
| country: ['IN']
| language.primary: ['hi']
|
| 100 IDs Found
|
|
|=============================== Adding to ahbi_test_hindi Collection
|
| 1/4 | ahbi_test_hindi Collection | = | 3 Idiots (2009)
| 2/4 | ahbi_test_hindi Collection | = | Jawan (2023)
| 3/4 | ahbi_test_hindi Collection | = | Pathaan (2023)
| 4/4 | ahbi_test_hindi Collection | = | Gadar 2 (2023)
|
| 4 Movies Processed 0 Movies Added
|
|========================= Updating Metadata of ahbi_test_hindi Collection
versed gorge
```
[INFO] | Processing IMDb Search:
[INFO] | limit: 100
[INFO] ...
This should hopefully be resolved
toxic vector
So was it something I did wrong?
I was originally using a three letter language code (hin) but was corrected to 2 letter (hi) according to the PMM WiKi
jolly hearth
I believe it was an issue on the PMM side; the nightly wiki still points to 639-2 language codes, so that probably needs updating.
toxic vector
Well I did try both hi and hin
as well as Marvel and Walt Disney not including a language parameter at all
Maybe I read what you said wrong. I guess you mean that something in the PMM code needs to be updated and I'll just have to wait till then?
Which is fine! I just want to give you guys whatever info possible to help solve this!
jolly hearth
Should be addressed in nightly, I believe.
toxic vector
Is there a way to check which nightly build I am on? I am running
meisnate12/plex-meta-manager:nightly
jolly hearth
In my case, this:
| limit: 100
| type: ['movie']
| country: ['IN']
| language.primary: ['hi']
|
| 100 IDs Found
produced those 100 movies, comnpared to:
| Processing IMDb Search: |
| limit: 100 |
| type: ['movie'] |
| country: ['IN'] |
| language.primary: ['hin'] |
| Traceback (most recent call last): |
| File "/opt/pmm/Plex-Meta-Manager/plex_meta_manager.py", line 775, in run_collection |
| builder.filter_and_save_items(builder.gather_ids(method, value)) |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| File "/opt/pmm/Plex-Meta-Manager/modules/builder.py", line 2201, in gather_ids |
| ids = self.config.IMDb.get_imdb_ids(method, value, self.language) |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| File "/opt/pmm/Plex-Meta-Manager/modules/imdb.py", line 580, in get_imdb_ids |
| return [(_i, "imdb") for _i in self._search(data)] |
| ^^^^^^^^^^^^^^^^^^ |
| File "/opt/pmm/Plex-Meta-Manager/modules/imdb.py", line 445, in _search |
| raise Failed("IMDb Error: No IMDb IDs Found") |
| modules.util.Failed: IMDb Error: No IMDb IDs Found |
| |
| During handling of the above exception, another exception occurred: |
| |
| Traceback (most recent call last): |
| File "/opt/pmm/Plex-Meta-Manager/plex_meta_manager.py", line 780, in run_collection |
| raise Failed(e) |
| modules.util.Failed: IMDb Error: No IMDb IDs Found |
The specific version is at both ends of the log.
toxic vector
Looks like its working!
Is there a way to change the limit to higher than 100?
The list has around 18000 movies in it. On my server I should have around 280 of them
jolly hearth
Sure, set limit to whatever you want. 100 is the default, but you can override it by providing a value.
lament kraken
@toxic vector anything further needed?
toxic vector
Nope that’s all! Working great now