Rescanning Happening Hourly, Ignoring Library Refresh setting? | ErsatzTV | Page 1

spark pulsar Jan 19, 2023, 11:09 PM

#

I've noticed that ErsatzTV appears to scan my library quite often, causing the fans on the machine I am running it on to go full blast for a few minutes. The library scanning is set to scan every 6 hours but judging from the number of times I was finding myself annoyed by the fan noise, it seemed far higher than that. I decided to set the library scanning to once every 24 hours, however looking at the logs, I can see that it appears to kick in hourly regardless of the setting and seems to take 5 minutes or so to scan. (My logs show all the different pieces of filler and media being skipped due to having zero duration and I see those hourly, which I assume happens during scanning the libraries, but please correct me if I am wrong.)

I am running ErsatzTV Version 0.7.2-f6c345d7-docker-vaapi on a Linux docker host.

blissful nexus Jan 19, 2023, 11:46 PM

#

The scanner checks every hour to see which libraries have a last successful scan time of older than now minus that setting

#

So if it's actually scanning stuff every hour then something is preventing the scan from completing successfully

#

Happy to look at logs

#

I don't think the duration stuff would crash the scanner.

spark pulsar Jan 20, 2023, 12:04 AM

#

I'll grab them shortly.

#

There's nothing there in the logs that's interesting, so I'm setting it to debug level logging and I'll wait until the next hourly scan and send it then.

blissful nexus Jan 20, 2023, 12:10 AM

#

Ok, I could also give a query or point you to the table that would show the last successful scan time. Would confirm my theory

spark pulsar Jan 20, 2023, 12:14 AM

#

Sure,

blissful nexus Jan 20, 2023, 12:31 AM

#

Ok, on my phone, will send once I'm back at my desk

blissful nexus Jan 20, 2023, 1:37 AM

#

alright, something like this should work from the host, if you have sqlite3 avail, otherwise you can shell into the container, apt update/install sqlite3 and do it there sqlite3 ersatztv.sqlite3 'select Path, LastScan from LibraryPath order by LastScan desc'

#

note that the db times are all UTC so it will be off by whatever your offset is

spark pulsar Jan 20, 2023, 3:15 AM

#

/content/wcw/worldwide/|2023-01-19 22:15:46.0588067
/content/wcw/pro/|2023-01-19 22:15:41.219581
/content/wcw/powerhour/|2023-01-19 22:15:36.3169754
/content/wcw/mainevent/|2023-01-19 22:15:31.472648
/content/nfl|2023-01-19 22:15:26.6454099
/content/pbs/carmen|2023-01-19 22:15:21.8823063
/content/pbs/squareone|2023-01-19 22:15:16.9478817
/content/rickmercer/|2023-01-19 22:15:11.9711981
/content/audio/GTAVC|2023-01-19 22:15:06.9553808
/content/deadlock|2023-01-19 22:15:01.8976094
/content/local58/|2023-01-19 22:14:56.8542542
/content/sf|2023-01-19 22:14:51.7518684
/content/commercials/christmas|2023-01-19 22:14:46.8274167
/content/levelup|2023-01-19 22:14:41.9956205
/content/gtav/Commercials|2023-01-19 22:14:37.1779493
/content/gtav/Weasel|2023-01-19 22:14:32.1483061
/content/gtav/CNT|2023-01-19 22:14:27.3334647
/content/popupvideo|2023-01-19 22:14:22.4403551
/content/raw|2023-01-19 22:14:17.5296191
/content/commercials/disney|2023-01-19 22:14:12.5363299
/content/thepriceisright|2023-01-19 22:14:07.570093
/content/judgejudy|2023-01-19 22:14:02.4667484
/content/maury|2023-01-19 22:13:57.5057938
/content/habsgames|2023-01-19 22:13:52.5982271
/content/failarmy|2023-01-19 22:13:47.8001747
/content/ppv/wwe|2023-01-19 22:13:42.9896254
/content/ppv/aew|2023-01-19 22:13:38.1116673
/content/concerts|2023-01-19 22:13:33.3201213
/content/musicvideos|2023-01-19 22:13:28.5281852
/content/commercials/90s/90stoys|2023-01-19 22:13:15.718009
/content/commercials/90s/90ssports|2023-01-19 22:13:10.8967862
/content/commercials/90s/90sphone|2023-01-19 22:13:06.0456275
/content/commercials/90s/90sgames|2023-01-19 22:13:01.0917544
/content/commercials/90s/90sfood|2023-01-19 22:12:56.093269
/content/commercials/90s/90scars|2023-01-19 22:12:51.1993146
/content/commercials/80s|2023-01-19 22:12:46.2432519
/content/commercials/2000|2023-01-19 22:12:41.1735439
/tv/Howard TV/|2023-01-19 22:12:36.313019
/tv|2023-01-19 22:12:36.1807589
plex://af7d45f7-7099-4446-a197-679147baa148|0001-01-01 00:00:00
plex://d59b5b03-c61e-4f1e-9e88-85eccfedd2ac|0001-01-01 00:00:00

Something I notice right off the bat here is that I have one Plex library that I'm playing movies from, but no idea why there are two plex entries here. Also, both look like they've never completed successfully, and yet movies show up in my channels... and even sometimes too often, where stations that have content from one of the other libraries plays movies instead. Hmm.

#

Every other library here is local type.

#

https://cloud.zerojay.com/s/RTWJxRAW8AnSCSm - Log file, debug enabled for the last 2 hours or so.

Nextcloud

ersatztv20230119.log

Nextcloud - a safe home for all your data

blissful nexus Jan 20, 2023, 3:20 AM

#

can you try sqlite3 ersatztv.sqlite3 'select Value from ConfigElement where Key = "scanner.library_refresh_interval"'

#

as far as i can tell from the logs, the scanning itself isn't the issue, it's the other hourly tasks that take a few minutes

spark pulsar Jan 20, 2023, 3:26 AM

#

root@cf2a84815e44:~/.local/share/ersatztv# sqlite3 ersatztv.sqlite3 'select Value from ConfigElement where Key = "scanner.library_refresh_interval"'
24

blissful nexus Jan 20, 2023, 3:27 AM

#

the scanner is different than before, in that it is its own executable, which probably trades off speed for long term memory use. the reason i made it its own process is so the OS can reclaim literally all memory the scanner uses once it shuts down (after scanning a single library)

#

it does look like it takes 3 minutes to skip scanning all of your libraries

#

maybe i can optimize that check so the other process doesn't get called when scanning isn't needed

#

but they all log things like 2023-01-19 20:07:43.534 -05:00 [DBG] Skipping unforced scan of local media library Local58 2023-01-19 20:07:43.610 -05:00 [DBG] Done scanning local library Local58

#

ill take a closer look at the separate scanner process tomorrow. 3 minutes to basically do nothing is a long time. thanks for posting

spark pulsar Jan 20, 2023, 3:30 AM

#

Thanks for being so reactive, it's appreciated.

#

If we're also able to lower the priority (niceness) on the scanning process, that would be great but that's probably a more long-term idea.

blissful nexus Jan 20, 2023, 3:44 AM

#

edit: found how to set priority. will give it more thought. perhaps i can lower for automatic background scans (vs clicking to scan in the ui)

spark pulsar Jan 20, 2023, 4:18 AM

#

Sounds good to me.

blissful nexus Jan 20, 2023, 12:00 PM

#

both of these changes have been made

spark pulsar Jan 20, 2023, 4:03 PM

#

I'll pull and let you know.

#Rescanning Happening Hourly, Ignoring Library Refresh setting?