#🔒 Is there any way to get ImageHash work with python 3.13
loud mist
is ImageHash a pypi package?
stiff aspen
Ok seems like discord bot may have deleted my original question. The question was is there a way to install ImageHash on python 3.13 on Windows 10? It gives some compilation errors. I want to know if there is some easy way to fix them myself or if waiting for package maintainer is the only way?
Installing it works on python 3.9.7 but not with 3.13
fast dockBOT
loud mist
bot probably deleted your original post because you uploaded a text file
wind inlet
The last version of ImageHash was uploaded using python 3.10.6
loud mist
last commit was 2 months ago
loud mist
erroring while installing pywavelets, which has a py3.13 wheel
stiff aspen
But I don't know how to tell pip install to use that github repository instad.
Or if version from their github would even fix it.
loud mist
Can you post pip debug?
wind inlet
the github hasn't been updated in 2 years either so i doubt that'll help
loud mist
Says yesterday. https://github.com/PyWavelets/pywt
wind inlet
Oh i meant the imagehash
loud mist
it's erroring on dependencies
muted spruce
These are the available distribution files on PyPI for PyWavelets 1.7.0:
https://pypi.org/project/PyWavelets/#files
It looks like there are wheels for many Python 3.13 variations.
Have you tried pip install --prefer-binary PyWavelets do you have any joy?
For clarity, Python distributi0oon files come in 2 forms: source distributes (which is what is being compiled on your machine) and wheels, which are prebuilt zips and are just unpacked, not requiring compliation.
stiff aspen
Interesting it seems like it errors when I create venv with python3.13t which disables GIL. But not normally.... Have to test more to be sure....
My pip debug as requested: https://paste.pythondiscord.com/QQNA
muted spruce
pip seems to have a preference for source, perhaps because it allows a package to be customised to your machine. But if the build fails, that's not a win.
loud mist
It's why I asked for pip debug. It shows the platforms supported.
sys.platform: win32 That checks out.
Try running pip with --prefer-binary
stiff aspen
pip install --prefer-binary PyWavelets had also C compilation error.
loud mist
Try with --upgrade as well
stiff aspen
pip install Imagehash --prefer-binaryseems to fail with same error. It still tries to compile it. And also pip install Imagehash --prefer-binary --upgrade
I did python.exe -m pip install --upgrade pip - seems like pip doesn't want to respect --prefer-binary option
muted spruce
That's the preferred way anyway.
Try replacing --upgrade and --prefer-binary with --force and --only-binary.
Sorry, --force-reinstall.
--force-reinstall Reinstall all packages even if they are already up-to-date.
--only-binary <format_control>
Do not use source packages. Can be supplied multiple times, and each time adds to the existing value. Accepts either ":all:" to disable all source
packages, ":none:" to empty the set, or one or more package names with commas between them. Packages without binary distributions will fail to
install when this option is used on them.
So I guess I mean:
python.exe -m pip install --force-reinstall --only-binary :all: PyWavelets
ERROR: Could not find a version that satisfies the requirement PyWavelets (from versions: none)
ERROR: No matching distribution found for PyWavelets
stiff aspen
pip 24.3.1 from C:\dir\py313testing\venv\Lib\site-packages\pip (python 3.13)
muted spruce
Silly question: do you need to use 3.13 at all? Versus some older version?
loud mist
pywavelets may not support freethreading.
stiff aspen
But the interesting thing is that this compilation only fails with when I create venv with python3.13t and not when normal python command (still 3.13 but not t) version.
loud mist
pywavelets doesn't have a wheel for 3.13t
stiff aspen
Python's Global Interpreter Lock (GIL) can limit the performance of multi-threaded programs,
especially for CPU-bound tasks. The GIL allows only one thread to execute Python bytecode at a time,
which can lead to suboptimal CPU utilization.
Which is why I wanted to try with GIL disabled version of python.
muted spruce
That's for pure Python code.
muted spruce
Serious computation runs compiled code and releases the GIL during the computation. The GIL's usually not a bottleneck for such things.
stiff aspen
I know there is version of threading that uses processes instead but it started to serialize things like database connections to send these also to new processes and then I gave up because it meant too much refactoring and thought I would try my luck with 3.13 GIL free version.
muted spruce
In the compiled side the Python extension code basicly goes: set it all, release the GIL (other Python threads can now run), compute, reacquire the GIL, mashal results back to Python.
The subprocessing module runs distinct executables. But I'm slightly skeptical that the GIL's actually an issue.
stiff aspen
With normal concurrent.futures.ThreadPoolExecutor it didn't use 100% CPU.
muted spruce
That may just mean it's blocked (eg on IO) and free threading won't change that.
Typical example of GIL compute bottleneck is 100% of one CPU and nothing using more CPU cores.
stiff aspen
I mean task manager showed combined usage.
So it could be that 1 cora already was 100%
muted spruce
Maybe so. I'm not sure I can help with that.
But it does sound like you may be out of luck using PyWavelets with the 3.13t stuff, alas.
muted spruce
But maybe you can run multiple python processes, each working on different datasets?
stiff aspen
So solution for now would be to post feature request to https://github.com/PyWavelets/pywt get 3.13t binaries out? And after that Imagehash would also start to install?
muted spruce
Even something as simple as explicitly:
py my_script.py dataset1 &
py my_script.py dataset2 &
Sorry the & is UNIX shell for "run in the background instead of waiting for this command to complete. Whatever you need on Windows to acheive that. Which might be as simple as just a few command windows, running a python in each.
muted spruce
So solution for now would be to post feature request to https://github.com/PyWav...
Could be.
stiff aspen
will then ImageHash automatically understand that I have installed PyWavelets myself and not attempt to install those anymore?
loud mist
once it gets installed.
muted spruce
It should do. pip is just seeing PyWavelets in the dpendencies and build it (well, failing to build it). If it's already there you shou;d be fine.
stiff aspen
But maybe you can run multiple python processes, each working on different datas...
Yea. That might work. I remember sqlite didn't like threads. So my original idea was to gather bulk of image hashes and insert them in main thread and only do hashing in threaded way. But if multiple process write it at the same time - not sure about that.
muted spruce
Yeah, I have problems with sqlite from multiple processes. I usually put a lock file around my use of the sqlite db.
So:
- take lock file (for me,
f'{the-sqlite-db-filename}.lock') - open the db, do the things, close
- remove the lock file
Step 1 tries to make the lock file exlusively (x open mode can do that) and if if fails, sleeps briefly then tries again.
fast dockBOT
Python help channel closed