2.3.0

[summer ravine]

its downloading the model weights

[delicate turret]

no i am talking about this thing i keep referring back to

[raw compass]

uname -a

[delicate turret]

that's going to report the right arch

but we want the python in question to report it, the same way the installer checks it

(unless there's a huge confusion going on here, haha)

[delicate turret]

in case it wasn't clear, you should definitely cancel that install

[summer ravine]

okay.. cancelling

I'm sorry, I'm lost.. what thing did you ask me to do that I didnt do?

trying to cancel.. seems it wont till its done downloading the models

[delicate turret]

this

[raw compass]

I wonder if this is a difference in shells between the install script and your current environment.

[summer ravine]

Okay, thought you were saying the act of installing it in a new location would make it use python 3.10

so how do I force it to use python 3.10?

[delicate turret]

okay so ... it seems like you're not really reading what i am writing

let me try again

[summer ravine]

please do

[delicate turret]

remember this? replace python3 with python3.10

[summer ravine]

yes.. I ran that.. and showed you the output

[delicate turret]

...

[summer ravine]

so run it again, replacing "python3" with "python3.10"?

[delicate turret]

YES PLEASE 🥹

[delicate turret]

dang...

[delicate turret]

yeah i think this is starting to make sense

[summer ravine]

dang?

[delicate turret]

but sorry, i've never seen such a weird thing

[summer ravine]

I thought we had confirmed this with @raw compass command

[delicate turret]

😐

anyhow. sorry, i don't know what else to check. this is a weird problem.

[summer ravine]

👆

[delicate turret]

right, that tells us what we already know. what i was asking was to check which python saw its system as x86

[summer ravine]

ah

[delicate turret]

because that is what the installer does, too.

do you have a conda environment activated or something?

[raw compass]

That would show up with which, though...

What if you run bash

And then my for loop up above.

[delicate turret]

yeah i'm grasping at straws here at this point; just remembering the one other person that had the issue and what they did

really good point about bash; i see the install.sh uses bash instead of zsh so i bet $PATH is different

[raw compass]

So you could just bypass install.sh and run the python line at the end, right?

[delicate turret]

yes, to be honest i was looking forward to the python installer because there would no longer be a sh file ... but i was too hopeful heh

[summer ravine]

dont know

[delicate turret]

the problem with adding all these layers to help people is that it's just another place where something can go wrong and obfuscates things. i'd much rather just be able to say "okay on linux or mac use python3.10 to launch the installer, or on windows py -3.10"

[summer ravine]

like this?

[raw compass]

And now if you run that script that @delicate turret sent you before...

(from within bash)

[summer ravine]

lord.. which one is that?

[raw compass]

python3 -c "import platform; print(platform.uname().machine)"

[summer ravine]

python3 or python3.10?

[raw compass]

And the same thing with python3.10, please...

[summer ravine]

that one IS 3.10

[raw compass]

python3, I mean

[raw compass]

So it seems that's not the problem at all. You can exit bash as that doesn't matter.

[summer ravine]

done

Seems to have installed with no issue!

frew issues though.. GFPGAN and CODEFORMER not found

[wispy wind]

Huh zsh? It should be bash.

[delicate turret]

?

that's what i said, right?

on macos the default shell is zsh; we were thinking that maybe the path env var differed between them, causing the problem

[wispy wind]

Oh. The script has #!/bin/bash at the top of it, so it should always run in bash.

[delicate turret]

right

[wispy wind]

PATH should behave the same also

[delicate turret]

that's the part we were unsure of

[wispy wind]

Gotcha

Ah so install was successful in the end?

Looks like we need to run the configure script again to get support models?

[raw compass]

Indeed.

@summer ravine Re-run and pick option 6 on the menu to re-download those tools that are missing.

[fervent mirage]

btw, it's a good idea. With our domain invoke.ai, we can add a new place for our nodes templates.

[worn river]

Yep. All in the plan

[raw compass]

Node™️ Marketplace™️

[worn river]

Built in Nodes.

[delicate turret]

nodega

[summer ravine]

Thanks, will do

@raw compass its running

[raw compass]

Excellent. And I am, too, to bed.

Hope it all works for you.

[summer ravine]

😉

[raw compass]

I'll probably be back around 5:30AM my time (currently 11PM).

[summer ravine]

wait.. option 6?

[raw compass]

Skip this step if you already have models installed.

Let the rest of the script run.

[summer ravine]

I installed all last time

[raw compass]

Sure, then you can type s to skip.

[summer ravine]

weird.. why wouldnt it install these the first time through?

[delicate turret]

may have failed without you noticing or similar

i am not sure what it looks like when a download fails/stops early/etc

[raw compass]

But this is good and exactly what you want.

[summer ravine]

okay.. going to test it.. Thanks so much to you both.. first time I've ever gotten real help on Discord

Still seeing alot of missing stuff.. Codeformer and GFPgan still missing

[delicate turret]

ok uh

the runtime dir being your installer folder is weird

no wonder it's not finding things

where did you choose to install to?

[summer ravine]

It chose to install to my User directory in an Invokeai folder... I just said yes, go ahead

[delicate turret]

yeah that should have been the right thing. when you run invokeai.sh, are you finding it there, in your home folder? not in the unzipped folder in downloads?

[summer ravine]

ohh.. I shoudl be running it from there, shouldnt I.. duh..

[delicate turret]

also, are you running it like this?
cd ~/invokeai (from opening terminal)
./invokeai.sh (then this once you're in the invokeai folder)

[summer ravine]

was running it from the install directory

swear this thing is taking the piss

[delicate turret]

basically we're back to the start. somewhere in here, you've got some x86_64 stuff and it's not going to work with the arm64 stuff ... i wish i could be more specific but that's all i can say from the errors

the people it's happened to in the past ... one of them had conda or similar, installed from an x86 version of some python IDE ... somebody else had x86_64 as an architecture target for homebrew ... that's all i can say

[summer ravine]

but why does it (almost) work, when I run invoke.sh from the install directory, as I was mistakenly doing?

[delicate turret]

my guess? you had an invokeai folder in your home dir from a previous failed install that you never cleared out

so the installer is installing on top of it, trying to upgrade, etc. using what's already at the right version

[summer ravine]

Hmmmm.. dont think so..

[solar barn]

I've been working on the handling of embeddings, and these error messages are cleaned up. The embedding manager was letting one load an incompatible embedding (trained on one model, used in another), and then issuing an error when generating the first image. This is fixed in 2.3.1

[delicate turret]

i will say ... this doesn't happen out of the blue; something python-related was installed or used at some point that was x86_64 - that's really the only common thread between the people it's happened to in the past. i have a feeling that when we/you figure it out, you'll say "oh, yeah, i did install that, but..."

[summer ravine]

LOL

well, I'm a visual effects artist, so its entirely possible I installed something.. but why would it hose Invoke.ai?

[delicate turret]

Because LOTS of things use python

[solar barn]

I've been having fun and the initial model installer frontend is really nice now. I just need to hook it up to the model manager so that it does something! If anyone would like to try out the interface, knowing that it won't actually do anything, check out https://github.com/invoke-ai/InvokeAI/pull/2644 , do a pip install -e . and then run invokeai-model-install. When you get to the screen that shows the install process, you'll see debug messages indicating that the arguments were gathered correctly.

[delicate turret]

and you could have installed an x86 version of something by necessity or because it works transparently and you never had to worry about it

but now it's not transparent, heh

[summer ravine]

and there's no way to uninstall x86 versions of python?

[delicate turret]

it's a bit more complex than that, something was installed from somewhere but without anybody knowing what it is, we can only guess. it looks like in your most recent error it had to do with opencv and/or numpy. did you ever use homebrew?

[summer ravine]

no.. never

not that I'm aware of

[delicate turret]

my fear is that this is going to be one of those "oh yeah i guess there was that one weird command i had to run to get this other thing to work but i didn't really take note of it" heh

[kindred grail]

have you ever put a sticker on your mac that says "Intel Inside"

[delicate turret]

😮

[summer ravine]

LOL

I shudder at the thought of booting up the PC just to use Automatic1111.. sounds like an aircraft taking off

[delicate turret]

essentially, with python stuff, especially with ML, there's going to be a lot of compiled shared libraries. that's why/how numpy is fast, pillow is fast, etc. some of your shared libraries are for the wrong architecture. they were put there or built and put there at some point, and they're being found when you're installing InvokeAI but they can't match up with the others that are the right architecture

[kindred grail]

have you ever taken the silicon out of your M1 chip and secretly replaced it with Folgers Crystals

[delicate turret]

the only thing instant coffee is good for, is developing film. and it's not even good at that! those dang crystals

[summer ravine]

I must have at least 3 dozen addons for Blender that all use python

[delicate turret]

yeah...

might be worth right clicking blender, 'get info' and seeing if it's (Universal), (Intel), etc

[summer ravine]

its the arm64 version

[delicate turret]

hmm

well this is a big troubling mystery unfortunately. another thing you could do as you go about your normal stuff on your laptop is open activity monitor and look at your processes and see which if any are running in intel/x86 mode

it's the 'Kind' column under the 'CPU' tab

[summer ravine]

let me try that

[weary atlas]

but was it updated from a version that wasn't?

Could be something got installed before there was arm64 support for it?

[summer ravine]

No.. this computer has always been running arm64 blender

[delicate turret]

it's kinda ironic ... apple made the x86 emulation work so well that people just didn't have to worry about it

[summer ravine]

sorted by kind

[delicate turret]

ah

that's quite a bit

can't say i'm familiar with all of those things but i would not be surprised at all if one of them tucked a lil .so/.dylib somewhere that's x86

[summer ravine]

is it.. mainly whatsapp and drobo driver

okay.. I'm going to uninstall invokeai by deleting the folder.. and try again. Is there a trick to uninstalling too?

[delicate turret]

just deleting the folder is all you need to do

one thing that wouldn't hurt at all: python3.10 -m pip cache purge - what this does is remove any previously-downloaded packages that it might otherwise try to use again for the install

i wouldn't expect pip to be dumb enough to use mixed architectures, but in the effort of being thorough...

[summer ravine]

run this after I delete the folder?

[wispy wind]

That looks good @summer ravine

[summer ravine]

should I delete the x86 versions of python from the frameworks directory too?

[wispy wind]

To clarify, which folder did you delete? ~/invokeai?

[summer ravine]

yes

[wispy wind]

Doing this could break your other stuff

What we need to ensure is that when the installer calls python, it uses the arm64 versions

[summer ravine]

right.. so how do we do that?

[wispy wind]

In the installer script, it will run some commands starting with python. Maybe python3? Not sure - I’m not at a computer

which <whatever python is in the script> tells you which python is using

I’ll be home in a few min and happy to hop on my mac and walk thru this with you

[summer ravine]

Thanks.. I'd appreciate it

[wispy wind]

Np, probably easiest to do a screen share with voice. I’ll dm you shortly if that’s ok with you.

[summer ravine]

sure.. no problem

[wispy wind]

dm sent

[summer ravine]

hmm not seeing it

[wispy wind]

Just to close the loop on @summer ravine 's install - the issue was indeed a x86-64 version of python installed. The install script used that to install and that screwed everything up.

The solution:

• Install brew https://brew.sh/ and add it to PATH per the standard brew install instructions
• Install python with brew install [email protected] (brew install python installed 3.11 which did not work!)
• brew list [email protected] to get the path to python3.10
• Edit install.sh with PYTHON=/path/to/python3.10 (from the previous command)
• Run the installer

At this point, the original x86-64 python has precedence in the PATH environment variable, so the brew installed python should never interfere with anything else. But, if there is a problem, a simple brew uninstall [email protected] will remove the python version we installed, reverting the system's python PATH stuff back to where it was.

[thorn loom]

Is there a way to have a custom H/W in Invoke? I’m looking to create a 2.5:3.5 ratio image, basically a 5x7. Closest I can get is 512x704, but then I still have to crop it. 500x700 would be perfect. Or maybe an auto crop option? That would be nice!

[wispy wind]

Stable diffusion has some limitations of image dimensions. Until recently image dimensions needed to be in 64px increments. There is an upstream change which we will implement in the relatively near future which allows for 8px increments. Unfortunately that only gets you 8px closer to 500x700 but that’s what the tech is capable of.

[raw compass]

1280x1792

640x896

960x1344

Those are 2.5:3.5 and multiples of 64. And yes, I have a program to do that for me.

You can use hires fix on any of those to create a more coherent image, though you'll lose coherence the bigger you get.

[thorn loom]

Kicking myself here, how did I miss 640x896. Thanks

[raw compass]

I recommend starting with 640x896 and going from there.

[raw compass]

You didn't have my program!

[thorn loom]

I actually did 640x960 and was like, hmm, close

The reason for the 5x7 is greeting cards. My daughters sketch them out and then let Invoke do its magic to make them special

[raw compass]

Cute!

[wispy wind]

Would love to see the before and after if you and your daughters are comfortable with that 🙂

[worn river]

There's a new feature in A1111 we'll likely see hitting us as a feature request soon regarding img2img control that i imagine would be good in this use-case

[worn river]

wow, very cool

[thorn loom]

This was very quick and dirty. You guys didn’t get to see the eye rolls when I told them why I needed a quick sketch!

[summer ravine]

Can invoke handle depth?

[worn river]

Not yet!

[raw compass]

™️

[solar barn]

@novel pilot @pastel elm @tired bluff @young steeple @kindred grail I have a significant PR ready for review that touches multiple code owners. It adds a nice console-based front end to the initial model installer and provides the user with the ability to do the following:

1. Add and remove models from the curated list at INITIAL_MODELS.yaml
2. Import HuggingFace diffusers models using their repo_ids
3. Import any checkpoint or safetensors file that has a reachable URL by cutting and pasting the URL
4. Import a local diffusers model using its directory path
5. Import a local checkpoint or safetensors file using its file path
6. Recurse through a local directory and import everything inside that's importable
7. Set a watch on a local directory so that whenever invokeai starts up, any new models found in that directory are imported
8. Do any of the above and convert to a diffusers model

Here is the current screenshot:

I should also mention that it detects inpainting and v2 models automatically and does "the right thing"' in most cases.
To try it out, run invokeai-model-install

[pastel elm]

Oh - did not realize that more than one CodeOwner can be necesarry to review and did not want to approve since I am not the best choice for Python Code Reviews 😅 But now I know better 🙈

Nicely done!!!

[solar barn]

I'm still trying to figure out how it works when a commit touches files that cross multiple code owners. I fear (or expect?) that if there are three files that were modified, each with a different code owner, then all three owners have to review and approve. Ouch, but logically correct.

BTW this PR scares me because of all the internal refactoring I had to do, so please test with a variety of scenarios - bad URLs, safetensors files that have the wrong suffix, no models.yaml file, a broken models.yaml, etc.

Last question is whether I should submit to the temptation to create a frontend for the invokeai-configure script which sets up the root directory, the invokeai.init file, and loads the support models. Right now there are just a couple of prompts that ask for confirmation of the output directory, whether to activate the NSFW checker, and enter the HuggingFace token for concept library downloading.

What we could do is to replace this with a form that has entries for multiple default settings, including the user's preferred sampler, number of steps, directories to automatically load models from, karras threshold, etc.

[raw compass]

I have a PR open that touches code from all over and it requires approval from at least 1 of the 6 code owners to merge.

"at least" meaning... 1? 2? Unknown.

Feel free to use it as a test case.

https://github.com/invoke-ai/InvokeAI/pull/2675

[solar barn]

Well, we'll find out. I'll review your PR today.

[raw compass]

I don't think it'll interfere too much with the nodes work, if at all.

[solar barn]

Nice detective work. For reference, the installer script looks for the following commands on PATH, in this order of priority: python3.10 python3.9 python3 python python3.11. It then calls $PYTHON -V to get the version (irrespective of the command name) and continues the search until it finds a version greater than 3.9.0. Leaving aside the problem that it is allowing unsupported version 3.11 to run, is there any change to the logic that would have caught and prevented @summer ravine 's issue? I'm hoping there is a way to determine which architecture python was built for, compare it with the current architecture, and skip if they don't match.

[delicate turret]

I'm hoping there is a way to determine which architecture python was built for, compare it with the current architecture, and skip if they don't match.
this is a great idea and definitely can be done

[wispy wind]

@solar barn

I tried to figure out checking the arch while on the call but my brief search was fruitless, so I just installed python the way I knew would work.

3.11 arm64 python caused an immediate failure during installation, before pip was able to install anything. IIRC it couldn’t resolve a particular dependency.

I can figure out / clarify both issues later

[delicate turret]

mostly i'd just like to know how this is happening ... the prior two people it's happened to, i think, were because they had brew setup for additional architectures (probably early apple silicon days) and another because they installed the x86 version of an IDE that also installed an x86 version of conda/python/forge/whatever

[wispy wind]

I think this is kinda necessary now that we have all tasted the cursed fruit (see what I did there?)! It’s so much nicer to use the arrows and toggle checkboxes.

[wispy wind]

In this case, python was a dependency of a number of blender plugins. I suspect they are older and installed to x86-64 python, in this case to the frameworks dir in library (sorry don’t have the full path but I think you’ll know where I’m talking about).

[delicate turret]

ah-ha

that explains it! and they did mention blender at some point. thanks!

[summer ravine]

greetings all.. I'm getting this error when Invoke starts up..

[delicate turret]

it's no problem, it just means patchmatch (an optional feature) isn't enabled

[summer ravine]

ok.. thanks

okay.. having a different issue then.. thought the patchmatch eirror was causing it, but apparently not.

when I generate a 512x512 image.. its fine, but if I try 512x1024.. Invoke crashes

[raw compass]

What's the error message?

[delicate turret]

that's probably too big for your available memory

but also on mac i know there's other problems aside from that

[summer ravine]

cant get it to do it again..

isnt there supposed to be a way to save negative prompts in Invoke?

[summer ravine]

Yes.. definitely a memory problem.. and thats fine, I can always reduce the size of the image. But Invoke should warn of that condition and give you a chance to reduce the file size, without crashing out and you lose everything

[delicate turret]

unfortunately it's more of an aspect of pytorch right now than anything ... but it does seem like they are adding some MPS memory management stuff as of late

[summer ravine]

I just added realisticVisionV13 in InvokeAi.. the results are.... unimpressive.. loads fine, no errors..but the generated art....

prompt: photorealistic beautiful rastafarian woman, fit, cinematic lighting, corpuscular rays

out of Automatic1111.. same prompt

I've heard Invoke and Automatic1111 generate different results from the same generation data.. but this is ridiculous

[delicate turret]

yeah something's wrong, but i think you may want to post in another channel at this point, either #🌏invoke-chat or #💡tools-and-tips depending

hm yeah that model is broken. seems people are complaining about it even with other webuis ... probably the usual stuff; they (or a model they merged in) included some stuff that's throwing things off

[weary atlas]

Looks like a broken text encoder. Converting to diffusers usually fixes that.

[hybrid nexus]

after I converted a safetensors/ckpt to diffuser format, I dont get the same image, nevertheless I use the same parameters and the same seed, the image is neither close to the one generated with the original model.

[worn river]

Was there a custom VAE that needs to be brought in?

If you can share models for others to test would be helpful

[hybrid nexus]

Indeed, I did some other test with other models, some are well converted, but some are not, for example this one is not https://civitai.com/models/4825/kalista
FYI, the VAE used with the Kalista.ckpt model was the usal vae-ft-mse-840000-ema-pruned.ckpt), and
And i say Yes to this question, Replace this model's VAE with "stabilityai/sd-vae-ft-mse"? [y/N]: y
If I answer No to the VAE question, then the image I get for the second one (diffuser model) is less saturated/fade]

I have update the issue https://github.com/invoke-ai/InvokeAI/issues/2713

[worn river]

Think you’d need to manually bring over the custom VAE if not using the stock SD VAE

[hybrid nexus]

I use the standard one, and in the optimisation process I answered YES to the question about VAE

[worn river]

And it worked, yes?

[hybrid nexus]

no, I have updated the #2713, with a log file to reproduce the issue...

[solar barn]

stabilityai/sd-vae-ft-mse is supposed to be identical to vae-ft-me-840000... which is why it is swapped in. I've assigned myself to the bug report and will see what's going on. When I've tested conversion, I get identical results.

[hybrid nexus]

IMHO vae is not the cause of this issue ....

[worn river]

So to clarify - if you use the stabilityai/sd-vae-ft-mse as your VAE repo ID, you are still experiencing significantly deteriorated results?

[hybrid nexus]

Yes, IMHO is not a mater of VAE, according to my test in any case. Indeed, I expect that when I convert a .ckpt/.safetensors model into a diffuser model, I get the same result when I use the same prompt, same parameters and same seed... or very close

[worn river]

It would only be relevant if there were a custom VAE or one was not being loaded in the diffusers format.

If the same VAE is being used in both ckpt & diffuser, then I'd agree