opal flume
What is this?
It's a node meant to be used with a text generation model for a prompt. The default is to use a model trained from other prompts; more info here: https://discord.com/channels/1020123559063990373/1093234472402563163
Usage Example:
(inputs are all optional) -> [Nightmare] -> (Prompt output can go to Compel prompt node or something else expecting a str prompt)
no longer latest
It's set by default to use the XL version of my nightmare promptgen model, which is about 1.52gb. It'll auto-download or use from cache. You can also supply a local folder path instead.
If you don't want to use the XL version, simply replace the Repo ID with cactusfriend/nightmare-invokeai-prompts or the repo ID of any textgen model you want!
devout canopy
I'm looking forward to trying this haha
wooden hull
I'm a little nervous to give this a go lol
@opal flume if you're inclined, mind adding it to the community nodes list?
opal flume
not the latest anymore
Was thinking about the upcoming nodes move to their own folders and being able to have a place for support files and whatnot, and decided to do a 'nightmare style' of optional censorship. Since this can be unpredictable, especially if you use my models, this'll help filter out stuff you're sick of seeing or sick of having your psyche damaged by.
For now, this is set in the script itself, by editing inside REPLACE - in the future, it'll be set in a flat file external to the code, for easier editing.
opal flume
latest as of ... still 7/25
updated to absolute imports for upcoming nodes location change, and made the censorship function case-insensitive (whoops). no more escaped orangutans!
opal flume
Are you using 3.1.0rc1 or later? WANT YOUR NIGHTMARES BACK!?
Okay, heard loud and clear. Here you go:
dusty steeple
**Are you using 3.1.0rc1 or later? WANT YOUR NIGHTMARES BACK!?**
Okay, heard lo...
couple suggestions
type is no longer needed in the class body. the new @invocation decorator handles providing the type - that's the first arg to the decorator.
same for the output type - use the output decorator @invocation_output("nightmare_str_output") to specify the type
for logging you can use context.services.logger.info()
opal flume
thanks! i'll get these changes in shortly
opal flume
old 3.1-compatible here, keep scrolling, buddy!
brisk wave
Why not GitHub? 
opal flume
github is for when you can install from a git pull
because i have ideas for a repo that's not just a file
ocean cypress
I guess there is still no way to include the Nightmare output within the image metadata?
opal flume
don't think so. not sure. if there is, i'll gladly update my node!
i wonder if embedding the workflow in the image would do it
ocean cypress
I have that set on two latent2image nodes currently. Like 3.0.2, it only includes the values that have been typed/copied into the fields, not values entered by another node.
brisk wave
It should be possible I think? Stalker indicated that it should be possible, but also said they haven't tried I believe.
#nodes-chat message
dusty steeple
we'll be tackling user-defined metadata relatively soon
the workflow is what is run to generate the outputs - not the outputs themselves
but when we update for user-defined metadata, you'll be able to put whatever you want in there
opal flume
Nightmare Promptgen (3.1 compatible)
opal flume
** 3.1.1 enjoyers rejoice **
TODO:
transformers updates (faster?)Nightmare Prompt Generator (3.1.1 compatible)
brisk wave
Considering looking into how to do this myself for prompts of my preferred style...
opal flume
the post i link to at the top has some example code for training a model on your own prompts, including a jupyter notebook. however, as of invokeai 3 the metadata format will be different so it probably needs some adjusting
brisk wave
Yeah, I was just looking at that.
Also, the HF space is brokeded. ... ed. (An extra ed for extra brokedededness)
opal flume
bleh yeah it happens a lot
the error is literally "huggingface can't connect to huggingface"
great. cool. thanks
brisk wave
LOL That's amazing.
low herald
More fun is the random server timeouts from huggingface when transformers tries to check to see if the repo has been updated. ran into it this morning on one of my nodes that uses hf repo/cache, and saw it before in the nightmare node also.
opal flume
the life and death cycle for my hf space is that somebody will tell me it's not working, then i reboot it, and it works. you'd think hf could automate this, but nah. if something upsets it, it just sits there forever, dead
opal flume
Nightmare Prompt Generator (3.4 compatible)
Here's the repo for the 3.4-compatible version. Currently, no other changes. https://github.com/gogurtenjoyer/nightmare-promptgen
ocean cypress
@opal flume I was just using the previous version last night in 3.4rc2, which worked fine as long as it was in the previous storage location. However, it didn't work as I expected with the queue and cache functionalities. I wanted to use batching to get, say, six generations with random seeds with one nightmare prompt, then move to another prompt with six more seeds. So I added six generations to the queue, which generated with prompt A. I then unchecked "use cache" on the nightmare node and added a single generation to the queue, which used a new prompt B as expected. But checking "use cache" again and adding five more generations to the queue, they generated with prompt A again instead of with prompt B. So the cache wasn't getting updated when the nightmare node fetched a new prompt. I don't know if there is a way to fix this.
opal flume
right, if you use cache, it'll cache that first one
the next version will have use cache disabled by default and maybe some different handling for batching
ocean cypress
Okay, so there is no way to force the cache to update without actually changing one of the settings within the node? This makes it more difficult for me to use.
opal flume
i mean ... 'use cache' is behaving as it should
but it's just not something you want (nor i)
just uncheck 'use cache'
use cache is for the output of the node
for what you want, i'd need to add some batch functionality, which i plan to, now that it exists in invokeai
but it has nothing to do with 'use cache'
opal flume
Latest update: There's now a nightmare.yaml file to configure your models and word replacement settings. Just pull the update from the repo 👍
ocean cypress
**Latest update:** There's now a nightmare.yaml file to configure your models an...
Thanks! Does the replacement only function on whole words (space on both sides)? I'm trying to keep the prompts relatively PG.
opal flume
hm, good question. it originally would do partial matches, so it'd see 'ape' in 'grapes' and change the word into, for example, grboggleurs
but i 'corrected' that so it tries to find full words
ocean cypress
but i 'corrected' that so it tries to find full words
Thanks. If it’s not too difficult, it would be great if the censorship parsing supported some sort of adornment for triggers that could selectively make the censorship “greedy”, i.e. detect the string embedded within a larger word. Many, but not all, English curses are selective (most words with that substring are also profane) while also coming in a multitude of forms that make it a pain to censor individually.
opal flume
Got it ... I've got some ideas about that, that I'll try to get into the next version
near meteor
thanks for this, I integrated into my discord bot 🙂
opal flume
something tells me the 'No' button gets a lot of use, when it comes to nightmares 😄
limpid sparrow
I've had problems running this node while developing as the conf path is resolving to a blank invoke_root and not picking up the nightmare.yaml file from the same path as the nightmare.py file.
as a suggested fix that worked for me, you could change your config path code from this:
INVOKEAI_ROOT = os.getenv('INVOKEAI_ROOT')
conf = OmegaConf.load(f'{INVOKEAI_ROOT}/nodes/nightmare-promptgen/nightmare.yaml')
to this:
CONF_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "nightmare.yaml")
conf = OmegaConf.load(CONF_PATH)
ah, thanks - i nearly did that but for some reason thought it'd resolve to the invokeai root anyhow, haha! i'll add this into the next version
limpid sparrow
oh good, then I can revert to your version and keep up to date with any changes 🙂
opal flume
Okay! I've added both of the above requested changes (better config file loading, optionally replacing strings without the 'word check'):
https://github.com/gogurtenjoyer/nightmare-promptgen
(it occurs to me that i should probably rename the yaml file so it's not replacing a custom one when you update ... 😬 )
opal flume
Nightmare Prompt Generator (3.5 compatible)
opal flume
1.0.2 released
transformers/quotation mark fixes
Fixes:
" marks so that they are passed to Compel intact
opal flume
in case anyone wants to try nightmare node on their gpu instead of cpu (it does seem faster after the initial load):
uncomment lines 10, 36, and 65. comment out 67.
if it doesn't work, reverse the above 🙂 if you're on MPS, top_k cannot go over 16 in hardware mode (last i checked)
low herald
hmm time to dig into some nodes
i have several mountain dew zero's and a 24oz NOS energy drink
this should take care of it
opal flume
update! MPS is fine with top_k over 16 now ... so it seems like hardware mode can be the default for everyone
ocean cypress
I've never bothered to change the parameter values. Do you find it makes a big (or valuable) difference? Do you prefer settings other than the defaults?
opal flume
i've seen some interesting results taking top_k down to the minimum the node allows (5) - i also tried removing the limit and disabling it, which made prompt generation take nearly a minute, and the prompt was huge and low quality, so i'd say 5 is a good minimum still ... i do plan to add some sampling options at some point ... but yeah the defaults included with the node are usually pretty fine.
two alternatives you can try just to see would be:
opal flume
1.1.0 released! - hardware accelerated nightmares going so fast
I've pushed a new version with hardware acceleration enabled by default - it uses the same device as InvokeAI, so it should work for basically everyone. If there are any problems, let me know! Speedup should range from like ... 30-75%
opal flume
1.1.5 exists, and you can use it!
Two related new features:
.and() style prompt when it suspects that it's longer than 77 tokens.NOTE: only enable this if you aren't joining text into your generated prompt externally! For example, you should only enable this if nightmare promptgen is outputting straight to a Compel prompt node! Otherwise, added/joined text could break the conjunction formatting that Compel expects.
opal flume
Nightmare Prompt Generator (3.3 compatible)
Nightmare Prompt Generator (3.6 compatible)
whoops, we're at 1.1.6 now
Small fix to 1.1.5.
opal flume
1.2.0
Added two new settings:
opal flume
1.2.1
just a small fix to allow for empty prompt starters again
opal flume
1.3.0: 'The Sampler Update'
Added two more settings. BIG LAD!
crisp plank
Is there a reason why every once in a while (like every 10-15 pictures) I get an ape as an image? Where is the ape coming from? It just happens when I use this node to generate the prompt. The generated prompt itself doesn't even contain the word "ape/monkey" or something alike. I am confused ...
opal flume
the ape is coming from your nightmares, i'm sad to say
crisp plank
The generated prompt is A photo of a (vivid white rose:1.2), cinematic composition, (exceptional lighting:1.1), chiaroscuro effect, deep contrasts between light and shadow, highlighting delicate petals, intricate texture, symbol of passion and beauty, high-resolution, Canon EOS R5, 1/250s, f/2.8, ISO 100, artistic palette, RAW capture, emotional resonance, visual poetry" and I get this ...
but fr now, why is there an ape? xD
am I missing something?
opal flume
it's the curse of the nightmare promptgen. but seriously, you've got incorrect prompt weighting there
it can cause all sorts of problems, but apes are a new one
crisp plank
well ... I see. So the problem is stuff like (vivid white rose:1.2)?
crisp plank
perfect. What would that be in invoke style? I just got it off civitai x)
opal flume
that link above should give you some ideas
crisp plank
Very good. Wanna see another ape?
opal flume
sure
crisp plank
❤️
Hmm, after changing the prompt I still get apes once in a while ... idk, maybe I am crazy. The generated prompt has nothing to do with apes ... Anyway, it's fine if that's the way it's suppose to be
opal flume
honestly that's confusing. though i will say nightmare promptgen does tend to create apes. but usually, there'd be ape-related words in the prompt
crisp plank
I did 15 pictures and 6 apes. 2 pictutes had apeish stuff in their prompt, 4 didn't have anything to do with apes ...
but that's fine. I can delete the occasional apes. Most pictures turn out good
opal flume
oh and one more thing ... make super sure 'use cache' isn't checked for nightmare promptgen
that's the only thing i could think, where it might keep using an old ape prompt
opal flume
1.3.2 released - Quote Escaper helper node, some fixes
" quotation marks, allowing for them to be passed to Compel properly, and for .and() functionality to work. However, it doesn't do this for the starter text you supply. This node will allow you to escape the quotes here, as well, if needed. Just feed it to nightmare promptgen's prompt input.
opal flume
1.4.0 released - no, it's not InvokeAI 4.0 compatible yet!
adds support for 'instruct/chat' style models: the interface for this may change, but right now here's the steps for using the SuperPrompt T5 model available here: https://huggingface.co/roborovski/superprompt-v1
roborovski/superprompt-v1Example prompt:
Expand the following prompt to add more detail: an orange cartoon cat eating a pizza
You might also want to lower the temperature to something closer to 1.0, and take another look at your min/max tokens, the split prompt feature, etc.
Example output from above:
("a sleek orange cat with big, round eyes sits cross-legged on the floor, munching on juicy slices of pizza. The room is filled with plush cushions and a cozy fireplace,","and the cat's tongue lolls out as it munches on the delicious food. The background is a vibrant yellow, reminiscent of an early morning sun, casting a warm glow on the scene.").and()
more detailed notes: this added support for the text2text-generation pipeline in HF transformers, whereas before we just used text-generation. This is for more 'transformation' type workflows rather than 'autocomplete'. In the future, I'll probably set the pipeline task/type alongside the model, and support more.
opal flume
just got a report that the new model wants a library - in your invokeai venv, pip install sentencepiece should do the trick!
remote sentinel
The addition of prompt works strangely
it often feels like he is writing random words )
I'm writing this:
Tea party with cartoon elephants+++
I get this:
Tea party with cartoon elephants+++ sitting around a table, sipping tea, and nibbling on cucumber sandwiches. cinestill, it picture's no beans) urns of meat with diagram of exploded body, movie movie, by David Lynch,omo burger, photograph,a photograph of a little girl, (elder)+ eating tomato pie, soggy quality, well-lit,sweatsies,friend's (maps),incerity,jongleap
close up, masterpiece, realism
Tell me, maybe I can experiment with some settings?
opal flume
it depends upon the model. if you want something less random, turn temperature down, and turn typical p up
remote sentinel
Yes, it has become much better, thank you)
although some of the texts are quite interesting)
Music festival with cartoon birds+++
Music festival with cartoon birds+++ dancing to live bands and enjoying outdoor concerts. great fun!! many love to go! intricate detailed digital art, best quality, high res, night time, stars!!! delicious, mouth-watering!!! delicious living sauce!!! delicious tasty living!!! delicious sauce!!!!!!1/4hemoth!!! delicious sauce!!!1/4hemoth!!!!!!1/1!!!!!!1/1!!!!!!1/1!!!!!!1/1!!!!!!!!!!!!
I will look for a balance here
opal flume
you can also try other models, like the superprompt one mentioned above - it will try to add details to your existing prompt
remote sentinel
hi. Tell me, is there a way to peek at the current resulting prompt on the node table or in the meta?
opal flume
there's no way to view output in the nodes workflow editor, but it prints its output to the terminal. also, you can use the typical metadata nodes to put the metadata into the image, or something like the addon metadata nodes in #1131656680388706394
limpid sparrow
you can use the Output tab to view what was generated after it has been run.
opal flume
oh yeah, forgot about that 🙂
crisp plank
I am getting this every time I use the Instruct-Mode and the superpromt-v1 model. You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 Is there anything I can do about that? It's kinda annoying in the console. I hate when transformers is spamming you with warnings
opal flume
yeah, sometimes i feel like half of the updates i do to promptgen are silencing transformers warnings
the T5 model support was a new quick thing i did and i expect to enhance it going forward. there's a few problems with it including this, but i'll try to appease transformers so that warning goes away when i update next
crisp plank
Very good :3 Also, we maybe need a 4.0 update anytime soon maybe? I think more and more people are moving on now. At least I did. I fixed it myself though for 4.0 (heresy!) ^^
opal flume
yes, that'll happen when 4.0 releases
opal flume
1.5.0 - InvokeAI 4.0 compatible!
That's it; that's the update. It works in InvokeAI 4.0.1 😄
1.4.0 now exists as a tagged release for people that need 3.x compatibility.
Nightmare Prompt Generator (4.0 compatible)
opal flume
If you're on MPS and InvokeAI 4.0, you may have noticed promptgen isn't working well. This is due to pytorch 2.2; I'm still researching it. (I think this is fixed now but you can always install pytorch 2.3 manually, or a more recent nightly, and you should be good)
regal berry
Is Prompt Generator similar to "Prompt Builder" on the roadmap of InvokeAI? Curious if it's similar to the concept of AI Agent? Which means people could fine tune the local knowledge base and then use it to guide the generation of AI prompts
opal flume
probably the best description of what nightmare prompt generator is, is at the repo's readme: https://github.com/gogurtenjoyer/nightmare-promptgen
regal berry
probably the best description of what nightmare prompt generator is, is at the r...
ok, somewhat understand... need more practical use of it 😋
opal flume
1.6.0 - Support for BRAND NEW Nightmare Promptgen 3 model!!
This is a provisional update to support my new model - some features are not final and/or are missing (for now). You'll need to remove any existing Promptgen node(s) from your workflow, then add this new one. Please also download the new nightmare.default.yaml config file, and/or add the new repo (for NP3) to your existing one.
Notable Additions: Include Starter, and the NP3 model support itself! Also, FP16 loading of models if supported.
Notable removals: Instruct Mode and the SuperPrompt model. They'll be returning in a new release soon. I've also removed the Split Prompt feature because this can be handled much better in the Prompt Tools node set.
Security info: This is the first model of mine that loads and uses remote code. In this case, it's from Apple's OpenELM model, which mine is based upon. It's totally safe and a supported HF feature, but you should consider taking a look at this custom .py code at the new model's repo. It's always a good idea to see what is actually going on.
Also: this model is MUCH more aware (closer to a LLM) and has the potential to make some really detailed nightmares so make liberal use of the censorship/replace feature!
opal flume
NOTE: to use the new model, depending upon a few things, you may get an error about SentencePiece - newer models require this to be installed, and you just need to activate your invokeai venv (or choose dev console from invokeai's launcher) and pip install sentencepiece
opal flume
1.6.3 - support for InvokeAI 5.10 and code cleanup
This removes the dependency on omegaconf, and cleans up some unused code, re-adding some potentially helpful unicode filtering that was mistakenly disabled.
You should update before updating to InvokeAI 5.10!
opal flume
1.6.4 - the one-character update
One tiny change to support python 3.12. You will need to update for sure for InvokeAI 5.10.
opal flume
Model update - Nightmare Promptgen 3
This is a weird update note; I had to update the custom code in nightmare promptgen 3 (the one based upon Apple's OpenELM) to work with the new version of Transformers that InvokeAI uses. It's a small update, but it seems like Huggingface wants to serve/download the entire ~2gb model again. Sorry about that!