Hey, I'm trying to find a way to generate image prompts based on a generated story.
My current plan is to get them into a comma separated list of keywords, mostly in a format to generate images in SDXL.
I currently tried to set up a pipeline in ComfyUI but am kinda stuck on story to image prompt part.
My idea was to use a LLM Node and feed a preset prompt but it's not really very consistent. I'm trying to achieve a 1:1 translation (story to image sequence)
#Suggestions for story to image prompt tools
valid palm
hardy oxide
I've thought about something like this and the best way I can think to achieve this would be to break down a workflow with a local LLM. Do things like:
funct() -> generate prompt from description.
- Isolate the scene
- identify characters who are in the scene
- what is the background -> pass to funct() above
- iterate through each character: what are the characters doing/how do they look like -> pass to funct() above
- Leverage something like comfyUI to stitch the generated images together.
Prompting for something like stable diffusion is has become a very particular language or structure compared to plain english. There are a lot of commas and few-word descriptors. Look for models that can do the general prompt -> SD-prompt conversion if you can.
valid palm
Thanks for the reply!
Hm what would be an example?
What exactly do you mean with the funct() workflow?