Neuro sama drawing stream idea (Not modern AI image generation)

[spark slate]

This topic comes up every now and again of how Neuro should be able to draw. I’ve stumbled on this project google did 6-7 years ago where they taught an AI to draw more traditionally where there’s more of an emphasis on traditional human sketches. Maybe the community could even help training the model? Here’s the link to the article:

https://blog.research.google/2017/04/teaching-machines-to-draw.html?m=1rr

[autumn bridge]

[spark slate]

It would also mean could be able to play this theoretically https://quickdraw.withgoogle.com

[lapis spindle]

She's gonna steal our jobs

[cedar coyote]

I like this because Neuro won't be scraping existing art without consent to mash art together, she'll be making her own style, just like any other artist

[lapis spindle]

Yes, AI art made the right way

[void turtle]

It would be cool if she could "trace over" the art she draws so it actually looks like she is drawing.

[spark slate]

it should mostly be amateur drawings, kind of like how a child draws basically

[autumn bridge]

the true AI art

this is what we need

[lapis spindle]

Using community art as references/tracing

[whole shell]

How much is your job paying you for your scribbles?

[lapis spindle]

Community work...

[void turtle]

From the little I understand of most AI art generators, the images are "formed" over iterations instead of "drawn". Thus, what I am thinking is

1. Have an AI that generates a scribble
2. Have another program that traces over the scribble (over time, not instantaneously) to create the illusion that Neuro is "drawing".

[autumn bridge]

whatever mess of scribbles she draws it will be art to me and I will frame it

[spark slate]

Forgot to mention kind of an important point, from my limited understanding of this project the sketches would all have to be digital and screen recorded showing the entire process of the sketch from start to finish and not a static image

[red island]

Yeah, any way of training an AI to attempt to draw will take recording actions in some manner instead of images. I think it would be cool to layer NN so you would have ones trained to draw different objects and some for picking ideas and composition and providing input for the LLM to respond to, it would be difficult to set up and train, but I think you could do some Bob Ross style landscapes and use the techniques and processes from those videos.

I think Krita would be a decent art program to use since it's free, the plugins for it use Python, and the paint brush tools can do some fairly realistic blending similar to traditional painting so you wouldn't need to try to teach the AI to use different layers and other complicated digital art things.

[red island]

You also need someone to validate the data to make sure it was for the right prompt and doesn't make any inappropriate shapes.

For the actual drawing stream itself (if it can get as much budget as the cave stream), Vedal could do something like VTubers do with the picture-in-picture IRL handcam type stuff so you would have like a shot of a desk and Neuro's arm holding a pencil and her moving it with the strokes on the paper for drawing, or similar for painting with the canvas on the eisel and everything else (swap colors and brushes just enough off camera that you have way less that needs animated).

I think the presentation of it is important to make it quality content so it can last enough streams to make the art training payoff. (and it could be super cute if she started off using crayons both to make horrible sketches and color in coloring book pages)

[spark slate]

Yeah it would be interesting if she showed improvements over time but I don’t expect her to be Picasso or anything

[twin fable]

I'm just gonna state the obvious of how this way we can finally get the mother/daughter art stream we all have been wanting for so long.
Anny and Neuro both start with some prompt, Anny draws as she usually does while Neuro gets the crayon box and just doodles innocently.

[autumn bridge]

https://cdn.discordapp.com/attachments/1127884706831147008/1129055626522275850/image.png?ex=655f607e&is=654ceb7e&hm=cef9847323115cde2e544442e66e086fe6d3d2342c3ee80428400e8f0f9e6a42&

[night wyvern]

then someone decides its a good idea to put a big 3d logo of vshojo in front of them which kills them

[spark slate]

Here’s another link that relates to the post above it has a demo if you want to try it out shouldve posted this sooner https://magenta.tensorflow.org/get-started/

[devout summit]

this would be cool to see.

[autumn bridge]

wait this is actually neat

[thorny tendon]

Anyone suggested Neuro to learn doing pixel art?

Pixels should be easier... Right? I'm

[autumn bridge]

and then she could draw vedal

[thorny tendon]

Oh god

[viral karma]

The main issue is its limited to specific categories. It can draw mosquito but not mosquito on a table.

[autumn bridge]

Yeah, it only has the sets of Quick, Draw

[woven grove]

If Neuro could draw/create images of the things she speaks about when telling bedtime stories or just random stories would be fun

[sweet gazelle]

[spark slate]

Yea it looks like it’s limited in what it can do but I think it’s fine if the goal is to just make neuro draw something

[sullen creek]

the

[gusty kernel]

Lets revive this

[autumn bridge]

yes, bump

[void turtle]

[autumn bridge]

🖼️ 🖌️

[sullen creek]

bump

[spark slate]

You don’t have to keep bumping it maybe do it when it becomes a topic of discussion again

[sullen creek]

pineapple

[strange linden]

So, it has been argued that AI can never come up with true art, because ai images are derivative. I was thinking about how ai comes with completely new and original solutions by altering its reward functions. https://twitter.com/DrJimFan/status/1715399011094679779 So I would say neuro learns the fundamentals of art like construction, composition, basic elements and learns how to put it together. One of the problems I have with AI images is that it is separated from personal experience. The "experience" of one entity, neuro, can be used to form an artistic identity that is not just ripping off someones style. Her memory and language model could be used to develop the topics of her arts and guide the style in which she develops as an artist, so that the subjective nature of art and the creative struggle can be reintroduced.

Jim Fan (@DrJimFan) on X

Eureka achieves human-level reward design by evolving reward functions in-context. There are 3 key components:

1. Simulator environment code as context jumpstarts the initial "seed" reward function.
2. Massively parallel RL on GPUs enables rapid evaluation of lots of reward…

Twitter · Oct 20, 2023, 4:06 PM

[grave maple]

I think focusing not on an optimal result but on associating parts to "emotions" or "objectives" that can combine in to something beyond what it knows is also an important way it could be taken

Since going for the process as designated by what Neuro feels or thinks about any part would make it unique to her

[spark slate]

Bump

[ivory sequoia]

I'd like to see her drawing the drawings in such a stream. Similar to how a pen or stylus creates a line over canvas instead of printing/generating. Which I think is what this forum is going for.

[median horizon]

This is awesome 😮
But looks like a ton of work for Vedal.
Hopefully after other things that are mid way get finished, this can be done 😄

[spark slate]

He doesn’t have to do it this way it’s more like suggesting an alternative, if people know any other ways they can post it in this thread

[median horizon]

Yeah, regardless of the method, i think its a good idea 😄

[vernal elm]

Going to link my post here too since it's a similar idea https://discord.com/channels/574720535888396288/1221083526754209914

[weary junco]

how AI art SHOULD be tbh

[shy sky]

Would love to see neuro hooked up to ms paint or something

[echo trout]

I'd love to see her do art like this

[autumn bridge]

friendly bump

[median horizon]

Wooo!

[grave maple]

Bump bump

[sullen creek]

surely she can draw me

[spark slate]

what even are you

[sullen creek]

three circles, three ovals, and a line for a mouth

[autumn bridge]

if she draws a circle I will be in awe

[sullen creek]

just saying if she draws anything that even remotely looks like me I will keep it as my pfp until the day I die

[autumn bridge]

#1174048649567801355 message

[last vault]

Perhaps a more feasible task than drawing is getting Neuro to write her own name with a brush tool
Then you can have Neuro sign things when doing cons

[echo trout]

yooo that'd be incredible

[last vault]

Or something that can fill in a colouring book

[strange linden]

this is what i want. observational drawing rather than relying on data sets (or rather, neuro makes her own data set through drawing experience). https://www.sciencedirect.com/science/article/abs/pii/S0097849313000149?via%3Dihub

https://www.youtube.com/watch?v=bbdQbyff_Sk

she can learn how to draw in different styles and different things too. the good old fashioned way

[twin fable]

Pog Anny training data inbound.

[strange linden]

idk if anny wants neuro to learn to draw cute things, but anny should know the best methods to properly learn to draw so maybe she has some ideas or insight, maybe she can walk neuro through that process like teaching a real beginner to draw and giving neuro exercises to do