#Bad Apple!! but it's implemented using KubeJS' Painter API

62 messages · Page 1 of 1 (latest)

fathom tendon
stray obsidian
#

WHAT

random yacht
#

there's no fucking way

west shell
#

i thought i saw it all...

ashen hearth
fathom tendon
# fathom tendon Yep title <:heh:504730156792021002> The video was sped up a bit and the audio wa...

Ok hello yes I’m back time for some promised

✨Technical detail ✨

Basic layout

The way this works is simple:

  1. Download the bad apple video from youtube
  2. Use a python script to split the video into frames, convert them to pure black-or-white, and encode them
  3. The encoding works by tracking changes for each pixel between the current frame and the previous, and then adding its x/y position to an array. So if a pixel was white a frame ago and now it’s black, it will get added; if it was white and stays white, nothing happens. Since the video only contains 2 colors this information is enough to then reconstruct the video in kjs
  4. Encode all frames and add them to a single array, then dump that array into a json file (then realize kjs jsonio has a bug which only allows it to read map objects from json files, so then instead of writing the array directly first add it to a dict and dump that instead)
  5. Read the encoded video json from a kjs startup script and write the result to global (this is done so that the game wouldn’t load the file each time a /reload occurs, causing an OutOfMemory error)
  6. Draw multiple black rectangles using painter, each one representing a single pixel of the video. Name them "{x position of the pixel}x{y position of the pixel}". Get the size of the client’s screen by sending a packet to the player and getting a response, and then based on that info scale the "pixels" accordingly
  7. Loop through every single frame and flip the color of the encoded pixel positions. Make sure to wait a bit before rendering each frame to make the final sequence not go apeshit fast on the screen
  8. Attach the activation of this animation to a custom command. Run that command in game, record the video, edit it a bit and profit! You’re done
#

Technical limitations

As said previously, this type of encoding (btw idk if it has a name I came up with it myself lmao) only allows for purely 2-colored animations, although you could probably do some jank shit like attaching the hex/palette representation of a pixel’s new color as the third element in each encoded pixel’s array or do some elaborate stuff with pixel regions and shit, but you can probably tell I cba heh
When I originally tried this project a couple of months ago I tried to use RLE as an encoding until I realized it’s very inefficient and that I had to implement it in JS didn’t help things heh. The new encoding does trade the freedom RLE can give you for its speed but since this is bad apple we’re talking about this doesn’t really matter.

A sane human being might also say that painter can draw textures, and that I should’ve instead just loaded the split frames directly into the game and drawn those instead. That would be more reasonable but I think by now you should’ve already guessed that adjective doesn’t really apply to me. Ig this way you can also display video with color but shut up idc.

#

As also said previously I had to speed up the video in post by about ~158%. This is because you can’t really do anything in mc sub-frame and there is only 20 frames in a second, whereas my original video is 30 fps. There are only 2 solutions to this problem: I could either render all of the frames sequentially (as mentioned before, the speed would go apeshit) or do what I did and render them a bit slower then in the actual video. As you might’ve also noticed the reference video and the actual in-game footage are not synced sometimes, this is partially due to this and also because it takes a bit of time to actually render the frame, not a lot but enough to cause desync. Audio was also added in editing, you can probably guess why. raha

The only other technical limitation I encountered is davinci resolve on linux go fuck yourself black magic and you shitty fucking encoding support (respectfully). The final .mov file of the video you’re seeing weighed 7 FUCKING GIGABYTES. After I converted it back to an .mp4 it only weighs 33 megs. (ty apple as well love your uncompressed formats)

The bug

One of the most critical bugs I’ve encountered during this is the final animation only showing the outlines for the pixels that have changed. I’ve spend a good 2 hours trying to figure out what went wrong and finally figured out that the problem was in encodings themselves (and not because of painter weirdness).
Turns out in my generation script, I had a for loop that looped over all of the frames except for the first one which was an artificially added purely black frame (this was done so that the first frame (which is completely black in bad apple btw) when encoded produced an empty array), and to check the pixel in the previous frame, I got that frame by getting the element indexed one below the current one in the frames array.

#

Because of that one frame being missing from the array I was looping over though, I was actually getting the frame 2 frames back, which lead to these unexpected results and wasted precious hours of my life.
Moral of the story: me dumb

#

Stuff used

  • Video (downloaded with yt-dlp, 144p, 30fps, no audio (downloaded separately when I added it in post))
  • The python script (will send it tmrw, on mobile rn and the script is locally on my pc)
  • The final encoding (only weighs about 67MB, which is a far cry from 110MB I was getting before I found "the bug")
  • All of the kjs scripts are available on the package repo, check it out (or don’t)

Special thanks

I’d like to thank all of the boosters for boosting this server which allowed me to upload the video directly to here instead of youtube. I also would like to thank my mom AND dad obviously. I would NOT like to thank davinci resolve

Sorry if this is a bit cluttered it’s almost 3am and I want to eep

Ok thank you for reading (or skipping to the bottom) bye!!!!11

random yacht
#

thanks for calling me out for skipping to the bottom raha

fathom tendon
#

Knew it

#

Uh so yeah last time I checked this is peak painter you can’t do much more with the api so uhhh yeah

#

This is gonna revolutionize modded minecraft frfr

#

@half walrus I didn’t see what you deleted

half walrus
#

just a larger version of this raha

fathom tendon
#

Lmao

fathom tendon
#

@covert gazelle your counters fucking broke btw

IMG_5347.png
covert gazelle
#

oopsies

haughty elm
#

i love it

covert gazelle
#

wait it works for other packages

#

so the runner is at fault ig

#

ah ik what happened

#

it checks if the package exists in the db and if it doesn't, ignores the request

#

cba to fix it rn doe

ocean elbow
#

absolute legend

turbid basalt
slender cosmos
#

that's amazing

#

but also

#

why

covert gazelle
#

cutscenejs

stray obsidian
fathom tendon
#

because if it exists, it can play bad apple
and I won't sleep until that has been proven heh

#

uh yeah join me next time when I run doom on painter

fathom tendon
brazen yokeBOT
#

Paste version of gen.py from @fathom tendon

fathom tendon
#

requires pillow and opencv-python

stray obsidian
fathom tendon
#

I mean it's technically possible

#

I think I'm just not qualified enough to do smth like this

covert gazelle
#

do a c-to-kjs thingy

#

then put doom into it

slender cosmos
covert gazelle
#

for guijs or whatever u were gonna call it?

#

oh yea talking about that.. any progress?

slender cosmos
#

not currently, im working on ichor

covert gazelle
#

fair

fathom tendon
#

yeah

#

dunno how good it is pepelaugh

slender cosmos
#

im.. going to guess not as good as mine pepeLaugh

random yacht
#

it's very primitive

#

just as painter is

loud void
humble orbit
cold badger
#

WTH? Now I've seen everything

bright osprey
stray obsidian
#

oh and also lat

#

don't forget about lat