AVS & avsutil | ASCII art video format / playback tool

[mild surge]

Earlier this week an idea popped into my head: what if there was a fully self-contained file format for storing ASCII art videos?

well, ladies and gents, i present:
ASCII Video Stream (.avs)
ASCII video stream (which is shortened to AVS) is a portable file format that contains a video made with ASCII art. It defines the dimensions, characters, and colors of each frame, and has (relatively) easy-to-understand header bytes.

as a format, AVS is capable of:

• monochrome, 8-bit, and 24-bit color playback (TTY, ANSI, and RGB, respectively)
• auto color remapping (enabling AVS files with higher color depths to be played on devices that can't show the specific colors)
• framerates ranging from 0-127 FPS
• a maximum resolution of 65535x65535 character cells (which is definitely overkill)
• direct conversion of common video formats such as MP4 to AVS via the command-line tool avsutil
• total platform agnosticity, such that AVS files can be played back on a webpage with minimal unpacking and display code
• high compression ratios with simple algorithms such as run-length encoding (not implemented yet)

besides being a format spec, AVS also has its own command-line tool that I wrote: avsutil

avsutil is a simple CLI utility that can be used to do two things (at the moment): encode a video file as AVS, and play back an existing AVS file.

when encoding, avsutil supports several user-configurable options (with more to be added) such as output frame width, exposure scaling, and custom framerate.

during decoding (playback, essentially), avsutil will sequentially display each frame found in the AVS file, much in the same way as any other video file. but this one's in your terminal!!

AVS and avsutil, coming soon to an Arch User Repository near you!

i would like to note that avsutil is much faster than it appears in the video; OBS was throttling my computer hard enough that it slowed down frame rendering significantly and caused terminal flicker during playback. neither of these problems are normally present.

[manic monolith]

Love the idea

[wary idol]

@mild surge can you explain a bit how it works looks good

can you make a recoriding of span and run it thruge this i wuld look funny

[mild surge]

what would you like to know

[wary idol]

like does it combine chunches of pixels for color or how does this work same for the ascci art

[mild surge]

dynamic pixel average

it splits the image into a greyscale luminance component and a full color component and processes them separately

the pixels are separated into blocks

the size of the blocks is determined by the downscaling factor of the image and the input size of the image

then each block's values are averaged together, which determines the character or color of the pixel

its an algorithm with very good balance between output quality and rendering speed

i could have theoretically gotten better resolution with fancy edge detection algorithms but those render slow as shit

[wary idol]

so the colour and ascci art are 2 separe things overlayed

[mild surge]

the characters and the colors are stored separately and recombined at runtime, yes

[wary idol]

does the downscaling have a max res for the output and input

[mild surge]

the limit is entirely dependent on the update speed of your terminal, the minimum font size of your terminal, and the processing power of your machine

the power of the machine itself doesn't really matter for playback but it matters for processing a video into avs

[wary idol]

ok

[mild surge]

playback is more dependent on the terminal

[wary idol]

makes sense

you shuld add a option to convert it to mp4 to easly shere and viev acros devices that doesent have this

[mild surge]

um

that would defeat the whole point

[wary idol]

ig

[mild surge]

i need to write the compression algorithm though because high res avs files are BEEFY

[wary idol]

yea do they store the raw text per frame

[mild surge]

yes it has to

[wary idol]

aprox for 20sec how large are thre

[mild surge]

but since most backgrounds are made of the same characters i can get awesome compression out of simple run-length encoding n shi like that

[mild surge]

that's resolution dependent

[wary idol]

yea

[mild surge]

low res avs files arent very large at all

especially when using 8-bit ANSI color or monochrome

[wary idol]

like how large was that demo clip

[mild surge]

lemme check

[wary idol]

does it use gpu or cpu mostly

[mild surge]

~20sec clip at 100x28 60fps with 24-bit fullcolor RGB is 6.4mb

without any compression at all

[mild surge]

during rendering or playback

[wary idol]

also for compression you shuld be cheching for the same frame twice in a row or what changes from frame to frame

[mild surge]

its not gpu accelerated if thats what you're asking

[wary idol]

ok

[wary idol]

idk if this is a good way

[mild surge]

the only significant cpu usage is during encoding when converting a video to avs and during decoding when caching the frames into ram

[mild surge]

difference checking is my fallback if RLE doesnt give good results

[wary idol]

ok

[mild surge]

also most of the space is taken up by the color matrix especially in 24-bit mode because the color matrix is always 3x the size of the frame itself

[wary idol]

you shuld add live decoding while watching

also does fileformat include audio

[mild surge]

though my solution to that is just going to be caching all the colors in the video and then replacing the bulky per-pixel rgb definition with some predefined shorthand

[wary idol]

what do u meean by predefined shorthand

[mild surge]

no but i plan on building it into avsutil if i can get the compression good enough

[wary idol]

if u cant you shuld just add like .mp3 with evey file by deafoult

or make it an option

[mild surge]

posterizing the colors, counting which ones are used, then storing those colors so that they can be accessed with a much smaller variable call

[wary idol]

ok

the only thing u got to add is audio and compreation

and some orther stuff like gpu accalertaion witch not easy

[mild surge]

https://github.com/3xionDev/avs out on github

no aur yet because the binary is obscenely large

i have to exclude all the unnecessary modules and then rebuild it

-# cough cough numpy

[viscid crown]

really cool project

think maybe you can support unicode too?

[mild surge]

so currently python's ord() function doesnt support unicode

also i want to stick to 1 byte per char and unicode is variable width which i dont like the prospect of dealing with

but i have been toying with the idea of just replacing some of the ascii escape characters like \b and \t with a couple unicode characters and then just remapping them during runtime

[viscid crown]

Here’s an idea

Define a charset in header

And just reference that inside the actual content

[wary idol]

add a readme @mild surge and what fileformats of vidio are supported

and can i get help with ./avsutil -b 50 -r 60 -o test.avs enc /home/duck/Downloads/sample-5s.mp4
Traceback (most recent call last):
File "avsutil.py", line 301, in <module>
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
[PYI-8355:ERROR] Failed to execute script 'avsutil' due to unhandled exception!

[mild surge]

./avsutil enc -w 50 -b 2 -r 60 -o test.avs ~/Downloads/sample-5s.mp4

idk why youre trying to set the brightness to 50 but it doesnt like that you supplied no argument for width

[mild surge]

header must stay minimal due to autism

i want most of the actual playback logic to be written into avsutil instead of the file

which means charset stays defined in avsutil.py

however i do plan on running some tests with my ascii escape code replacement strategy this week and if it works then a select few unicode chars will be supported and you will (at least halfway) get your wish

[mild surge]

i wrote the most beautiful readme of all time yesterday and then accidentally wiped it out of existence by making a git mistake

so the readme will be replaced with github wiki, which i can't accidentally vaporize with one command

ill lyk when its done