Better/Different sound component

[quick current]

Duplicate of a few others suggestions. Would be nice if we can get a few concrete suggestions

"work like in real life" would basically require analogue inputs, which is not really a good option

[versed ore]

Make cool stuff better /j

[nimble spire]

I suspect the only "real life" equivalent function it could use would be something like MIDI - something that can be preprogrammed and sent rather than having to control the timing.. I don't think the tick mechanism would really allow sufficient audio control to sound good..

either that or a "here's an mp3 go play it" type component but that's boring 😛

more in line with programming tools but not really hardware level matching

[north drum]

Add the functionality of importing multiple sound files at once, it'll make it even better

[hot plume]

It might be interesting to have a component connected to a RAM block or something and you can use that to play raw audio

Or some way to play audio defined during the simulation

[nimble spire]

I suspect raw audio would be a problem because the sim runs at different speeds for everyone based on their (real) device specs (never mind if someone tries to step through).. sound is a very temporal experience so that kind of speed variation would be troublesome.. that's why I think it would need to be some sort of programmable system (a "sound card" that can process and play the data asynchronously apart from the in-game driving circuit) rather than raw audio. MIDI is just a well-known protocol to drive such a system but its not the only way it could be done

[quick current]

raw audio via a RAM block would work, it would basically be a looping buffer that get's played back, while the buffer get's overriden in an async matter. This is actually a pattern that can be used in higher level languages as well, I have successfully used it in python to play basic sounds (however, using numpy for math, so this would assume the compiler can compile the circuit fast enough)

[nimble spire]

there's two concerns: first, the "fast enough" but secondly there's synchronization issues ("overridden in an async manner").. the circuit would need a way to know where the play is in the buffer but because the playing is async that progression could be jittery, especially on lower end machines.. it would work for "here's a fixed buffer play it until I say stop" but that's in a sense what I'm suggesting already (just with a much simpler "protocol" than MIDI).. I mean whatever works is great, I just have some concerns about raw audio being able to work (at least with any sort of reasonable fidelity.. I'm not expecting full symphonic quality but I'd like it to sound better than an array of PC speakers)

[quick current]

I am not that much of a fan of having a system as complex and powerful as MIDI builtin. I would prefer much simpler interfaces

raw audio can definetely work, especially if you don't try to modify the buffer that is currently being played

[nimble spire]

well yeah MIDI was just an example because its well known.. but some sorted of coded way to do for example the music maker from MarioPaint (if you remember all the way back to the SNES days lol)

that might be cool

[quick current]

The only problem with raw audio is that it's uncompressed and a lot of data, i.e. 44kHz, double channel, 16bit samples are a lot to write out

[nimble spire]

well yes, that's the problem I'm referring to 😛

I know raw audio works as a concept.. the question is whether the game engine can process fast enough (and consistently enough) for it work in-game

[quick current]

By using a RAM (or better, a WOM) and playback being async, consistency is not a problem if you are a bit careful

[nimble spire]

so.. "here's a .mp3 go play it" but with an extra step of copying from a ROM loader to a RAM block? :P.. or I guess a .pcm file as itd need to be already raw

[quick current]

You can have two buffers and generate stuff in one of them while the other is being played

[nimble spire]

that's the synchronization part.. how do you direct it to switch buffers at exactly the right time? a 1-tick delay even running at 1Mhz could potentially be noticeable

[quick current]

The sound component only reads in the next task when the current one is finished (or one tick earlier). As long as it's the same one, it loops

[nimble spire]

anyway we're just arguing in circles at this point.. Stuffe will make what he makes whenever he gets back to the sound component and we'll get to find out how well it works when we see it

[quick current]

Stuffe has asked multiple times for suggestion on how the sound components should work

[nimble spire]

and I've made my suggestion.. beating our heads against each other without changing our respective positions won't help him much 😄

[quick current]

Hm. If you consider that a concrete suggestion, ok

[nimble spire]

I don't know what more you'd want from it. I disagree (or at least have concerns) with doing raw audio, I made a suggestion that we don't do raw audio.. no I don't have a full design document laying out the entire structure because I haven't thought about it that deeply and probably won't do so anytime soon.

[quick current]

I would actually suggest at least two or three sound components: one with raw audio, one somewhat similar to the current one with predefined notes, one with some kind of simplified file format, one where you can define small looping bits of samples accross multiple channels (with control of raw audio) similar to the old console era around NES and SNES

[nimble spire]

hah well sure "just do everything" is a fine option I hadn't considered 🤣

[quick current]

We do the same with display components already. If we at some point get programmable components/shader components, we can definitely have at least an internal interface for raw audio and define small custom formats for use by the CPU we build