Dynamic Music System Help

[runic crater]

Hello! So, I'm trying to make a dynamic music system. I currently have my music being held in a audioQueue list. I'm what I'm attempting to achieve is:

• Have my music play, one clip after another. (1, 2, 3, 4...)
• Have a fall back once the music clips played reach a specific song. This will repeat the current song playing if its position in the queue is the same as the measure where it should stop. This is defined in inputMusic.Measure. The songs playing would look something like this: (1, 2, 3, 3, 3, 3).
• If the inputMusic.Measure variable is changed, it should finish playing that song, and then continue to play the songs with the pattern mentioned previously until it reaches the next instance of inputMusic.Measure.
  (I apologize for the poor description. Please ask questions and I will clarify if you are confused)

Currently, I am able to run this code and it nearly functions as intended. The audio clips play as follows: (0, 1, 2, 3, 2, 2, 2...) My issue: When it is supposed to loop on the 3rd clip, it returns back to the second clip instead. Any help or input would be appreciated, as well as any input on how I can improve.

Here is my code:
https://pastebin.com/Az1Vj3T7

As an update, I found out my issue is the fact that I limit the running of my code if it passes an if statement:

if (clipsPlaying.Count < 1)

What can I do to work around this or even what can I redo to make this function without removing things from my audioQueue? This line of code is important to making sure multiple clips dont play at once unless intended to, however I dont know what I can do to fix this...

Does anyone have any ideas?

[tame yoke]

It would probably help to see the code for SoundFXHandler and MusicHandler

however, generally, you probably want to decouple the control of the music and the music itself

basically, you should track the data for which track should be playing with its own separate variable(s), and then control the music according to that data

this would eliminate the need for keeping track of three separate List<>s

at least in theory

I'm still not 100% sure what your code does, as I don't know what those other two classes do

[runic crater]

Yeah I understand however they both function as intended. I’ll upload these tomorrow morning when I have access to my computer again. My apologies

[runic crater]

As a little explanation right now. The music is held in a scriptable object’s property drawer. The read music method returns the audio clip of a specified index of the drawer. The getsound2draw method creates an audio source and queue’s it into my music handler.

What do you mean? As further explanation, (I forgot to explain this) I have a list for music playing because I am going to have multiple clips play at the same time. This may only be for milliseconds, however, it can help with the transition of my songs. I haven’t set it up yet. Although, I’m not opposed to removing the use of lists if you could aid me in figuring out how to achieve that haha

[runic crater]

As a little bit more information on this. The amount of clips are undefined, meaning it could be anywhere from 3-15+ (although it’s not likely we’ll have that many)

[runic crater]

public class MusicHandler : ScriptableObject
{
    [SerializeField] private bool _advancedProperties = false;
    public bool AdvancedProperties => _advancedProperties;
    [SerializeField] private int _mainBpm;
    public int MainBPM => _mainBpm;
    [Range(0, 100)]
    [SerializeField] private int _volume = 50;
    public int Volume => _volume;


    [Header("Sections")]
    [SerializeField] private MusicSection[] _sections;
    public MusicSection[] Sections => _sections;

    [SerializeField] private int _measure = 0;
    public int Measure => _measure;
}

public class MusicSection
{
    [SerializeField] private MusicPhaseType _phaseType = MusicPhaseType.Section;
    [SerializeField] private AudioClip _audio;
    [SerializeField] private int _bpm;
    [SerializeField] private float _ms;

    public AudioClip ReadAudio()
    {
        return _audio;
    }

    public float getDelay()
    {
        return _ms;
    }
}

SoundFXHandler: https://pastebin.com/EBeMPGfz

[tame yoke]

ok, a couple questions

1. is this intended to support layering tracks, or just sequentially played tracks
2. is this working as intended at the moment? I suppose I'm confused about what I'm supposed to be helping with... (however I generally dislike this approach)

[runic crater]

1. This was set up to support multiple tracks playing at once, that being for a couple of milliseconds. This can make my transition sound smoother, at times there are small gaps at the end of the audio clips.
2. This is nearly working. My main issue is the if (clipsPlaying.Count < 1) line, which limits my code to only run when nothing is playing. This prevents all of my clips in the queue from being played all at once. This is an issue because, at times, I need two clips to play at once. I'm just looking for assistance on fixing the logic for this, as well as understanding what I can change / work on to improve.

As further clarification, I'm not asking for you to rewrite anything, I just require help past this minor inconvienience I've been stuck on for a while now.

[tame yoke]

sorry, I was out in the wilderness for a couple of days, but I'm available to help now

you may want to look into controlling your queue with coroutines, that way each track in the queue can be handled in parallel

[runic crater]

hey no worries

[runic crater]

What do you mean? Sorry, I require further elaboration, as I am not as experienced with coroutines and lack the knowledge to see how this may bee helpful

[cunning bough]

what are you trying to achieve

i understand the music queue part, but i don't get the fallback/default clip and how you're controlling when to fallback

[tame yoke]

so basically, using a you could start a coroutine for every track in the queue that is played

so something like

IEnumerator PlayTrack(AudioClip track)
{
  // Instantiate a track object

  // Wait for the length of the track

  // Terminate track
}

[runic crater]

Currently I am trying to have somewhat of a dynamic music system.

Basically, I have a song which is split up into clips. Each clip it created to both flow into the next clip or loop. I am trying to control the current section of the song through a set number. This is currently stored in my scriptable object, accessed under InputMusic.Measure. Each song clip is stored in the InputMusic scriptable object's property draw (with the full scriptable object being one full song). Each song clip, if its less than the current measure should play until completion, and remove itself after. If there is another song clip in the queue, it should play, starting at an "inverted delay" or a set time from the end of the previous clip. For example, clip 1 plays. Clip 2 would play when clip 1's time is >= clip1's length - clip2's delay. If the current measure is equal to the clip that is playing currently's index in the inputmusic.sections property drawer, then loop the music until the inputmusic.measure is changed to a higher number, in which the clip should play through fully and progress as normal.

Does this make sense?

[runic crater]

I'll look into this more, it could work

[cunning bough]

Like songs loop to fill up a certain time before moving to the next song?

[runic crater]

Just having the song repeat until the measure changes. For example, a player is in room 1 and the room 1 section of the song plays, but then the player enters room 2, and the room 2 section plays. If they each were say, 5-10 seconds long, it would need to loop (indefinetly until told otherwise)

Does this make more sense?

Also this example is irrelevant to what I am trying to achieve

[cunning bough]

I'm confused what measure means

[runic crater]

measure is just the name of a variable I have. Basically, the measure is the clip that is intended to play. If the measure was 4, clips 1, 2, 3 would play, following after 4 would play and loop until its changed.

I hope I did better at clarifying?

[cunning bough]

Oh yes that makes more sense

I would rename the variable ClipIndexToLoop or something like that

[runic crater]

Yeah, that would probably make more sense

Do you have any suggestions on how to achieve the outcome mentioned earlier?

[runic crater]

Hello, although this would be useful, my main issue is handling the input of the music, specifically this section of code:

            if (clipsPlaying.Count < 1) // Checks if clips playing list is empty
                foreach (AudioSource source in audioQueue)
                    if (clipsPlayed.IndexOf(source) < 0)
                    {
                        Debug.Log("Breaking at " + audioQueue.IndexOf(source)); // Outputs the clip that is needed to be played
                        audioQueue.Remove(source);
                        clipsPlaying.Add(source);
                        break;
                    }
                    else
                    {
                        Debug.Log("Continuing...");
                        continue; // potentially not needed
                    }
            else
                if (clipsPlaying[0].time >= (clipsPlaying[0].clip.length - 0.035f))
                    Debug.Log("Attempting to add another clip");

This limits my ability to add the audio into the playing list based on how many are currently playing. My issue is the fact that this does not allow me to play multiple clips at once based on my logic. This logic however, prevents the clips playing array from flooding with a ton of clips and playing every part of the song at the same time, or even multiple times infinitely.

[runic crater]

hey sorry, new to coroutines. How can I run multiple instances of a coroutine per object?

[tame yoke]

everytime that you run StartCoroutine() it creates an instance of one

you just have to call that mutliple times

if you do not call StartCoroutine(), the method will run as a normal function

[runic crater]

oh alright

cool, thanks, sorry

[runic crater]

Okay so I've implemented this, and its NEARLY nearly working. Here's my code

    public void playUpTo(int measure)
    {
        foreach (AudioSource audio in clipsPlaying)
            if(!audio.isPlaying)
            {
                audio.Play();
                StartCoroutine(destroyAfterTime(audio, measure));
            }

        if (audioQueue.Count == 0)
            PrepSong();

        if (clipsPlayed.Count == 0 && clipsPlaying.Count == 0)
            clipsPlaying.Add(audioQueue[0]);
    }

    IEnumerator destroyAfterTime(AudioSource _source, int measure)
    {
        float transitionTime = 0f;
        float repeatTime = 0f;
        float waitTime = 0f;

        foreach (MusicSection section in inputMusic.Sections)
        {
            if (audioQueue.IndexOf(_source) + 1 < audioQueue.Count)
                if (section.ReadAudio() == audioQueue[audioQueue.IndexOf(_source) + 1])
                    transitionTime = section.getTransDelay();
            else
                if (section.ReadAudio() == audioQueue[audioQueue.IndexOf(_source)])
                    repeatTime = section.getTransDelay();
        }

        waitTime = _source.clip.length - 0.035f;

        if (audioQueue.IndexOf(clipsPlaying[0]) < measure)
        {
            yield return new WaitForSeconds(waitTime);

            if (audioQueue.IndexOf(clipsPlaying[0]) < inputMusic.Sections.Length - 1)
            {
                if (audioQueue.IndexOf(_source) + 1 < audioQueue.Count)
                    clipsPlaying.Add(audioQueue[audioQueue.IndexOf(_source) + 1]);

                yield return new WaitForSeconds(transitionTime);
            }
            else
            {
                Debug.Log("playing last clip");
            }
        }
        else
        {
            clipsPlaying.Add(audioQueue[audioQueue.IndexOf(_source)]);
            yield return new WaitForSeconds(repeatTime);
        }

        clipsPlaying.Remove(_source);
            clipsPlayed.Add(_source);

    }

Im attempting to attach my issue because for some reason it just changed

I'm going to try something first

neverminded, no idea why it did that but I'm getting the same error / 2 errors:
InvalidOperationException: Collection was modified; enumeration operation may not execute.
SerializedObjectNotCreatableException: Object at index 0 is null (It doesnt tell me where this issue is happening)

[tame yoke]

InvalidOperationException occurs because you are changing the collection that you are looping through in that foreach loop

[runic crater]

Why might this be though? I dont believe that I am modifying this list at all?

oh man i may just be stupid

What would you recommend to get around this?

[tame yoke]

man, this really seems over complicated

I don't understand why your queue is even considered within destroyAfterTime()

[runic crater]

Yeah. I tend to do that. 😅

[tame yoke]

you seemed to have just moved most of your code into that coroutine
I suggested it only handle individual tracks

[runic crater]

Yeah im pretty stupid in general. I just thought that since it was already getting the time of which a clip was getting destroyed, I could subtract the transition time from that and then add the new clip in on that. I also wasn't sure of how to do this without it being in the function, since I was already getting a direct reference to the current source playing.

[tame yoke]

no, no, you're good

I haven't explained what I'm trying to describe well enough

[runic crater]

Thank you.

How can I improve my code that I have currently as well as fixing this issue?

I know I should take out the logic which adds a new clip, so I'll do that first

[tame yoke]

I feel like I've said this, but I think you should take a whole different approach to your control loop

right now, you're taking a very functional approach, where you don't globally store the important information about the current state, and I think that is making things inherently messy

[runic crater]

Do you know if it would happen to return this first error when I remove my _source clip from the clipsPlaying list?

[tame yoke]

additionally, I suggesting how you should split the control algorithim from the bits that actually control music playing and stopping (hence the coroutines)

[tame yoke]

which line is this occuring on?

there's two foreach loops, so I'm unsure

[runic crater]

What would you recommend to do to store this information globally?

[runic crater]

Oh I apologize, I thought I mentioned that. Its ocurring on the line where I run audio.Play(); in my playUpTo method

[tame yoke]

you may only need to track a few ints, one for the current track index, and one for the index you loop back to
additionally, you could keep track of how long the current track plays for, but you could also just query the AudioClip's length

[runic crater]

Yeah, I thought of something like that, and I don't remember why but I did have a reasoning I did it the way I did

I guess I dont need a clipsPlayed array and instead should have an int to store which measure I should go to.

[tame yoke]

this line:
clipsPlaying.Add(audioQueue[audioQueue.IndexOf(_source) + 1]);
is called without ever yielding, meaning that it runs while the foreach loop that calls destroyAfterTime() is still running

[tame yoke]

yes, that is much preferable

[runic crater]

Yeah, I think I even used some semi complicated math to get to this from my list haha

[runic crater]

This is under a yield, right?

[tame yoke]

oh, bruh, yeah

[runic crater]

haha

[tame yoke]

discord needs text coloring, I swear

[runic crater]

oh wait, hold up

[tame yoke]

actually if you prefix your code block with "```cs", that should fix it

[runic crater]

    public void playUpTo(int measure)
    {
        foreach (AudioSource audio in clipsPlaying)
            if(!audio.isPlaying)
            {
                audio.Play();
                StartCoroutine(destroyAfterTime(audio, measure));
            }

        if (audioQueue.Count == 0)
            PrepSong();

        if (clipsPlayed.Count == 0 && clipsPlaying.Count == 0)
            clipsPlaying.Add(audioQueue[0]);
    }

    IEnumerator destroyAfterTime(AudioSource _source, int measure)
    {
        float transitionTime = 0f;
        float repeatTime = 0f;
        float waitTime = 0f;

        foreach (MusicSection section in inputMusic.Sections)
        {
            if (audioQueue.IndexOf(_source) + 1 < audioQueue.Count)
                if (section.ReadAudio() == audioQueue[audioQueue.IndexOf(_source) + 1])
                    transitionTime = section.getTransDelay();
            else
                if (section.ReadAudio() == audioQueue[audioQueue.IndexOf(_source)])
                    repeatTime = section.getTransDelay();
        }

        waitTime = _source.clip.length - 0.035f;

        if (audioQueue.IndexOf(clipsPlaying[0]) < measure)
        {
            yield return new WaitForSeconds(waitTime);

            if (audioQueue.IndexOf(clipsPlaying[0]) < inputMusic.Sections.Length - 1)
            {
                if (audioQueue.IndexOf(_source) + 1 < audioQueue.Count)
                    clipsPlaying.Add(audioQueue[audioQueue.IndexOf(_source) + 1]);

                yield return new WaitForSeconds(transitionTime);
            }
            else
            {
                Debug.Log("playing last clip");
            }
        }
        else
        {
            clipsPlaying.Add(audioQueue[audioQueue.IndexOf(_source)]);
            yield return new WaitForSeconds(repeatTime);
        }

        clipsPlaying.Remove(_source);
            clipsPlayed.Add(_source);

    }

Is this any better?

[tame yoke]

ha, yeah

[runic crater]

Hooray

[tame yoke]

if section.getTransDelay() returns zero, that would cause that error

but again, I think you're using the coroutine in a very incorrect way

I'm trying to think about how I could better explain how it should be used without just giving you a code block (which I'm not really supposed to do)

maybe I can draw a diagram real fast

[runic crater]

Oh yeah yeah, that might be useful

[runic crater]

Yeah, working on fixing this right now

[tame yoke]

alright, screw it, I don't really know how to properly draw a diagram

[runic crater]

No worries, maybe its easier for you to explain this in a different scenario so I still have to apply it to my situation?

Also I really should go to bed now, I have a class early in the morning

[tame yoke]

int currentPlayingIndex, returnIndex;
float playTime;

List<AudioClip> currentSongQueue; // Attained from reading the data from a MusicHandler object

void Update()
{
  if (playTime < currentSongQueue[currentPlayingIndex].length) playTime += Time.deltaTime;
  else
  {
    // Spawn next track, iterate currentPlaying index
    // Return to "returnIndex" if queue is exhausted
  }
}

[runic crater]

Also just in case you were curious, here is a little bit of logic + some notes which I wrote out which was supposed to handle my code, but was overcomplicated and them "simplified" into the small coroutine.

[tame yoke]

here's a simple control loop that would fit your needs

all it does is track which track it's playing, and then iterates depending on whether or not the loop has waited for that track to play

this is probably all you need, along with a coroutine (the one previously discussed) to do the spawning, and a function to set returnIndex and currentSongQueue

[runic crater]

Ah okay, also we could get the playtime through audiosource.time I think

[tame yoke]

oh, shoot, missed that bit

[runic crater]

no worries

[tame yoke]

it's AudioSource.length, fyi
and fixed.

[runic crater]

I know there is both an audiosource.length and potentially audiosource.clip.time

I just forget if time is taken from the clip or source

oh cool there is a AudioSource.Time https://docs.unity3d.com/ScriptReference/AudioSource-time.html

[tame yoke]

the clip
you shouldn't ever need to query the AudioSource's time, all that should be handled by the coroutine

[runic crater]

Ah okay okay

[runic crater]

Could I get a little more elaboration on what you mean by "queue is exhausted"?

Also I am going to go to bed now, I'll work on this more tomorrow. Thank you for your help. I'll keep you updated.

[tame yoke]

if currentPlayingIndex is greater than the amount of tracks in the queue

[runic crater]

Ah okay thanks lol