Realtime API: Audio + Function Calling in same response?

[tribal bronze]

If I include a tool with tool_choice: 'required', then I get 'response.function_call_arguments' messages but no 'response.audio'. If I remove the tool and tool_choice, I get audio again. I would like to receieve both response.audio and response.function_call_arguments messages from the same response. Using node.

[tribal bronze]

Figured it out. Tools need their results added to the conversation before being able to respond with audio.

[shut pollen]

Hello mate, sorry I can't help you with your issue but maybe you can help me with mine ?

const playAudio = (base64Audio) => {
        if (typeof window !== 'undefined') {
            const binaryString = window.atob(base64Audio); // Décodage base64
            const len = binaryString.length;
            const bytes = new Uint8Array(len);
            for (let i = 0; i < len; i++) {
                bytes[i] = binaryString.charCodeAt(i);
            }
            const blob = new Blob([bytes.buffer], { type: 'audio/wav' });

            const audioURL = URL.createObjectURL(blob);
            const audio = new Audio(audioURL);
            audio.play().catch((error) => {
                console.error('Audio playback failed:', error);
            });
        }
    };

This is my function to convert the response to an audio that the browser can play

But it's not working I get this error :
Audio playback failed: NotSupportedError: Failed to load because no supported source was found.
window.console.error @ app-index.js:33
console.error @ hydration-error-info.js:63
eval @ VM371 useVirtualMeetingMessageEvent.js:218
Promise.then
playAudio @ VM371 useVirtualMeetingMessageEvent.js:217
handleWebSocketMessage @ VM371 useVirtualMeetingMessageEvent.js:72
eval @ VM371 useVirtualMeetingMessageEvent.js:38
Emitter.emit @ index.js:140
emitEvent @ socket.js:545
onevent @ socket.js:532
onpacket @ socket.js:502
Emitter.emit @ index.js:140
eval @ manager.js:219
Promise.then
eval @ globals.js:11
ondecoded @ manager.js:218
Emitter.emit @ index.js:140
add @ index.js:156
ondata @ manager.js:205
Emitter.emit @ index.js:140
_onPacket @ socket.js:272
Emitter.emit @ index.js:140
onPacket @ transport.js:108
onData @ transport.js:100
ws.onmessage @ websocket.js:57
Show 19 more frames
Show less

Do you know what is the format I have to use