Bug with voice clone | ElevenLabs | Page 1

lethal glacier Dec 13, 2024, 3:04 PM

#

Hello everyone, recently i have started get error with voice cloning:

Traceback (most recent call last):
File "/tg_bots/lipsync_bot_employers_for_voices/scheduler.py", line 82, in generate_audio
voice = client.clone(
File "/usr/local/lib/python3.10/dist-packages/elevenlabs/client.py", line 108, in clone
add_voice_response = self.voices.add(
File "/usr/local/lib/python3.10/dist-packages/elevenlabs/voices/client.py", line 435, in add
_response = self._client_wrapper.httpx_client.request(
File "/usr/local/lib/python3.10/dist-packages/elevenlabs/core/http_client.py", line 198, in request
response = self.httpx_client.request(
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 812, in request
request = self.build_request(
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 367, in build_request
headers = self._merge_headers(headers)
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 430, in _merge_headers
merged_headers.update(headers)
File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 275, in update
headers = Headers(headers)
File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 156, in init
bytes_value = _normalize_header_value(v, encoding)
File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 81, in _normalize_header_value
raise TypeError(f"Header value must be str or bytes, not {type(value)}")

Maybe anyone know how to solve it?

modern tartan Dec 13, 2024, 3:08 PM

#

Can you share your code?

lethal glacier Dec 13, 2024, 3:31 PM

#

modern tartan Can you share your code?

voice = client.clone(
name=audio.split('/')[-1],
description="Empty",
files=[audio]
)

modern tartan Dec 13, 2024, 3:34 PM

#

lethal glacier voice = client.clone( name=audio.split('/')[-1], ...

that's all the code you have for the elevenlabs implementation?

lethal glacier Dec 13, 2024, 3:36 PM

#

modern tartan that's all the code you have for the elevenlabs implementation?

from elevenlabs.client import ElevenLabs
client = ElevenLabs(
api_key=elevenlabs_api_key,
)

#

Now, that`s all

modern tartan Dec 13, 2024, 4:16 PM

#

Are you passing the audio as bytes or just the file path?

If that's the case you'd need something like

# Open the file in binary mode and read its content
with open(audio, "rb") as audio_file:
    audio_bytes = audio_file.read()

# Use the binary data in the clone method
voice = client.clone(
    name=audio.split('/')[-1],
    description="Empty",
    files=[audio_bytes]  # Pass the binary data instead of the file path
)```

lethal glacier Dec 13, 2024, 4:48 PM

#

It helps, but now:

  File "/tg_bots/lipsync_bot_employers_for_voices/scheduler.py", line 85, in generate_audio
    voice = client.clone(
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/client.py", line 111, in clone
    files=[open(file, 'rb') for file in files],
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/client.py", line 111, in <listcomp>
    files=[open(file, 'rb') for file in files],
ValueError: embedded null byte```

modern tartan Dec 13, 2024, 5:53 PM

#

Is the audio one file or multiple?

lethal glacier Dec 13, 2024, 6:02 PM

#

One file

modern tartan Dec 13, 2024, 7:07 PM

#

Sorry maybe I'm wrong, I thought it could be been this..

What is "audio"?
The library is most likely expecting it to be a full file path from your system

lethal glacier Dec 13, 2024, 7:25 PM

#

‘audio’ is the path to an audio file of roughly the format “users/1979922062file_474.mp3” and I have now checked against the logs and the file is readable, the bytes are retrieved. And in general, everything was working stably until the last few days, but as I understand, after ElevenLabs update something changed in the code and now it doesn't work(
Here is the whole code with bug:

    client = ElevenLabs(
      api_key=elevenlabs_api_key,
    )
            with open(audio, "rb") as audio_file:
                audio_bytes = audio_file.read()
                
                voice = client.clone(
                    name=audio.split('/')[-1],
                    description="Empty",
                    files=[audio_bytes]
                )

random crown Dec 13, 2024, 7:52 PM

#

lethal glacier ‘audio’ is the path to an audio file of roughly the format “users/1979922062file...

Here you are passing the binary content of the audio not the path

#


from elevenlabs.client import ElevenLabs
import numpy as np
import wave
import struct

# Initialize the client
client = ElevenLabs(api_key="sk_")

# Generate mathematical audio for voice cloning
def create_math_audio(filename, duration=3, sample_rate=44100):
    t = np.linspace(0, duration, int(sample_rate * duration))
    frequency = 440  # A4 note
    wave_data = 32767 * (
        0.4 * np.sin(2 * np.pi * frequency * t) +
        0.3 * np.sin(3 * np.pi * frequency * t) +
        0.3 * np.cos(np.pi * frequency * t)
    )
    
    with wave.open(filename, 'w') as wave_file:
        wave_file.setnchannels(1)
        wave_file.setsampwidth(2)
        wave_file.setframerate(sample_rate)
        for sample in wave_data.astype(np.int16):
            wave_file.writeframes(struct.pack('h', sample))
    return filename

# Create math-based audio file
source_audio = create_math_audio("math_source.wav")

# Clone voice using the client method
with open(source_audio, 'rb') as file:
    voice = client.voices.add(
        name="MathVoice",
        description="Voice cloned from mathematical waveforms",
        files=[file]
    )
print(voice)```

lethal glacier Dec 13, 2024, 7:56 PM

#

random crown Here you are passing the binary content of the audio not the path

I know I originally passed the path, but on FrankieJonez's advice, I'm now passing bytes. Although it is written in the official documentation that it is necessary to pass the path

modern tartan Dec 13, 2024, 7:56 PM

#

lethal glacier I know I originally passed the path, but on FrankieJonez's advice, I'm now passi...

Sorry I was mistaken in this one while trying to work out what the issue might be.. 🙏

#

But if you were passing the path originally before we switched incorrectly then I don't know what the issue is..

lethal glacier Dec 13, 2024, 8:01 PM

#

modern tartan Sorry I was mistaken in this one while trying to work out what the issue might b...

No problem, then I return the code to its original state. But I still have no idea how to solve this problem...

lethal glacier Dec 13, 2024, 8:04 PM

#

modern tartan But if you were passing the path originally before we switched incorrectly then ...

The whole error, if it will help:

  File "/tg_bots/lipsync_bot_employers_for_voices/scheduler.py", line 82, in generate_audio
    voice = client.clone(
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/client.py", line 108, in clone
    add_voice_response = self.voices.add(
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/voices/client.py", line 435, in add
    _response = self._client_wrapper.httpx_client.request(
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/core/http_client.py", line 198, in request
    response = self.httpx_client.request(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 812, in request
    request = self.build_request(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 367, in build_request
    headers = self._merge_headers(headers)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 430, in _merge_headers
    merged_headers.update(headers)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 275, in update
    headers = Headers(headers)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 156, in __init__
    bytes_value = _normalize_header_value(v, encoding)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 81, in _normalize_header_value
    raise TypeError(f"Header value must be str or bytes, not {type(value)}")
TypeError: Header value must be str or bytes, not <class 'dict'>```

#

  File "/tg_bots/lipsync_bot_employers_for_voices/scheduler.py", line 82, in generate_audio
    voice = client.clone(
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/client.py", line 108, in clone
    add_voice_response = self.voices.add(
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/voices/client.py", line 435, in add
    _response = self._client_wrapper.httpx_client.request(
  File "/usr/local/lib/python3.10/dist-packages/elevenlabs/core/http_client.py", line 198, in request
    response = self.httpx_client.request(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 812, in request
    request = self.build_request(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 367, in build_request
    headers = self._merge_headers(headers)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 430, in _merge_headers
    merged_headers.update(headers)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 275, in update
    headers = Headers(headers)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 156, in __init__
    bytes_value = _normalize_header_value(v, encoding)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 81, in _normalize_header_value
    raise TypeError(f"Header value must be str or bytes, not {type(value)}")
TypeError: Header value must be str or bytes, not <class 'dict'>

During handling of the above exception, another exception occurred:```

#

  File "/tg_bots/lipsync_bot_employers_for_voices/handlers/hand_start.py", line 249, in get_text
    new_voice = await scheduler.generate_audio(message.text, split_audio, state)
  File "/tg_bots/lipsync_bot_employers_for_voices/scheduler.py", line 117, in generate_audio
    await bot.send_message(
  File "/usr/local/lib/python3.10/dist-packages/aiogram/bot/bot.py", line 346, in send_message
    result = await self.request(api.Methods.SEND_MESSAGE, payload)
  File "/usr/local/lib/python3.10/dist-packages/aiogram/bot/base.py", line 236, in request
    return await api.make_request(await self.get_session(), self.server, self.__token, method, data, files,
  File "/usr/local/lib/python3.10/dist-packages/aiogram/bot/api.py", line 140, in make_request
    return check_result(method, response.content_type, response.status, await response.text())
  File "/usr/local/lib/python3.10/dist-packages/aiogram/bot/api.py", line 115, in check_result
    exceptions.BadRequest.detect(description)
  File "/usr/local/lib/python3.10/dist-packages/aiogram/utils/exceptions.py", line 140, in detect
    raise err(cls.text or description)
aiogram.utils.exceptions.CantParseEntities: Can't parse entities: unsupported start tag "class" at byte offset 64```

modern tartan Dec 13, 2024, 8:14 PM

#

Are you running the latest library version?

#

And when was the last time this code ran without error?

lethal glacier Dec 13, 2024, 8:17 PM

#

modern tartan Are you running the latest library version?

Yes

lethal glacier Dec 13, 2024, 8:17 PM

#

modern tartan And when was the last time this code ran without error?

Two days ago

modern tartan Dec 13, 2024, 8:22 PM

#

Very strange, I can't test it myself right now unfortunately but all I can suggest is looking at dawns code above or trying a dkrecty http api call instead..

lethal glacier Dec 13, 2024, 8:29 PM

#

I read the documentation, but I couldn't find how to make a clone of the voice via the http api. Can you give me a link?

modern tartan Dec 13, 2024, 8:33 PM

#

https://elevenlabs.io/docs/api-reference/add-voice

ElevenLabs

Add Voice - ElevenLabs

Add a new voice to your collection of voices in VoiceLab.

#

You do need to pass binary data for the audio, not the file paths..

lethal glacier Dec 13, 2024, 9:32 PM

#

I have now rolled back to version 0.2.27, which is the version I was originally using and on which the error appeared. Here is the error code:

  File ‘/tg_bots/lipsync_bot_employers_for_voices/scheduler.py’, line 81, in generate_audio
    voice = clone(
  File ‘/usr/local/lib/python3.10/dist-packages/elevenlabs/simple.py’, line 29, in clone
    return Voice.from_clone(VoiceClone(**kwargs))
  File ‘/usr/local/lib/python3.10/dist-packages/elevenlabs/api/voice.py’, line 124, in from_clone
    voice_id = API.post(url, data=data, files=files).json()[‘voice_id’].
  File ‘/usr/local/lib/python3.10/dist-packages/elevenlabs/api/base.py’, line 79, in post
    return API.request(url, method=‘post’, *args, **kwargs) # type: ignore
  File ‘/usr/local/lib/python3.10/dist-packages/elevenlabs/api/base.py’, line 46, in request
    error = json.loads(response.text)
  File ‘/usr/lib/python3.10/json/init.py’, line 346, in loads
    return _default_decoder.decode(s)
  File ‘/usr/lib/python3.10/json/decoder.py’, line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File ‘/usr/lib/python3.10/json/decoder.py’, line 355, in raw_decode
    raise JSONDecodeError(‘Expecting value’, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```
The code that causes the error:
```set_api_key(elevenlabs_api_key[‘elevenlabs’])
voice=clone(
name=audio.split(‘/’)[-1],
files=[audio]
)```

#

Maybe this error will be easier to solve

lethal glacier Dec 14, 2024, 4:39 PM

#

@modern tartan Do you have any ideas?

modern tartan Dec 14, 2024, 5:21 PM

#

So you rolled back from 1.13.15 to 0.2.27? Any reason you rolled back to a release from a year ago when you were not having issues just a few days ago? Just trying to understand the whole picture..

lethal glacier Dec 14, 2024, 5:34 PM

#

modern tartan So you rolled back from 1.13.15 to 0.2.27? Any reason you rolled back to a relea...

I had no problems a few days ago, then I was using version 0.2.27. Unexpectedly, I did not make any changes to the code, but in the last few days an error started to occur, which I sent in the last message. Therefore, I tried to upgrade to the newest version and after that I started getting an error related to TypeError (the first message in thread). And now I've rolled back to version 0.2.27 again, because I thought it might be easier for you to solve the problem on this version. Otherwise, I can try to upgrade to the newest version again if you know how to fix the bug that occurs on it.

modern tartan Dec 14, 2024, 6:56 PM

#

Are you sure the name is a string?

lethal glacier Dec 14, 2024, 7:42 PM

#

Yes

modern tartan Dec 14, 2024, 7:52 PM

#

Well apart from suggesting that you try the implementation of t
this seperate from the rest of your code, I'm about out of troubleshooting steps at this stage.

#

I've not used the lib to clone a voice in the last week or two but I know the end point is up and running with no reported issues as of the past few days. M

lethal glacier Dec 14, 2024, 10:10 PM

#

Judging by the error code, there is a feeling that the request simply does not reach the server. Could it be that the bot is running on a Russian server?

modern tartan Dec 15, 2024, 8:16 AM

#

lethal glacier Judging by the error code, there is a feeling that the request simply does not r...

What bot are you referring to?

lethal glacier Dec 15, 2024, 8:18 AM

#

modern tartan What bot are you referring to?

The ElevenLabs api is used in my telegram bot, which is uploaded to a Russian cloud server

modern tartan Dec 15, 2024, 8:23 AM

#

If the API call is coming from a Russian server then it's not going to work I would assume.

modern tartan Dec 15, 2024, 8:23 AM

#

modern tartan Well apart from suggesting that you try the implementation of t this seperate fr...

Can you please test like this..

#

Locally from your system, seperate from all other code..

lethal glacier Dec 15, 2024, 9:00 AM

#

modern tartan Locally from your system, seperate from all other code..

That's not the point, and I already realized that the point was precisely that the api received a request from the Russian server. Therefore, now I use a proxy and the request to the server arrives, but..
My code:

                    'name': audio.split('/')[-1],
                    'files': [audio]
                }
                headers = {
                    'xi-api-key': Config.elevenlabs
                }
                url = 'https://api.elevenlabs.io/v1/voices/add'

                async with s.post(url=url, headers=headers, data=body, proxy=proxy) as resp:
                    print(resp)
                    res = await resp.text()
                    print(res)

And server response:

<CIMultiDictProxy('Date': 'Sun, 15 Dec 2024 08:56:31 GMT', 'Server': 'uvicorn', 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Headers': '*', 'Access-Control-Allow-Methods': 'POST, PATCH, OPTIONS, DELETE, GET, PUT', 'Access-Control-Max-Age': '600', 'strict-transport-security': 'max-age=31536000; includeSubDomains', 'x-trace-id': 'c21d6c3a1548c5ec60edaddfa02229f4', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000', 'Transfer-Encoding': 'chunked')>

{"detail":[{"loc":["body","files",0],"msg":"Expected UploadFile, received: <class 'str'>","type":"value_error"}]}```

brazen sphinxBOT Dec 15, 2024, 9:01 AM

#

Hello, @lethal glacier

Unfortunately, we're unable to assist with this issue through Discord. However, if you reach out to our Customer Support team by opening a ticket using this button, they'll be more than happy to help with your inquiry.

modern tartan Dec 15, 2024, 9:02 AM

#

You'll need to contact the support team to see if they can help you, I can't help with the information I have available..

lethal glacier Dec 15, 2024, 9:05 AM

#

modern tartan You'll need to contact the support team to see if they can help you, I can't hel...

What other information do you need? I am ready to provide it

lethal glacier Dec 15, 2024, 9:05 AM

#

lethal glacier That's not the point, and I already realized that the point was precisely that t...

Obviously, the server gives such a response, because I give it a list of paths to audio recordings as "files" (as it is written in the documentation). But for some reason, does the server need to transfer binary data?..

modern tartan Dec 15, 2024, 9:24 AM

#

I just tested myself for the first time, I've been on my phone all weekend so I've had no opportunity to test it.

#

And yes I see the same issue, when I first asked I was told there was no issue with the end point by someone from the team so I didn't go further than that..

#

I've now forwarded it back to them and will see what they say..

#

Can you please kill your API key too? Yes it's not the full key but could be brute forced easily enough.. 🤙

lethal glacier Dec 15, 2024, 9:29 AM

#

OK, thanks a lot for the answer, but is there any way I can influence the solution to this problem? Or do I just have to wait for someone from your team to fix it?

modern tartan Dec 15, 2024, 9:34 AM

#

I'm assuming you need to be able to dynamically add the voices based on user interaction?

lethal glacier Dec 15, 2024, 9:35 AM

#

modern tartan I'm assuming you need to be able to dynamically add the voices based on user int...

If I understood your wording correctly, then yes

modern tartan Dec 15, 2024, 9:37 AM

#

Like you can't add the voices yourself via the Webapp?

lethal glacier Dec 15, 2024, 9:48 AM

#

To be honest, I didn't fully understand you... but in my application, the user uploads a video and his text, an audio track is separated from this video, a voice clone is created from this audio track, and then audio received from the elevenlabs api is generated from this audio track and the text sent by the user

modern tartan Dec 15, 2024, 10:46 AM

#


client = ElevenLabs(
    api_key="APIKEY",
)
client.voices.add(
    name="AlexAAA",
    files=open[("/Users/user/Downloads/asd.mp3", "rb")]
)```

#

Can you try this?

lethal glacier Dec 15, 2024, 12:23 PM

#

modern tartan Can you try this?

I can't try this construction, because as I wrote above, I use api requests via http to use a proxy. Or is it possible to use a proxy using the library?

modern tartan Dec 15, 2024, 12:56 PM

#

But this is not what you were using easier in this thread?

                    name=audio.split('/')[-1],
                    description="Empty",
                    files=[audio]
                )```

#

Where is the audio being processed that the users are uploading? You're doing to processing of the video files on your server? You need to pass the API a list of local paths to your setup that it can then read and pass the data though..

lethal glacier Dec 15, 2024, 1:22 PM

#

modern tartan But this is not what you were using easier in this thread? ```voice = client.cl...

Yes, it would be easier, but wouldn't it be possible to use a proxy?

lethal glacier Dec 15, 2024, 1:23 PM

#

modern tartan Where is the audio being processed that the users are uploading? You're doing to...

I save audio tracks from video files in the users folder and pass the local path to the audio file to the api

lethal glacier Dec 15, 2024, 1:24 PM

#

modern tartan I've now forwarded it back to them and will see what they say..

Is there any news from them?

modern tartan Dec 15, 2024, 1:26 PM

#

lethal glacier Is there any news from them?

I already posted the reply above, I tested it myself and confirm that it works


client = ElevenLabs(
    api_key="APIKEY",
)
client.voices.add(
    name="AlexAAA",
    files=open[("/Users/user/Downloads/asd.mp3", "rb")]
)```

lethal glacier Dec 15, 2024, 1:31 PM

#

modern tartan I already posted the reply above, I tested it myself and confirm that it works ...

I have no doubt that this works, but I repeat once again: Because my bot is on a Russian server, when accessing the api using the library, I get an error. Therefore, now I access the http api directly using a proxy server. Therefore, if you cannot configure a proxy server for the library, then you will not be able to use the library for implementation

lethal glacier Dec 15, 2024, 1:35 PM

#

lethal glacier That's not the point, and I already realized that the point was precisely that t...

I've checked it now and the problem is still relevant (On the site with the api reference, too)

modern tartan Dec 15, 2024, 1:35 PM

#

lethal glacier I have no doubt that this works, but I repeat once again: Because my bot is on a...

The docs site does not function as expected at this stage.

lethal glacier Dec 15, 2024, 1:37 PM

#

modern tartan The docs site does not function as expected at this stage.

When is it possible to expect a solution to the problem? If you have such information, of course

modern tartan Dec 15, 2024, 1:38 PM

#

lethal glacier When is it possible to expect a solution to the problem? If you have such inform...

But what is the problem still? If you can go the direct HTTP route you should be fine

lethal glacier Dec 15, 2024, 1:39 PM

#

lethal glacier I've checked it now and the problem is still relevant (On the site with the api ...

This problem

lethal glacier Dec 15, 2024, 1:41 PM

#

lethal glacier This problem

modern tartan Dec 15, 2024, 1:54 PM

#

The docs site does not function as expected at this stage.

#

import os

api_key = os.getenv("ELEVEN_LABS_API_FJ")
url = "https://api.elevenlabs.io/v1/voices/add"

headers = {
    "Accept": "application/json",
    "xi-api-key": api_key
}

files = {
    'files': ('audio.mp3', open(r"C:\Users\users\Downloads\test.mp3", "rb"), 'audio/mpeg'),
    'name': (None, 'Clone test2'),
    'description': (None, 'A cloned voice')
}

response = requests.post(url, headers=headers, files=files)
print(response.json())```

lethal glacier Dec 15, 2024, 2:01 PM

#

modern tartan The docs site does not function as expected at this stage.

I mean, that's a problem, right? When will it be fixed approximately?

modern tartan Dec 15, 2024, 2:01 PM

#

This HTTP method should work

#Bug with voice clone