HuggingFace error

[austere lotus]

I get this error when I try to use the simple text generation end point of the hugging face inference. Can anyone help how to rectify?

Error: An error occurred while fetching the blob

[warm lantern]

Can you share your code, or a link to a Scrim

[austere lotus]

Here's my code -

`import { HfInference } from '@huggingface/inference'

const hf = new HfInference(process.env.hfKey)

const textToGenerate = "The definition of machine learning inference is "

const response = await hf.textGeneration({
inputs: textToGenerate
})

console.log(response)`

[austere lotus]

I had saved my access token with the name of hfKey in the Scrim Evnironment variables.
I also tried consoling the Access token, and it did get logged in the console. So I am pretty sure the access token is valid.
But not sure why I get the error each time....
Thank you for looking into this.... Awaiting...!

[warm lantern]

I couldn't replicate the error using your code, but I did get an error saying that I needed to specify a model. Unfortunately, I can't find a model that works with the textGeneration method.
Can I ask how you decided on textGeneration and where you found info to get you started with it? The only information I can find on it is an example from the documentation that doesn't work.

[austere lotus]

I just tried the same set of code as explained in the course of "The AI Engineer Path".

Here's the lesson for reference -
Intro to huggingface.js Inference
https://scrimba.com/the-ai-engineer-path-c02v/~01b

Whatever challenges are listed in that lesson, I am unable to perform due to this error....
I too didn't find relevant textgeneration documentation to understand this error

[warm lantern]

Ah right, I see. Are you running your code locally or within Scrimba?
I had to downgrade to version 2.6.1 of @huggingface/inference and add a model to get it working on my end and I get no error:```js
const response = await hf.textGeneration({
inputs: textToGenerate,
model: 'HuggingFaceH4/zephyr-7b-beta'
})

[austere lotus]

I am running on Scrimba...
I in fact did on the spot there, when the lesson asked to submit the challenge.

Thanks, this code did work now 👍

So how do we get to know which version to use?

[warm lantern]

You shouldn't have to worry about the version if you are running within the lesson on Scrimba. I was originally testing your code locally and none of the models were working on 3.12.1. Once you shared a link to the lesson I was able to get it working with the exact same code there, so put it down to my local version being the issue.
Can you share a link to the specific Scrim in which you are getting the error.

It might just be an issue with your instance of Scrimba, in which case I'll share a few steps you can try to get it working. Just need to confirm it isn't your code first.

[warm lantern]

Oh, I misunderstood. So it's all working fine now?

[austere lotus]

Its working now when I use your code with the model mentioned.
But in the lesson, there was no mention about adding any model there...
So that confused me....
I am doing everything only on Scrimba first to get a hang of it and not working on local machine...!

Here's the link of my Scrimb -
https://scrimba.com/the-ai-engineer-path-c02v/~01b/s0re22n4m5/head

So it doesn't work if I remove the model....!!

[warm lantern]

I haven't listened to the lesson, but I did copy the line model: "HuggingFaceH4/zephyr-7b-beta" directly from the lesson and it appears before the first challenge. so 🤷‍♂️.
In any case, glad it's working for you now

Oh... I see, yeah, I don't get why it works when he runs it without a model

The API must have changed since the lesson was created. I think because providers were added (?), since the error I was getting was related to being unable to automatically select a model without specifying a provider.

Odd that they would update their API in a breaking way.

[austere lotus]

Oh... Perhaps I might have missed taking note of that model if that's mentioned.
I shall recheck it.

[austere lotus]

Oh I understand...
Thanks a lot for helping...
I shall check them

[warm lantern]

Nah, he ran it without the model initially. Just the API updating in a non-backwards-compatible way.

I think that at the time of recording the Scrim there was only the single provider (hf-inference), so now that there are many to choose from it just breaks preexisting code. Given that's the case, I imagine this would work without supplying a model, but I can't test it without making a new account as I'm out of free credits for now:```js
const response = await hf.textGeneration({
inputs: textToGenerate,
provider: 'hf-inference'
})

Edit: It doesn't, since in 2.6.1 "provider" isn't an expected key and in 3.12.1 no models seem to work anyway.

[austere lotus]

Oh... Get it..
That's ok..
It works anyways now.
I shall try checking similarly for other ones where I get the same error..

Thanks again 👍

[warm lantern]

Just to add for the sake of posterity for when I inevitably have to search for this error again in future:

• The extremely vague "Error: An error occurred while fetching the blob" message only appears in the earlier version used by Scrimba. In version 3.12.1 you get a more useful error message: "Error: Specifying a model is required when provider is 'auto'".
• When specifying a provider without a model in 3.12.1, as in #1375340885155385536 message, you get "Error: We have not been able to find inference provider information for model google/gemma-2-2b-it.".
• When adding a model in version 3.12.1 you get "Error: Model HuggingFaceH4/zephyr-7b-beta is not supported for task text-generation and provider hf-inference. Supported task: conversational". This error appears for every model I tested. The only working solution I have found is to downgrade to 2.6.1 (which has the useless error messages) and specify a model (provider is not an expected option in this version, as providers weren't a thing at the time).
• In version 3.12.1 HFInference is actually deprecated and exists only for legacy code, instead you should use InferenceClient. Though, this seems to make no real difference as you still get the same error message: "Error: Model HuggingFaceH4/zephyr-7b-beta is not supported for task text-generation and provider hf-inference. Supported task: conversational."
• I still don't know how to get textGeneration working in 3.12.1, but chatCompletion works fine instead:

const response = await hf.chatCompletion({
  model: 'HuggingFaceH4/zephyr-7b-beta',
  messages: [{ role: 'user', content: textToGenerate }]
})

• This Text Generation article even demonstrates with the chatCompletion method (using model mistralai/Mistral-Nemo-Instruct-2407), so I'm not really sure where the textGeneration method used in The AI Engineering Path was plucked from.