🔒 Huggingface Inference Pipeline AttributeError: ‘NoneType’ object has no attribute ‘n_samples’

[placid parrot]

Can anyone help me figure the problem I'm facing here? So I tried to pass raw audio stream as numpy array into hunggingface's inference pipeline for transformers ASR but got got AttributeError: ‘NoneType’ object has no attribute ‘n_samples’ as shown on the pastebin below
here's the error message
and this is code snippet in question

[mighty roost]

@placid parrot

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[tiny shore]

‘NoneType’ object has no attribute ‘n_samples’

[placid parrot]

that's the thing I already print out the numpy array and they have shape and values in them though.

[fluid tusk]

you should send your actual code

[placid parrot]

right

from simple_diarizer.diarizer import Diarizer
from transformers import WhisperForConditionalGeneration
from transformers import AutomaticSpeechRecognitionPipeline as pipeline
 
import torch
import torchaudio
import whisper
 
...
    filename = from_path.get()
    split_ext = os.path.splitext(filename)
    audio = read_audio(filename, 16000)
    tempname = os.path.join(HOME_DIR, "Musics", f"{split_ext[0]}.wav")
    if not os.path.exists(tempname):
        save_audio(tempname, audio, sampling_rate=16000)

    signal = read_audio(tempname, 16000)
    diar = Diarizer(
                      embed_model='xvec',  # 'xvec' and 'ecapa' supported
                      cluster_method='sc'  # 'ahc' and 'sc' supported
                   )

    segments = diar.diarize(tempname, num_speakers=speaker_count.get(), silence_tolerance=1)

    diarizeProgress.set(100)
    popup.update()

    if model_type.get() != "custom":
        print("whisper")
        model = whisper.load_model("small", download_root="model")
    else:
        print("custom")
        customModel = WhisperForConditionalGeneration.from_pretrained(
            pretrained_model_name_or_path="whisper-small-ina/"
        )
        model = pipeline(model=customModel)

    speeches = []
    progress_step = float(100.0 / len(segments))
    for i, segment in enumerate(segments):
        popup.update()
        speech = signal[segment['start_sample']:segment['end_sample']]

        if model_type.get() != "custom":
            result = model.transcribe(speech)
        else:
            speechArray = speech.numpy()
            speechDict = dict(sampling_rate=16000, raw=speechArray)
            result = model(speechDict)

[fluid tusk]

your actual code is over 130 lines no?

[placid parrot]

yes

[fluid tusk]

often when people cut parts out to try and save space they remove crucial info for people trying to reproduce/debug

[placid parrot]

I'm pretty sure the tkinter has nothing to do with the hungging face inference, which is why i cut it out. but if you say so

[fluid tusk]

it also just helps to know exactly where the traceback is referencing, deleting parts of it changes the line count

[tiny shore]

*it

https://tenor.com/view/nerd-ackchyually-actually-gif-11156171

[fluid tusk]

idek why I added the s, it was right the first time lol

[placid parrot]

Here's the full code

[fluid tusk]

where in your code exactly did you print your array

right after line 133?

[tiny shore]

!

[placid parrot]

yeah but I delete it again sorry

it's a simple print(speechArray.shape)

[fluid tusk]

I see

what was the output?

[placid parrot]

the shape of every segment which should return shape(n, )

[fluid tusk]

what was the shape of the last one before it errored

just trying to see if there might be some incompatible shape or something

[placid parrot]

I'll try to run it again just to see where it went wrong

hang on this might take a while

[fluid tusk]

take your time, ping me when you have it

[placid parrot]

Much obliged for your patience 🎩 👌

[placid parrot]

I got only one item from the iteration

(326079,)

weirdly enough if I divide it by 16000 i got around 20 second which is quite long for a segment

[fluid tusk]

and it still errored?

[placid parrot]

yess

so the error happens at the first iteration

if the proble was with one of the segment it should have atleast reach few other segment first

i'm currently looking into the segmentation

pasting it as a json file and try using other audio file

[fluid tusk]

https://huggingface.co/openai/whisper-small this is the model you're using right?

[placid parrot]

technically yes, but I fine-tune it to my dataset

[fluid tusk]

gotcha, as long as the usages of the model are be the same, despite the parameters being different

[placid parrot]

Should be since I manage to pretrain it with no problem.

[fluid tusk]

I see that you're not using the WhisperProcessor that is shown in the huggingface page, seems like you're using the diarizer instead. Any particular reason for that?

[placid parrot]

I did use WhisperProcessor for fine-tuning, the diarizer is just an extra step I add before inferencing the raw audio stream to the pipeline.

So do I need to preprocess the audio with WhisperProcessor first before passing it into the Inference pipeline?

NOW THAT I THNK ABOUT IT

[fluid tusk]

yeah, the model should be taking the same inputs during training and inference

[placid parrot]

I probably should XD

[placid parrot]

never thought of it since I was trying to compare my model with openai's whisper model that's can just accept raw audio stream, thanks for the insight tonabrix1, this could be the solution I'm looking for

[fluid tusk]

fingers crossed 🤞

[placid parrot]

Unfortunately I'm getting a different error message but on the same line.

ValueError: We expect a single channel audio input for AutomaticSpeechRecognitionPipeline

[fluid tusk]

hm

[placid parrot]

oh wait I think I did it wrong

[fluid tusk]

does the diarizer make it multiple channels?

oh

[placid parrot]

lemme try something out

[placid parrot]

no but the feature extractor did xD, so I declare the feature extractor as a variable instead of passing it into the pipeline

[fluid tusk]

ahh lol

[gloomy island]

try this import os
import torchaudio
from simple_diarizer.diarizer import Diarizer
from transformers import WhisperForConditionalGeneration, WhisperProcessor, pipeline

Constants

SAMPLING_RATE = 16000
AUDIO_SAVE_DIRECTORY = "path/to/audio/save/directory" # Update with your directory
MODEL_DIRECTORY = "model" # Update if you have a different directory

def read_audio(file_path, sampling_rate=SAMPLING_RATE):
"""Reads an audio file and resamples it to the specified sampling rate.

Args:
    file_path (str): Path to the audio file.
    sampling_rate (int): Desired sampling rate (default: 16000 Hz).

Returns:
    torch.Tensor: The audio waveform as a PyTorch tensor.
"""
waveform, sr = torchaudio.load(file_path, mono=True)  # Load as mono
if sr != sampling_rate:
    resampler = torchaudio.transforms.Resample(orig_freq=sr, new_freq=sampling_rate)
    waveform = resampler(waveform)
return waveform

def save_audio(file_path, audio, sampling_rate=SAMPLING_RATE):
"""Saves an audio tensor to a file.

Args:
    file_path (str): Path to save the audio file.
    audio (torch.Tensor): Audio waveform as a PyTorch tensor.
    sampling_rate (int): Sampling rate (default: 16000 Hz).
"""
torchaudio.save(file_path, audio, sampling_rate)

if name == "main":
# Get the audio file path from user input or your preferred method
audio_file_path = input("Enter the path to your audio file: ")

# File handling and format conversion
audio_filename = os.path.basename(audio_file_path)
audio_name, audio_ext = os.path.splitext(audio_filename)
wav_file_path = os.path.join(AUDIO_SAVE_DIRECTORY, f"{audio_name}.wav")

if not os.path.exists(wav_file_path):
    audio_data = read_audio(audio_file_path)
    save_audio(wav_file_path, audio_data)

# Load the audio signal and diarize
signal = read_audio(wav_file_path)
diarizer = Diarizer(embed_model="xvec", cluster_method="sc")
segments = diarizer.diarize(wav_file_path, num_speakers=2, silence_tolerance=1)

# Load the Whisper model and processor
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
processor = WhisperProcessor.from_pretrained("openai/whisper-small")
asr_pipeline = pipeline(task="automatic-speech-recognition", model=model, tokenizer=processor.tokenizer)

# Transcribe each segment
transcriptions = []
for segment in segments:
    start_sample = segment["start_sample"]
    end_sample = segment["end_sample"]
    segment_audio = signal[:, start_sample:end_sample]

    # Preprocess audio with WhisperProcessor
    input_features = processor(segment_audio.squeeze(0).numpy(), sampling_rate=SAMPLING_RATE, return_tensors="pt").input_features

    # Pass preprocessed audio to the pipeline
    transcription = asr_pipeline(input_features)
    transcriptions.append(transcription["text"])

# Print or process transcriptions (e.g., save to file, display in UI)
for i, transcription in enumerate(transcriptions):
    print(f"Segment {i+1}: {transcription}")

[mighty roost]

Hey @gloomy island!

It looks like you are trying to paste code into this channel.

You seem to be using the wrong symbols to indicate where the code block should start. The correct symbols would be ```, not """.

Here is an example of how it should look:
```
Hello, world!
```

This will result in the following:

Hello, world!```
You can **edit your original message** to correct your code block.

[fluid tusk]

is this from chatGPT?

[gloomy island]

No, its a intertwinement of chatgpt4o, my personal AI i made, gemini advanvded and claude opus. '

[placid parrot]

I never used ChatGPT so I couldn't tell.

[gloomy island]

I dont use one lol.

[fluid tusk]

that would be against the rules of this server to post an answer from a language model, at least as a response to a question

!rule gpt

[mighty roost]

Rules

10. Do not copy and paste answers from ChatGPT or similar AI tools.

[placid parrot]

well at least for coding

[gloomy island]

Using one is silly.

[placid parrot]

I mostly use it for grocery list and whatnot

[gloomy island]

Well, dont let pride overcome your ability to solve a problem

[placid parrot]

IT

WORKS

YESSSS

[gloomy island]

lol

[placid parrot]

Thanks again @fluid tusk 🎩 👌

[placid parrot]

ngl this is my final thesis so I really need to get this before 29th of June 😭

[fluid tusk]

grats

[placid parrot]

https://tenor.com/bWbzqInHUIF.gif

consider this post solved

!solved

[mighty roost]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.