🔒 Inhomogeneous numpy shape when trying to save.

[analog maple]

I am trying to save a numpy array but I find that when I do I get the error ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (10, 8) + inhomogeneous part.
This doesnt happen when adding any elements to the array, only when I try and save it. My line to save the array is: np.save("emg_datasets.npy", emg_datasets)

[twilit echo]

@analog maple

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[burnt delta]

!traceback

[twilit echo]

Traceback

Please provide the full traceback for your exception in order to help us identify your issue.
While the last line of the error message tells us what kind of error you got,
the full traceback will tell us which line, and other critical information to solve your problem.
Please avoid screenshots so we can copy and paste parts of the message.

A full traceback could look like:

Traceback (most recent call last):
  File "my_file.py", line 5, in <module>
    add_three("6")
  File "my_file.py", line 2, in add_three
    a = num + 3
        ~~~~^~~
TypeError: can only concatenate str (not "int") to str

If the traceback is long, use our pastebin.

[burnt delta]

@analog maple

[analog maple]

Traceback (most recent call last):
File "c:\Users\AndrewWPI\Documents\GitHub\STEMI\main.py", line 55, in <module>
np.save("emg_datasets.npy", emg_datasets)
File "C:\Users\AndrewWPI\AppData\Roaming\Python\Python312\site-packages\numpy\lib_npyio_impl.py", line 586, in save
arr = np.asanyarray(arr)
^^^^^^^^^^^^^^^^^^
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (10, 8) + inhomogeneous part.

[burnt delta]

And where did emg_datasets come from

How did you create it

[analog maple]

It was data gathered through an emg sensor. Should I include the code for that?

[burnt delta]

The error is because your array is staggered

it's not rectangular

numpy can't represent arrays like that

well, not well

and apparently it wont save them to file

[analog maple]

huh. Why wouldnt it through this error when I was adding data to the dataset?

[burnt delta]

If it wasn't erroring earlier then it's because your array was homogeneous back then

now it isn't

If you are expecting your array to be rectangular, you can share your code and maybe I can help you figure out why it isn't

[analog maple]

import numpy as np
import time
from brainflow.board_shim import BoardShim, BrainFlowInputParams, BoardIds
from brainflow.data_filter import DataFilter, FilterTypes

def collect_data():
    params = BrainFlowInputParams()
    params.serial_port = "COM4"
    sampling_rate = 125
    duration = 1  # seconds

    datasets = []

    board = BoardShim(BoardIds.CYTON_BOARD.value, params)

    try:
        # Prepare session and start streaming
        board.prepare_session()
        board.start_stream()
        print("Starting data collection...")

        for i in range(10):
            print(f"Collecting dataset {i + 1}...")
            time.sleep(duration)

            # Retrieve raw data
            raw_data = board.get_board_data()

            # Extract EMG channels
            emg_channels = BoardShim.get_emg_channels(BoardIds.CYTON_BOARD.value)
            emg_data = raw_data[emg_channels, :]
            print(f"Shape of emg_data before clipping: {emg_data.shape}")

            # Clip the first and last second of data
            samples_to_clip = sampling_rate  # 1 second of data
            emg_data_clipped = emg_data[:, samples_to_clip:-samples_to_clip]

            # Store processed data
            datasets.append(emg_data_clipped)

        print("Data collection complete.")

    except Exception as e:
        print(f"Error: {e}")
    finally:
        board.stop_stream()
        board.release_session()

    return datasets

# Collect and save EMG data
emg_datasets = collect_data()
np.save("emg_datasets.npy", emg_datasets)

@burnt delta

[burnt delta]

Each of those emg_data arrays are going to be a different shape

you're throwing a bunch of differently-shaped arrays into one big list

numpy doesn't work well with that kind of big list

[analog maple]

alright. So should I just save them individually them?

[burnt delta]

you can store each of the differently-shaped arrays in different files, or perhaps you could pickle the data

[analog maple]

what do you mean by pickle the data?

[burnt delta]

you can serialize it in many different ways

[burnt delta]

!docs pickle

[twilit echo]

pickle

Source code: Lib/pickle.py

The pickle module implements binary protocols for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream (from a binary file or bytes-like object) is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as “serialization”, “marshalling,” [[1]](https://docs.python.org/3/library/pickle.html#id7) or “flattening”; however, to avoid confusion, the terms used here are “pickling” and “unpickling”.

[analog maple]

alright. Thanks for your help

[burnt delta]

np

[twilit echo]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.