#NameError: name 'dataset' is not defined

17 messages · Page 1 of 1 (latest)

hollow ravine
#

When I run
instruction = "You are an expert radiographer. Describe accurately what you see in this image."

def convert_to_conversation(sample):
conversation = [
{ "role": "user",
"content" : [
{"type" : "text", "text" : instruction},
{"type" : "image", "image" : sample["image"]} ]
},
{ "role" : "assistant",
"content" : [
{"type" : "text", "text" : sample["caption"]} ]
},
]
return { "messages" : conversation }
pass

then
converted_dataset = [convert_to_conversation(sample) for sample in dataset]

I get this error
NameError Traceback (most recent call last)
<ipython-input-4-a32c10305d17> in <cell line: 1>()
----> 1 converted_dataset = [convert_to_conversation(sample) for sample in dataset]

NameError: name 'dataset' is not defined

I have no idea how to fix it.

vital nacelle
#

Did you run the dataset loader ?
There should be 2 lines to load the dataset first then convert it to the format

# 
from datasets import load_dataset
dataset = load_dataset("unsloth/Radiology_mini", split = "train")
#

Run this before the instruction part then the instruction then convert the dataset

#

Should work just fine

#

If you are using colab free or have limited RAM , i suggest adding this to your dataset loader code ,

dataset = dataset.select(range(500))
#

This will load 500 samples from the 2.3k records , should be faster and use less ram to convert

hollow ravine
#

I did. I only need to load the data set once before this point, correct?

vital nacelle
#

yes, the error means the variable "dataset" was never initialized , nor given value

hollow ravine
#

I ran all of that before the error point, I still get that error.

vital nacelle
#

could you share your code / notebook ?

hollow ravine
vital nacelle
#

I am pretty sure you need to run the load dataset code block with each time , if you press run all , it should work

#

I just ran it and it loaded the data correctly, it did get stuck on the pushing to hub part

hollow ravine
#

i hit run all and it stopped at the error location

hollow ravine
#

I have figured out the issue. It was running out of system ram but wasnt informing me of this. I reduced the data select ranged down to 250 and it works now.

vital nacelle
#

Oh wow , that was the last thing i was gonna check tbh