just a hunch, try
inputs.append(keras.layers.Input(batch_shape=(None,), dtype="int64", name=col)) (note batch_shape instead of shape) in your for col in... loop
#'NoneType' object is not subscriptable during fit()
ruby estuary
safe wyvern
Now it gives me :
ValueError: Input 0 of layer "bidirectional_10" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 256)
ruby estuary
eh that doesnt sound like what i was looking for 😦
safe wyvern
No worries 😉
ruby estuary
maybe print(embedded) before creating your LSTM layer?
safe wyvern
Also to make it clearer:
for i, j in train_ds:
print("len of i", len(i))
print("len of j", len(j))
print("keys of i", i.keys())
print("keys of j", j.keys())
break
len of i 4
len of j 2
keys of i dict_keys(['text', 'prompt_question', 'prompt_title', 'prompt_text'])
keys of j dict_keys(['content', 'wording'])
(I had changed the names and structure in my question on pupose to make it easier to understand)
safe wyvern
maybe `print(embedded)` before creating your LSTM layer?
Ok
safe wyvern
maybe `print(embedded)` before creating your LSTM layer?
KerasTensor(type_spec=TensorSpec(shape=(None, 256), dtype=tf.float32, name=None), name='embedding_10/embedding_lookup/Identity:0', description="created by layer 'embedding_10'")
is the error during fit() or when you generate_model()? (you say in the OP that its during fit(), but then you have a comment saying the error is in generate_model)
safe wyvern
this one is during model generation
but the first one was during fit
when I change from batch_shape to shape again the generation works
I then get this error during fit() : ```TypeError: Exception encountered when calling layer 'forward_lstm_15' (type LSTM).
'NoneType' object is not subscriptable
Call arguments received by layer 'forward_lstm_15' (type LSTM):
• inputs=tf.Tensor(shape=, dtype=float32)
• mask=tf.Tensor(shape=, dtype=bool)
• training=True
• initial_state=None
can you share your call to fit?
safe wyvern
Sure!
history = model.fit(train_ds, epochs=20, callbacks=[callbacks])
Note that I have also this for the batches : ```py
train_ds = train_ds.batch(32)
what is text_vectorization?
safe wyvern
max_tokens = 20000
max_length = 600
text_vectorization = keras.layers.TextVectorization(
max_tokens=max_tokens,
output_mode="int",
output_sequence_length=max_length,
)
text_vectorization.adapt(all_texts)
The keras layer
ruby estuary
yeah i find it hard to believe a none is making its way past the other layers before finally causing the LSTM to fail... sorry im not much help lol
safe wyvern
No worries x)
I'm dying
It's been 13 hours now
@ruby estuary Is there a way to print the data? So maybe I can check where the None value occurs
ruby estuary
you could probably print the same thing youre yielding in your generator, just before you yield it
safe wyvern
I'm not sure I understood...
def generator():
for text, prompt_question, prompt_title, prompt_text, content, wording in zip(train_features["text"], train_features["prompt_question"], train_features["prompt_title"], train_features["prompt_text"], train_targets["content"], train_targets["wording"]):
print({"text": text_vectorization(text), "prompt_question": text_vectorization(prompt_question), "prompt_title": text_vectorization(prompt_title), "prompt_text": text_vectorization(prompt_text)}, {"content": content, "wording": wording})
yield {"text": text_vectorization(text), "prompt_question": text_vectorization(prompt_question), "prompt_title": text_vectorization(prompt_title), "prompt_text": text_vectorization(prompt_text)}, {"content": content, "wording": wording}
this is what you meant?
to print before the yield
ruby estuary
yep
safe wyvern
then how to call it?
ruby estuary
itll be spammy, and idek if thats where the nones are coming from
ruby estuary
then how to call it?
fit will probably call it for you
safe wyvern
? I use it before
right here ```py
train_ds = tf.data.Dataset.from_generator(generator, output_types=({"text": tf.int64, "prompt_question": tf.int64, "prompt_title": tf.int64, "prompt_text": tf.int64}, {"content": tf.float32, "wording": tf.float32}))
i suppose you could just iterate over it
for foo in generator:
pass
thats what tf dataset will do anyway
safe wyvern
i have no none values
hmm
it means that one of my layers creates a none value
most probably the embedding layer
ruby estuary
it could be, the thing that gives you a bit more of a clue is that whatever is expecting the none wants to subscript it (expecting a dict? list?)
its possible its trying to subscript a tensor (but getting None), but i feel like whatever would be producing that tensor would be giving you an error first
safe wyvern
It’s so strange
Each of my inputs are tensors of shape (600,)
And my outputs are scalars so tensors of shape ()
