#GPT2 model training went wrong.

1 messages · Page 1 of 1 (latest)

topaz rivet
#

Hello. I was training gpt2 model 124M. And it went wrong. Like after extensive training of 1day nearly complete pc on. lots of computation and resources. It gave an attribute error.

This was the code:

import gpt_2_simple as gpt2
import os

# Path to your training data file (process.txt)
data_file_path = 'process.txt'

# Set up the GPT-2 model
model_name = "124M"  # You can also use "774M" or "1558M" for larger models
sess = gpt2.start_tf_sess()

# Download the GPT-2 model
gpt2.download_gpt2(model_name="124M")

# Train the GPT-2 model
gpt2.finetune(sess,
              dataset=data_file_path,
              model_name=model_name,
              steps=1000,  # You can adjust the number of training steps as needed
              restore_from='fresh',
              run_name='chatbot_model')

# Save the trained model
model_directory = os.path.join(gpt2.models.GPT2_MODEL_DIR, model_name)
gpt2.save_gpt2(sess, run_name='chatbot_model', model_name=model_directory)

and it gave the error: ```
Saving checkpoint/chatbot_model/model-1000
Traceback (most recent call last):
File "/home/ahmed/bots/train.py", line 23, in <module>
model_directory = os.path.join(gpt2.models.GPT2_MODEL_DIR, model_name)
^^^^^^^^^^^
AttributeError: module 'gpt_2_simple' has no attribute 'models'. Did you mean: 'model'?

so please help me i don't want to loose the trained model.
boreal heathBOT
#

Hi @topaz rivet. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

I'm sorry, but the information extracts provided do not contain the necessary information to solve the issue you're experiencing with the GPT-2 model training. The extracts mainly discuss issues and examples related to OpenAI's GPT-3 and GPT-3.5-turbo models, and the OpenAI Python API library. They do not provide information on the GPT-2 model or the 'gpt_2_simple' module you're using.

The error message you're seeing suggests that the 'gpt_2_simple' module does not have an attribute 'models'. It might be a good idea to check the documentation or source code of the 'gpt_2_simple' module to see if 'models' is a valid attribute, or if there's a typo or misunderstanding in your code.

If you're still having trouble, you might want to consider asking for help on a platform like Stack Overflow, or on a relevant GitHub repository if one exists. Be sure to include all relevant information in your question, including the full error message and the code that's causing it.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@unreal quail kapa.ai is still learning and improving, please let us know how it did by reacting below

topaz rivet
#

@versed oasis this was those old days and error.

versed oasis
topaz rivet
runic skiff
#

if it was tensorflow or pytorch i could help but i idk what is that code