sour gorge
I'm currently working on building stable diffusion from the ground up (yes, reimplement all components) and I'm super excited to share my progress with you all. I'll be posting updates on my journey as I go, including any challenges I encounter and how I overcome them.
I can't wait to share my thought process and get feedback from the community. This is going to be so much fun!
Github: https://github.com/xrsrke/stable-diffusion-from-scratch
Paper: https://arxiv.org/abs/2112.10752
amber vessel
Follwoing along with this 🙂
sour gorge
Follwoing along with this 🙂
I'm excited to get started on implementing CLIP next week. It's such an interesting concept – a multimodal neuron. Can't wait to dive into it!
sour gorge
Follwoing along with this 🙂
Update: got the pipeline working (check out github). This week, the goal is to reimplement CLIP from scratch. I'm finding the Clip tokenizer to be a bit challenging
tame crypt
have you seen the latest fast.ai course? it's not complete yet, but I believe the idea is to do something just like this. I think their dataset will be MNIST or something just like MNIST.
training on a dataset similar to stable diffusion is just out of reach for a hobbyist, but possible to train a similar model on a much smaller dataset
sour gorge
have you seen the latest fast.ai course? it's not complete yet, but I believe th...
are you taking the course? does Jeremy implement CLIP tokenizer from scratch?
sour gorge
have you seen the latest fast.ai course? it's not complete yet, but I believe th...
yes. i will train the model on MNIST data, but can generate other images using the pre-trained weights
tame crypt
I've watched 3 out of the 4 videos available so far, and they haven't gotten into the details of CLIP yet
do you really think you'll be able to build a model from scratch that is compatible with the pretrained weights? there seems like there are a lot of details you'd need to get right
sour gorge
do you really think you'll be able to build a model from scratch that is compati...
yeah. i will use https://github.com/geohot/tinygrad/blob/master/examples/stable_diffusion.py as a ground truth. just realize stable diffusion is big and complex
sour gorge
Update: I'm having some hard time with CLIP. But I'm working on improving my NLP skills. I'll definitely get back to the project soon
restive orchid
i think i will contribe, so damn good!
sour gorge
i think i will contribe, so damn good!
I’m working on CLIP. You do the U-Net
sour gorge
Update: I'm having some hard time with CLIP. But I'm working on improving my NLP...
yes.. working on NLP....
sour gorge
Yes. I’m actively working on this project
vivid topaz
Do you plan to be faithful to the original architecture or do you plan to improve a bit on it?
sour gorge
Do you plan to be faithful to the original architecture or do you plan to improv...
yep. the original architecture only
sour gorge
Do you plan to be faithful to the original architecture or do you plan to improv...
the project would be my deep learning foundation. then i move to something else
sour gorge
yes: im actively working on this project
restive orchid
Epic
restive orchid
yes: im actively working on this project
Any news?
sour gorge
Any news?
Im considering drop this project 🫣 Currently active working on implement RLHF (ChatGPT), Muzero and apply AI in quantum mechanics
Now I want to work on NLP
Sorry buddy
restive orchid
Im considering drop this project 🫣 Currently active working on implement RLHF (...
Bro, you can't continue on one project for just two weeks?
funny
sour gorge
Bro, you can't continue on one project for just two weeks?
It's not what i want to get done
I dont want to work on image generation anymore 
restive orchid
aren't chatgpt just gpt-3.5? you could use gpt-neo and yeah, that's it
im not a lot into chatgpt so idk
sour gorge
aren't chatgpt just gpt-3.5? you could use gpt-neo and yeah, that's it
I like NLP. Natural language is the univerval interface
restive orchid
I hate ai, it's a mess of ancien greek symbols and numbers