Add a New official template

[limber dirge]

Please add a template with PyTorch 2.7, CUDA 12.8, Python 3.12, and Ubuntu 22.04

I made mine, but it downloads it every time I start a pod.

If you want I can give you mine I just need it to be cached in the runpod servers so that it doesn't download everytime I start a new pod.

I have also tried using sha256 but it only works for same pods and with a short period of time.

Please add this please.

[full island]

Working on something similar to this today I'll let you know how that goes soon

[limber dirge]

Thanks I look forward to it

[limber dirge]

@full island Any update on this?

[full island]

Let me figure out how to deploy this, I made the image but they're not publically available yet.

[limber dirge]

Thanks I will wait for your reply

[limber dirge]

You mean adding an official runpod template, right? Because I have mine already.

[limber dirge]

@full island did you see this?

[grave portal]

@full island any updates on this?

[full island]

I'm not ready to add a template, but you should be fine with the 0.7.1-dev versions of the new pytorch image:

PyTorch 2.7.0:
runpod/pytorch:0.7.1-dev-ubuntu2204-cu1281-torch270

PyTorch 2.7.1:
runpod/pytorch:0.7.1-dev-ubuntu2204-cu1281-torch271

These containers have python installed as python3.12 not just python. I'm not sure how else I can safely accommodate Python versioning as just the python3 binary without making it something the builder accommodate for (and adding another variable to the tag)

[limber dirge]

So python still points to 3.11?

You can use sym link

Are these images cached?

[full island]

Let me check where exactly it goes, it might be python 3.8 or whatever ubuntu22.04 needs

No these are still dev images, you will be the first person on the entire platform to request any of them we can't do any of our caching until the images actually exist.

I can look into promoting them to be the default but that changes a lot of things for a lot of people and is kind of a scary change to make 👀

[limber dirge]

What you mean by images actually exist?

[limber dirge]

And this

[full island]

The images need to be requested by the server via a docker pull before they can be cached onto that specific node. Because these are still dev images noone has done that before to any of the servers. You will have to wait for them to be downloaded, which will be kind of rough for your cold starts initially but as time goes on it will be less of a problem as more of the servers in the platform actually do that initial work.

One way of getting around this would be to publish these as the templates we recommend (and taking the -dev out of the version) - but I can't just do that without a little bit of prep work and more confidence in these images.

I am the only engineer at Runpod working on this container revamp project, preparing for a full scale release will require a little bit of planning and just isn't something I'm ready for just yet.

[limber dirge]

I completely understand that

[full island]

(This is 0.7.1-dev-ubuntu2204-cu1281-torch271)

root@aa1157985797:/# python -V
bash: python: command not found
root@aa1157985797:/# python3 -V
Python 3.10.12
root@aa1157985797:/# python3.12 -V
Python 3.12.11

One thing I could do to make python3 the python3.12 binary is add another layer of depth to the tags I generate and make it a variable but I don't think I like the idea of publishing 0.7.1-ubuntu2204-py312-cu1281-torch271 it's a little unwieldily. I'd probably just recommend, where you can, using the specificly versioned binary or adding the symlink yourself as a user.

I would be open to installing python-is-python3 which would make the python binary link to python3 - in that example it makes it 3.10.12, I think that's a reasonable request

[limber dirge]

So the 502 error is also solved?

[limber dirge]

Will it work if I do it on my end?

[full island]

I would test it out, ubuntu can be a little fragile about major version changes but it should

[full island]

I don't think there's a specific 502 error, I think we run nginx in the base images and we shouldn't be. When I added the port 8080 and ran a service myself python3 -m http.server 8080 I was just fine.

I'm going to modify the base image to have a cleaner nginx config but that shouldn't affect your pods connectivity. It's just by default nothing is running lol

I also noticed since you're (likely) using a template with this container manually - you need to manually expose the HTTP ports you want and add 22 to the TCP ports for ssh without the proxy.

[limber dirge]

When I tried it in morning today I got 502 error when accessing jupyter lab at port 8888

[limber dirge]

I exposed the ports but not sure about 22 TCP doesn't it add that by default?

[full island]

Runpod offical templates do, maintaining your own template doesn't

I had to add the port and my public key manually which sucked so I'll work on making that a cleaner setup (I was thinking a button in the templates page that says "enable ssh")

[full island]

Let me check again, I'll let you know here soon.

[limber dirge]

Do I also need to add any CMD commands in the template for it to work properly?

[full island]

No, our official template just does these

[limber dirge]

Thanks I will give this a try again tomorrow morning

[full island]

I found the bug here, I'll trigger a couple CI jobs to release the fix as:

PyTorch 2.7.0:
runpod/pytorch:0.7.2-dev-ubuntu2204-cu1281-torch270

PyTorch 2.7.1:
runpod/pytorch:0.7.2-dev-ubuntu2204-cu1281-torch271

[limber dirge]

This is not fixed yet right?