Unable to run jupyter on custom docker image

[tender saddle]

I am basically trying to build my own custom container and run jupyter lab in it. I've copied the configuration from base rundpod image from github. Everything is fine there. The problem occurs when the pod is booted up as it is supposed to start jupyterlab but we get the jupyter command not found issue which is why port 8888 is not ready. First screenshot is from my local machine where i get into the docker and i see jupyter --help works. Second one is from the runpod instance where it complains that jupyter is not found but then when i install in my conda env it only updates certifi version (as you can see from the screenshot) and then after that jupyter command is available again. I'm puzzle by this behaviour. How can a same docker container behave differently in different environments?

[lyric owl]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[vestal mason]

Jupyter Lab and its dependencies may not be properly managed or installed in the container's environment, leading to the "jupyter command not found" issue.

do you mean you need to reinstall the jupyter everytime you start the pod?

hows your dockerfile, and can i see also the script that installs your jupyterlab

[tender saddle]

the first screenshot is from my docker container running locally. As you can see jupyter is installed. it works fine there. The command not found is happening only on runpod

I've digged deep into the start.sh that runpod has, it invokes jupyter if JUPYTER_PASSWORD set. https://github.com/runpod/containers/blob/main/container-template/start.sh#L75

However setting environment variables is happening after that:

https://github.com/runpod/containers/blob/main/container-template/start.sh#L96

I'm wondering how that lines gets invoked. Right now i have to open a shell into my pod, do conda install -y jupyter and it just barely updates a file called certifi and then jupyter command can be found. so I launch it and connect it.

Perhaps something to do with start jupyter notebook button that default runpod templates have during gpu deployment screen. I don't have that option since I have a custom template.

This is the checkbox i mean

[vestal mason]

Check the pod after deployment does it generate one automatically? I think it does

[tender saddle]

I have the port link but it always says port is not ready. When i click it's HTTP 502 which is why i have to open the web terminal, run 1-2 commands and then it becomes ready

Here is my dockerfile for reference:

ARG MINICONDA_VERSION=24.4.0-0

FROM continuumio/miniconda3:${MINICONDA_VERSION}

SHELL ["/bin/bash", "-o", "pipefail", "-c"]

ENV DEBIAN_FRONTEND=noninteractive
ENV SHELL=/bin/bash

WORKDIR /

RUN mkdir /workspace

# git, wget is installed in the base image
RUN apt-get update --yes && \
    apt-get upgrade --yes && \
    apt install --yes --no-install-recommends curl libgl1 software-properties-common openssh-server nginx && \
    apt-get autoremove -y && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* && \
    echo "en_US.UTF-8 UTF-8" > /etc/locale.gen

COPY requirements/ requirements/

RUN conda env create -n py -f requirements/dev/environment-gpu-full.yml && \
    conda clean -afy

# Activate conda env inside
SHELL ["conda", "run", "-n", "py", "/bin/bash", "-c"]

# This line is necessary to run the container interactively
RUN echo "conda activate py" >> ~/.bashrc

# Remove existing SSH host keys
RUN rm -f /etc/ssh/ssh_host_*

# NGINX Proxy
COPY scripts/runpod/docker/nginx.conf /etc/nginx/nginx.conf
COPY scripts/runpod/docker/readme.html /usr/share/nginx/html/readme.html

# Start Scripts
COPY scripts/runpod/docker/start.sh /
RUN chmod +x /start.sh

COPY . /workspace

# Set the default command for the container
CMD [ "/start.sh" ]

start.sh, nginx.conf is copied from the runpod github repo.

[vestal mason]

What commands is it

Can you add it on the start script

[tender saddle]

I have done this on runpod (overriding the start cmd)

bash -c 'source ~/.bashrc && conda activate py && conda install -y jupyter && ./start.sh'

but somehow i still needed webterminal launch jupyter. 🤷‍♂️

[vestal mason]

Maybe it's not run?, it is better to put it inside your docker file

[vale cloud]

@tender saddle did you end up figuring this out? im also having the same problem

@tender saddle @vestal mason Start Jupyter Notebook boolean is not even appearing for me on the deployment configuration

[vale cloud]

added this inside my dockerfile:

`RUN echo '#!/bin/bash \n
echo "Starting Jupyter Lab..." \n
cd / && \n
conda run -n automasher jupyter lab --allow-root \n
--no-browser \n
--port=8888 \n
--ip=* \n
--FileContentsManager.delete_to_trash=False \n
--ContentsManager.allow_hidden=True \n
--ServerApp.terminado_settings='"'"'{"shell_command":["/bin/bash"]}'"'"' \n
--ServerApp.token="" \n
--ServerApp.password="" \n
--ServerApp.allow_origin=* \n
--ServerApp.preferred_dir=/workspace &> /workspace/logs/jupyter.log & \n
echo "Jupyter Lab started" \n
conda activate automasher \n
tail -f /dev/null' > /start_jupyter.sh &&
chmod +x /start_jupyter.sh

ENTRYPOINT ["/start_jupyter.sh"]`

i must be doing somethign wrong

[vestal mason]

You don't need to use the start jupyter notebook switch,just hardcode it if you want to start jupyter everytime

[tender saddle]

You have to launch the pod as is and port will be available after a while if your configuration is correct. The boolean box is somehow for rundpod template so it won't appear for custom dockers

[vale cloud]

what does just hardcode it mean?

also how do i know if my configuration is correct?

[vestal mason]

Hard code to start your jupyter without any switch or conditions

[vale cloud]

got it to work, thanks guys