Собрать и использовать кастомный Docker-образ для Jupyter Server

Пример создания кастомного образа из контейнеров NVIDIA

# Use NVIDIA's PyTorch image based on version 22.12 with Python 3
FROM nvcr.io/nvidia/pytorch:22.12-py3

# Set the time zone for the container to Moscow time
ENV TZ=Europe/Moscow
# Update time zone information in the system
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

# Specifies that all subsequent commands must be executed as the root user
USER root

# Create a new group 'jovyan' and a new user 'jovyan' who is a member of this group
RUN groupadd -g 1000 jovyan
RUN useradd -g jovyan -u 1000 -m jovyan

# Create the necessary directories and change the owner to 'jovyan'
RUN mkdir -p /tmp/.jupyter_data && chown -R jovyan /tmp/.jupyter_data && \
    mkdir -p /tmp/.jupyter && chown -R jovyan /tmp/.jupyter
RUN mkdir -p /home/jovyan && chown -R jovyan /home/jovyan

# Copy the 'InternImage' directory to the container and changing the owner to 'jovyan'
COPY InternImage /InternImage
RUN chown -R jovyan /InternImage

# Update the list of packages in the system and already installed packages
RUN apt-get update --fix-missing && apt-get upgrade -y  &&\
    echo "8" apt-get install -y software-properties-common && apt-get update

# Install the necessary programs and libraries
RUN DEBIAN_FRONTEND=noninteractive apt-get install tzdata -qy &&\
    apt install -qy \
        python3-pip \
        build-essential \
        cmake \
        git \
        wget \
        zip \
        unzip \
        unrar \
        yasm \
        python3-dev \
        nano \
        vim \
        neovim \
        pkg-config \
        ffmpeg \
        libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio &&\
    apt upgrade -qy &&\
    apt clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Set environment variables for Python
ENV PYTHONUSERPATH="/home/user/.local"
ENV PATH=$PYTHONHOME/bin:$PATH

# Install Python libraries that will be required for work
RUN pip3 install -U --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
RUN pip3 install -U --no-cache-dir \
    openmim \
    opencv-python-headless \
    timm \
    mmdet \
    termcolor \
    yacs \
    pyyaml \
    scipy \
    pyyaml \
    transformers \
    huggingface_hub \
    safetensors \
    Pillow \
    psutil \
    PyYAML \
    requests \
    thop \
    tqdm \
    matplotlib

# Change user to 'jovyan' again
USER jovyan

# Launch Jupyter Notebook with certain settings when starting the container
CMD jupyter notebook \
              --notebook-dir=/home/jovyan \
              --ip=0.0.0.0 \
              --no-browser \
              --allow-root \
              --port=8888 \
              --NotebookApp.token='' \
              --NotebookApp.password='' \
              --NotebookApp.base_url=${NB_PREFIX} \
              --NotebookApp.allow_origin='https://*.ai.cloud.ru'

Фрагмент Docker-файла для добавления SSH

1. Создайте пользователя и группу:

   1. Создайте группу jovyan с GID 1000.

   2. Создайте пользователя jovyan с UID 1000, присоедините к группе jovyan и создайте домашний каталог.

2. Установите OpenSSH:

   1. Обновите пакеты в вашем контейнере.

   2. Установите openssh-server.

   3. Создайте необходимый каталог для sshd.

3. Настройте SSH:

   1. Измените конфигурацию SSH, чтобы отключить проверку ключей хоста (StrictHostKeyChecking no).

   2. Дайте права на чтение всем пользователям для всех файлов в /etc/ssh/.

   3. Отключите PAM в конфигурации SSH (UsePAM no).

   4. Установите PubkeyAcceptedAlgorithms для использования алгоритма ssh-rsa.

   5. Настройте авторизацию в параметрах конфигурации:

      AuthorizedKeysCommand /usr/bin/cat /etc/ssh/.ssh_aicloud/%u_keys
      AuthorizedKeysCommandUser %u
      
      

   6. Установите порт SSH на 2222.

4. Установите права доступа к файлам:

   1. Установите права доступа 600 для всех файлов в /etc/ssh/.

   2. Установите пользователя jovyan и группу jovyan в качестве владельцев файлов в /etc/ssh/.

5. Настройте запуск сервера SSH и Jupyter Notebook на необходимых портах с определенными конфигурационными параметрами (CMD).

6. Соберите образ из Docker-файла с помощью команды docker build, указав путь к вашему Docker-файлу.

   Пример: docker build -t my-image-name:path-to-your-dockerfile.

   Фрагмент Docker-файла для добавления SSH

   # Create group with ID 1000 called 'jovyan'
   RUN groupadd -g 1000 jovyan
   
   # Creates a user with ID 1000 belonging to the group 'jovyan' and creates a home directory
   RUN useradd -g jovyan -u 1000 -m jovyan
   
   # Updates the list of packages in the repositories Ubuntu
   RUN apt-get update
   
   # Installs OpenSSH Server to enable communication between containers via MPI (Message Passing Interface)
   RUN apt-get install -y --no-install-recommends openssh-server && \
       mkdir -p /var/run/sshd
   
   # Sets SSH setup to connect to containers without checking the host key
   RUN cat /etc/ssh/ssh_config | grep -v StrictHostKeyChecking > /etc/ssh/ssh_config.new && \
       echo "    StrictHostKeyChecking no" >> /etc/ssh/ssh_config.new && \
       mv /etc/ssh/ssh_config.new /etc/ssh/ssh_config
   
   # Grants read rights to SSH configuration files to all users
   RUN chmod -R o+r /etc/ssh/*
   
   # Disables use PAM (Pluggable Authentication Modules) in SSHD
   RUN cat /etc/ssh/sshd_config | grep -v UsePAM > /etc/ssh/sshd_config.new && \
       echo "UsePAM no" >> /etc/ssh/sshd_config.new && \
       mv /etc/ssh/sshd_config.new /etc/ssh/sshd_config
   
   # Sets key authentication algorithms, allowing ssh-rsa
   RUN cat /etc/ssh/sshd_config | grep -v "PubkeyAcceptedAlgorithms " > /etc/ssh/sshd_config.new && \
       echo "PubkeyAcceptedAlgorithms +ssh-rsa" >> /etc/ssh/sshd_config.new && \
       mv /etc/ssh/sshd_config.new /etc/ssh/sshd_config
   
   # Set up authorization
   RUN echo "AuthorizedKeysCommand /usr/bin/cat /etc/ssh/.ssh_aicloud/%u_keys" >> /etc/ssh/sshd_config
   RUN echo "AuthorizedKeysCommandUser %u" >> /etc/ssh/sshd_config
   
   # Sets port 2222 for SSH
   RUN cat /etc/ssh/sshd_config | grep -v "Port " > /etc/ssh/sshd_config.new && \
       echo "Port 2222" >> /etc/ssh/sshd_config.new && \
       mv /etc/ssh/sshd_config.new /etc/ssh/sshd_config
   
   # Sets SSH file permissions and changes owner to 'jovyan'
   RUN chmod 600 /etc/ssh/* && \
       chown jovyan:jovyan /etc/ssh/*
   
   # Starts the SSHD server and Jupyter Notebook with the specified parameters
   CMD /usr/sbin/sshd && jupyter notebook \
                       --notebook-dir=/home/jovyan \
                       --ip=0.0.0.0 \
                       --no-browser \
                       --allow-root \
                       --port=8888 \
                       --NotebookApp.token='' \
                       --NotebookApp.password='' \
                       --NotebookApp.base_url=${NB_PREFIX} \
                       --NotebookApp.allow_origin='https://*.ai.cloud.ru'
   
   

Создание и локальная проверка Docker-образа

1. Создайте Docker-образ локально:

docker build --tag 'jupyter-custom-image' .

docker run -p 8888:8888 -e NB_PREFIX="/notebook/ai000001-00001/mynotebookname" -it jupyter-custom-image