Chapter 2. Creating a modelcar image and pushing it to a container image registry

Procedure

1. Create a Python virtual environment and install the huggingface_hub Python library:

   python3 -m venv venv && \
   source venv/bin/activate && \
   pip install --upgrade pip && \
   pip install huggingface_hub

2. Create a model downloader Python script:

   vi download_model.py

3. Add the following content to the download_model.py file, adjusting the value for model_repo as required:

   from huggingface_hub import snapshot_download
   
   # Specify the Hugging Face repository containing the model
   model_repo = "ibm-granite/granite-3.1-2b-instruct"
   snapshot_download(
       repo_id=model_repo,
       local_dir="/models",
       allow_patterns=["*.safetensors", "*.json", "*.txt"],
   )

4. Create a Dockerfile for the modelcar:

   FROM registry.access.redhat.com/ubi9/python-311:latest as base
   
   USER root
   
   RUN pip install huggingface-hub
   
   # Download the model file from Hugging Face
   COPY download_model.py .
   
   RUN python download_model.py
   
   # Final image containing only the essential model files
   FROM registry.access.redhat.com/ubi9/ubi-micro:9.4
   
   # Copy the model files from the base container
   COPY --from=base /models /models
   
   USER 1001

5. Build the modelcar image:

   podman build . -t modelcar-example:latest --platform linux/amd64

   Example output

   Successfully tagged localhost/modelcar-example:latest

6. Push the modelcar image to the container registry. For example:

   $ podman push modelcar-example:latest quay.io/<your_model_registry>/modelcar-example:latest

   Example output

   Getting image source signatures
   Copying blob b2ed7134f853 done
   Copying config 4afd393610 done
   Writing manifest to image destination
   Storing signatures