Chapter 2. Creating a modelcar image and pushing it to a container image registry

Procedure

1. Create a Python virtual environment and install the huggingface_hub Python library:

   python3 -m venv venv && \
   source venv/bin/activate && \
   pip install --upgrade pip && \
   pip install huggingface_hub

   Copy to Clipboard Toggle word wrap

2. Create a model downloader Python script:

   vi download_model.py

   Copy to Clipboard Toggle word wrap

3. Add the following content to the download_model.py file, adjusting the value for model_repo as required:

   from huggingface_hub import snapshot_download
   
   # Specify the Hugging Face repository containing the model
   model_repo = "ibm-granite/granite-3.1-2b-instruct"
   snapshot_download(
       repo_id=model_repo,
       local_dir="/models",
       allow_patterns=["*.safetensors", "*.json", "*.txt"],
   )

   Copy to Clipboard Toggle word wrap

4. Create a Dockerfile for the modelcar:

   FROM registry.access.redhat.com/ubi9/python-311:latest as base
   
   USER root
   
   RUN pip install huggingface-hub
   
   # Download the model file from Hugging Face
   COPY download_model.py .
   
   RUN python download_model.py
   
   # Final image containing only the essential model files
   FROM registry.access.redhat.com/ubi9/ubi-micro:9.4
   
   # Copy the model files from the base container
   COPY --from=base /models /models
   
   USER 1001

   Copy to Clipboard Toggle word wrap

5. Build the modelcar image:

   podman build . -t modelcar-example:latest --platform linux/amd64

   Copy to Clipboard Toggle word wrap

   Example output

   Successfully tagged localhost/modelcar-example:latest

   Copy to Clipboard Toggle word wrap

6. Push the modelcar image to the container registry. For example:

   $ podman push modelcar-example:latest quay.io/<your_model_registry>/modelcar-example:latest

   Copy to Clipboard Toggle word wrap

   Example output

   Getting image source signatures
   Copying blob b2ed7134f853 done
   Copying config 4afd393610 done
   Writing manifest to image destination
   Storing signatures

   Copy to Clipboard Toggle word wrap