How to run and deploy LLMs using Red Hat OpenShift AI on a Red Hat OpenShift Service on AWS cluster

Learn how to install the Red Hat® OpenShift® AI (RHOAI) operator and Jupyter notebook, create an Amazon S3 bucket, and run the LLM model on a Red Hat OpenShift Service on AWS (ROSA) cluster.

Disclaimer: this content is authored by Red Hat experts, but has not yet been tested on every supported configuration.

This learning path is for operations teams or system administrators.

Developers might want to check out how to create a natural language processing (NLP) application using Red Hat OpenShift AI on developers.redhat.com.

Get started on developers.redhat.com

Creating and granting access to Amazon S3 bucket

10 mins

There are actually several ways to go about granting S3 access to the pods running in your Red Hat® OpenShift® Service on AWS (ROSA) cluster. For example, you can set the credentials as environment variables in the notebook using pod identity/Identity Access Management (IAM) Roles for Service Accounts (IRSA) to authenticate the pods to S3. Or, you can install the AWS command line interface (CLI) in the cluster, among others. 

For the sake of simplicity, we’ll install the CLI in the cluster and then use the command aws configure to provide the credentials. Be sure that you have your AWS access key and secret access key handy. You could create new keys in the IAM section from the AWS console if you have lost yours.  

What will you learn?

  • Creating the Amazon S3 bucket
  • Granting access to the Amazon S3 bucket

What do you need before starting?

  • Met all prerequisites
  • Have AWS access key and secret access key

Steps to create the Amazon S3 bucket

  1. Now log into your cluster and go to the namespace where your notebook is located: oc project rhods-notebooks
  2. Run the following command and make sure that your pods are running: oc get pods
  3. Once you have the name of the pod (in this case it is called jupyter-nb-admin-0), execute into it: 
    oc exec -it jupyter-nb-admin-0 -- /bin/bash
  4. Next, let's install the AWS CLI in that pod:

    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    unzip awscliv2.zip
    ./aws/install -i ~/.local/aws-cli -b ~/.local/bin
  5. Then modify your PATH environment: export PATH=~/.local/bin:$PATH
  6. Once it is correctly installed, be sure that you have your AWS Access Key ID and AWS Secret Access Key ready, and run the following command: aws configure
  7. Select the region where your cluster is located. You could verify the configuration by running simple command such as listing the S3 buckets: aws s3 ls
  8. Once the credentials matter is sorted, let's create an S3 bucket in your AWS account. Again, there are many ways to go about this. The easiest would be to go to your AWS console and create the bucket in your region from there and leave all the settings to default. Alternatively, you can run this command to create a bucket (in this case, it’s called llm-bucket-dsari, and the cluster region is us-west-2): aws s3 mb s3://llm-bucket-dsari --region us-west-2

Once these steps are complete, you’re ready to begin training the LLM model in the next resource. 

Red Hat logoGithubredditYoutubeTwitter

자세한 정보

평가판, 구매 및 판매

커뮤니티

Red Hat 문서 정보

Red Hat을 사용하는 고객은 신뢰할 수 있는 콘텐츠가 포함된 제품과 서비스를 통해 혁신하고 목표를 달성할 수 있습니다. 최신 업데이트를 확인하세요.

보다 포괄적 수용을 위한 오픈 소스 용어 교체

Red Hat은 코드, 문서, 웹 속성에서 문제가 있는 언어를 교체하기 위해 최선을 다하고 있습니다. 자세한 내용은 다음을 참조하세요.Red Hat 블로그.

Red Hat 소개

Red Hat은 기업이 핵심 데이터 센터에서 네트워크 에지에 이르기까지 플랫폼과 환경 전반에서 더 쉽게 작업할 수 있도록 강화된 솔루션을 제공합니다.

Theme

© 2026 Red Hat
맨 위로 이동