How to run and deploy LLMs using Red Hat OpenShift AI on a Red Hat OpenShift Service on AWS cluster

Learn how to install the Red Hat® OpenShift® AI (RHOAI) operator and Jupyter notebook, create an Amazon S3 bucket, and run the LLM model on a Red Hat OpenShift Service on AWS (ROSA) cluster.

Disclaimer: this content is authored by Red Hat experts, but has not yet been tested on every supported configuration.

This learning path is for operations teams or system administrators.

Developers might want to check out how to create a natural language processing (NLP) application using Red Hat OpenShift AI on developers.redhat.com.

Get started on developers.redhat.com

Creating and granting access to Amazon S3 bucket

10 mins

There are actually several ways to go about granting S3 access to the pods running in your Red Hat® OpenShift® Service on AWS (ROSA) cluster. For example, you can set the credentials as environment variables in the notebook using pod identity/Identity Access Management (IAM) Roles for Service Accounts (IRSA) to authenticate the pods to S3. Or, you can install the AWS command line interface (CLI) in the cluster, among others. 

For the sake of simplicity, we’ll install the CLI in the cluster and then use the command aws configure to provide the credentials. Be sure that you have your AWS access key and secret access key handy. You could create new keys in the IAM section from the AWS console if you have lost yours.  

What will you learn?

  • Creating the Amazon S3 bucket
  • Granting access to the Amazon S3 bucket

What do you need before starting?

  • Met all prerequisites
  • Have AWS access key and secret access key

Steps to create the Amazon S3 bucket

  1. Now log into your cluster and go to the namespace where your notebook is located: oc project rhods-notebooks
  2. Run the following command and make sure that your pods are running: oc get pods
  3. Once you have the name of the pod (in this case it is called jupyter-nb-admin-0), execute into it: 
    oc exec -it jupyter-nb-admin-0 -- /bin/bash
  4. Next, let's install the AWS CLI in that pod:

    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    unzip awscliv2.zip
    ./aws/install -i ~/.local/aws-cli -b ~/.local/bin
  5. Then modify your PATH environment: export PATH=~/.local/bin:$PATH
  6. Once it is correctly installed, be sure that you have your AWS Access Key ID and AWS Secret Access Key ready, and run the following command: aws configure
  7. Select the region where your cluster is located. You could verify the configuration by running simple command such as listing the S3 buckets: aws s3 ls
  8. Once the credentials matter is sorted, let's create an S3 bucket in your AWS account. Again, there are many ways to go about this. The easiest would be to go to your AWS console and create the bucket in your region from there and leave all the settings to default. Alternatively, you can run this command to create a bucket (in this case, it’s called llm-bucket-dsari, and the cluster region is us-west-2): aws s3 mb s3://llm-bucket-dsari --region us-west-2

Once these steps are complete, you’re ready to begin training the LLM model in the next resource. 

Red Hat logoGithubredditYoutubeTwitter

詳細情報

試用、購入および販売

コミュニティー

会社概要

Red Hat は、企業がコアとなるデータセンターからネットワークエッジに至るまで、各種プラットフォームや環境全体で作業を簡素化できるように、強化されたソリューションを提供しています。

多様性を受け入れるオープンソースの強化

Red Hat では、コード、ドキュメント、Web プロパティーにおける配慮に欠ける用語の置き換えに取り組んでいます。このような変更は、段階的に実施される予定です。詳細情報: Red Hat ブログ.

Red Hat ドキュメントについて

Legal Notice

Theme

© 2026 Red Hat
トップに戻る