このコンテンツは選択した言語では利用できません。

Chapter 3. Red Hat Enterprise Linux AI hardware requirements


Various hardware accelerators require different requirements for serving and inferencing as well as installing, generating and training the Granite starter model on Red Hat Enterprise Linux AI.

3.1. Hardware requirements for end-to-end workflow of Granite models

The following charts show the hardware requirements for running the full InstructLab end-to-end workflow to customize the Granite student model. This includes: synthetic data generation (SDG), multi-phase training, and evaluating a custom Granite model.

3.1.1. Bare metal

Hardware vendorSupported accelerators (GPUs)Aggregate GPU memoryRecommended additional disk storage

NVIDIA

2xA100

4xA100

8xA100

160 GB

320 GB

640 GB

3 TB

NVIDIA

2xH100

4xH100

8xH100

160 GB

320 GB

640 GB

3 TB

NVIDIA

4xL40S

8xL40S

192 GB

384 GB

3 TB

AMD (Technology preview)

2xMI300X

4xMI300X

8xMI300X

384 GB

768 GB

1536 GB

3 TB

3.1.2. IBM Cloud

Hardware vendorSupported accelerators (GPUs)Aggregate GPU MemoryIBM Cloud InstancesRecommended additional disk storage

NVIDIA

8xH100

640 GB

gx3d-160x1792x8h100

3 TB

3.1.3. Amazon Web Services (AWS)

Hardware vendorSupported accelerators (GPUs)Aggregate GPU MemoryAWS InstancesRecommended additional disk storage

NVIDIA

8xA100

320 GB

p4d.24xlarge

3 TB

NVIDIA

8xA100

640 GB

p4de.24xlarge

3 TB

NVIDIA

8xH100

640 GB

p5.48xlarge

3 TB

NVIDIA

8xL40S

384 GB

g6e.48xlarge

3 TB

3.1.4. Azure

Hardware vendorSupported accelerators (GPUs)Aggregate GPU MemoryAzure InstancesRecommended additional disk storage

NVIDIA

8xA100

640 GB

Standard_ND96amsr_A100_v4

3 TB

NVIDIA

4xA100

320 GB

Standard_ND96asr_A100_v4

3 TB

NVIDIA

8xH100

640 GB

Standard_ND96isr_H100_v5

3 TB

AMD (Technology preview)

8xMI300X

1536 GB

Standard_ND96isr_MI300X_v5

3 TB

3.1.5. Google Cloud Platform (GCP)

Hardware vendorSupported accelerators (GPUs)Aggregate GPU MemoryGCP InstancesRecommended additional disk storage

NVIDIA

8xA100

640 GB

a2-highgpu-8g

3 TB

NVIDIA

8xH100

640 GB

a3-highgpu-8g

a3-megagpu-8g

3 TB

3.2. Hardware requirements for inference serving Granite models

The following charts display the minimum hardware requirements for inference serving a model on Red Hat Enterprise Linux AI.

3.2.1. Bare metal

Hardware vendorSupported accelerators (GPUs)minimum Aggregate GPU memoryRecommended additional disk storage

NVIDIA

A100

80 GB

1 TB

NVIDIA

H100

80 GB

1 TB

NVIDIA

L40S

48 GB

1 TB

NVIDIA

L4

24 GB

1 TB

AMD (Technology preview)

MI300X

192 GB

1 TB

3.2.2. Amazon Web Services (AWS)

Hardware vendorSupported accelerators (GPUs)Minimum Aggregate GPU MemoryAWS Instance familyRecommended additional disk storage

NVIDIA

A100

40 GB

P4d series

1 TB

NVIDIA

H100

80 GB

P5 series

1 TB

NVIDIA

L40S

48 GB

G6e series

1 TB

NVIDIA

L4

24 GB

G6 series

1 TB

3.2.3. IBM cloud

Hardware vendorSupported accelerators (GPUs)Minimum Aggregate GPU MemoryIBM Cloud Instance familyRecommended additional disk storage

NVIDIA

L40S

48 GB

gx series

1 TB

NVIDIA

L4

24 GB

gx series

1 TB

3.2.4. Azure

Hardware vendorSupported accelerators (GPUs)Minimum Aggregate GPU MemoryAzure Instance familyRecommended additional disk storage

NVIDIA

A100

80 GB

ND series

1 TB

NVIDIA

H100

80 GB

ND sereis

1 TB

AMD (Technology preview)

MI300X

192 GB

ND series

1 TB

3.2.5. Google Cloud Platform (GCP)

Hardware vendorSupported accelerators (GPUs)Minimum Aggregate GPU MemoryGCP Instance familyRecommended additional disk storage

NVIDIA

A100

40 GB

A2 series

1 TB

NVIDIA

H100

80 GB

A3 series

1 TB

NVIDIA

4xL4

96 GB

G2 series

1 TB

Red Hat logoGithubRedditYoutubeTwitter

詳細情報

試用、購入および販売

コミュニティー

Red Hat ドキュメントについて

Red Hat をお使いのお客様が、信頼できるコンテンツが含まれている製品やサービスを活用することで、イノベーションを行い、目標を達成できるようにします。

多様性を受け入れるオープンソースの強化

Red Hat では、コード、ドキュメント、Web プロパティーにおける配慮に欠ける用語の置き換えに取り組んでいます。このような変更は、段階的に実施される予定です。詳細情報: Red Hat ブログ.

会社概要

Red Hat は、企業がコアとなるデータセンターからネットワークエッジに至るまで、各種プラットフォームや環境全体で作業を簡素化できるように、強化されたソリューションを提供しています。

© 2024 Red Hat, Inc.