Red Hat SummitEste contenido no está disponible en el idioma seleccionado.
Chapter 3. Red Hat Enterprise Linux AI hardware requirements
Various hardware accelerators require different requirements for serving and inferencing as well as installing, generating and training the granite-7b-starter model on Red Hat Enterprise Linux AI.
3.1. Hardware requirements for end-to-end workflow of Granite models
The following charts show the hardware requirements for running the full InstructLab end-to-end workflow to customize the Granite student model. This includes: synthetic data generation (SDG), training, and evaluating a custom Granite model.
3.1.1. Bare metal
| Hardware vendor | Supported accelerators (GPUs) | Aggregate GPU memory | Recommended additional disk storage |
|---|---|---|---|
| NVIDIA | 2xA100 4xA100 8xA100 | 160 GB 320 GB 640 GB | 1 TB |
| NVIDIA | 2xH100 4xH100 8xH100 | 160 GB 320 GB 640 GB | 1 TB |
| NVIDIA | 4xL40S 8xL40S | 192 GB 384 GB | 1 TB |
3.1.2. Amazon Web Services (AWS)
| Hardware vendor | Supported accelerators (GPUs) | Aggregate GPU Memory | AWS Instance | Recommended additional disk storage |
|---|---|---|---|---|
| NVIDIA | 8xA100 | 640 GB | p4de.24xlarge | 1 TB |
| NVIDIA | 8xH100 | 640 GB | p5.48xlarge | 1 TB |
3.2. Hardware requirements for inference serving Granite models
The following charts display the minimum hardware requirements for inference serving a model on Red Hat Enterprise Linux AI.
3.2.1. Bare metal
| Hardware vendor | Supported accelerators (GPUs) | minimum Aggregate GPU memory | Recommended additional disk storage |
|---|---|---|---|
| NVIDIA | A100 | 80 GB | 1 TB |
| NVIDIA | H100 | 80 GB | 1 TB |
| NVIDIA | L40S | 48 GB | 1 TB |
| NVIDIA | L4 | 24 GB | 1 TB |
3.2.2. Amazon Web Services (AWS)
| Hardware vendor | Supported accelerators (GPUs) | Minimum Aggregate GPU Memory | Recommended additional disk storage |
|---|---|---|---|
| NVIDIA | A100 | 80 GB | 1 TB |
| NVIDIA | H100 | 80 GB | 1 TB |
| NVIDIA | L40S | 48 GB | 1 TB |
| NVIDIA | L4 | 24 GB | 1 TB |
3.2.3. IBM cloud
| Hardware vendor | Supported accelerators (GPUs) | Minimum Aggregate GPU Memory | Recommended additional disk storage |
|---|---|---|---|
| NVIDIA | L40S | 48 GB | 1 TB |
| NVIDIA | L4 | 24 GB | 1 TB |
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}