7.7. 配置 Guardrails Detector Hugging Face service runtime

要使用 Hugging Face 模型的子集，称为 AutoModelsForSequenceClassification 和 Guardrails Orchestrator，您需要首先配置 Hugging Face service runtime。

guardrails-detector-huggingface-runtime 是 Hugging Face 模型的 KServe 服务运行时，用于检测和缓解文本数据中的某些类型的风险，如 Red Hateful speech。这个运行时与大多数 Hugging Face AutoModelsForSequenceClassification 模型兼容，并允许 ibm-granite/granite-guardian-hap-38m 等模型在 TrustyAI Guardrails 生态系统中使用。

自定义服务运行时示例

此 YAML 文件包含一个自定义服务运行时示例，它带有四个 worker 用于 Prompt Injection detector：

apiVersion: serving.kserve.io/v1alpha1
kind: ServingRuntime
metadata:
  name: guardrails-detector-runtime-prompt-injection
  annotations:
    openshift.io/display-name: Guardrails Detector ServingRuntime for KServe
    opendatahub.io/recommended-accelerators: '["nvidia.com/gpu"]'
  labels:
    opendatahub.io/dashboard: 'true'
spec:
  annotations:
    prometheus.io/port: '8080'
    prometheus.io/path: '/metrics'
  multiModel: false
  supportedModelFormats:
    - autoSelect: true
      name: guardrails-detector-huggingface
  containers:
    - name: kserve-container
      image: quay.io/trustyai/guardrails-detector-huggingface-runtime:v0.2.0
      command:
        - uvicorn
        - app:app
      args:
        - "--workers=4"  # Override default
        - "--host=0.0.0.0"
        - "--port=8000"
        - "--log-config=/common/log_conf.yaml"
      env:
        - name: MODEL_DIR
          value: /mnt/models
        - name: HF_HOME
          value: /tmp/hf_home
      ports:
        - containerPort: 8000
          protocol: TCP

apiVersion: serving.kserve.io/v1alpha1
kind: ServingRuntime
metadata:
  name: guardrails-detector-runtime-prompt-injection
  annotations:
    openshift.io/display-name: Guardrails Detector ServingRuntime for KServe
    opendatahub.io/recommended-accelerators: '["nvidia.com/gpu"]'
  labels:
    opendatahub.io/dashboard: 'true'
spec:
  annotations:
    prometheus.io/port: '8080'
    prometheus.io/path: '/metrics'
  multiModel: false
  supportedModelFormats:
    - autoSelect: true
      name: guardrails-detector-huggingface
  containers:
    - name: kserve-container
      image: quay.io/trustyai/guardrails-detector-huggingface-runtime:v0.2.0
      command:
        - uvicorn
        - app:app
      args:
        - "--workers=4"  # Override default
        - "--host=0.0.0.0"
        - "--port=8000"
        - "--log-config=/common/log_conf.yaml"
      env:
        - name: MODEL_DIR
          value: /mnt/models
        - name: HF_HOME
          value: /tmp/hf_home
      ports:
        - containerPort: 8000
          protocol: TCP

Copy to Clipboard

Toggle word wrap

下表描述了 Guardrails Detector Hugging Face service runtime 的配置值：

Expand

表 7.1. 模板配置
属性	value
模板名称	`guardrails-detector-huggingface-serving-template`
运行时名称	`guardrails-detector-huggingface-runtime`
显示名称	`Hugging Face Detector ServingRuntime for KServe`
模型格式	`guardrails-detector-hf-runtime`

Expand

表 7.2. 服务器配置
组件	配置	value
服务器	uvicorn	`app:app`
port	Container	`8000`
指标端口	Prometheus	`8080`
指标路径	Prometheus	`/metrics`
日志配置	路径	`/common/log_conf.yaml`

Expand

表 7.3. 参数
参数	默认	描述
`guardrails-detector-huggingface-runtime-image`	-	容器镜像（必需）
`MODEL_DIR`	`/mnt/models`	模型挂载路径
`HF_HOME`	`/tmp/hf_home`	HuggingFace 缓存
`--workers`	`1`	Uvicorn worker
`--host`	`0.0.0.0`	服务器绑定地址
`--port`	`8000`	服务器端口

Expand

表 7.4. API 端点的参数
端点	方法	描述	Content-Type	Headers
`/health`	GET	健康检查端点	`-`	`-`
`/api/v1/text/contents`	POST	内容检测端点	`application/json`	3 types: * `application/json` * `detector-id: {detector_name}` * `Content-Type: application/json`

7.7. 配置 Guardrails Detector Hugging Face service runtime

学习

尝试、购买和销售

社区

关于红帽文档

让开源更具包容性

關於紅帽

Theme

Red Hat legal and privacy links

Red Hat legal and privacy links