이 콘텐츠는 선택한 언어로 제공되지 않습니다.

Chapter 11. Recover from an out-of-sync passive site


This chapter describes the procedures required to synchronize the secondary site with the primary site in a setup as outlined in Concepts for active-passive deployments together with the blueprints outlined in Building blocks active-passive deployments.

11.1. When to use procedure

Use this after a temporary disconnection between sites where Data Grid was disconnected and the contents of the caches are out-of-sync.

At the end of the procedure, the session contents on the secondary site have been discarded and replaced by the session contents of the primary site. All caches in the secondary site have been cleared to prevent invalid cached contents.

See the Multi-site deployments chapter for different operational procedures.

11.2. Procedures

11.2.1. Data Grid Cluster

For the context of this chapter, Site-A is the primary site and is active, and Site-B is the secondary site and is passive.

Network partitions may happen between the site and the replication between the Data Grid cluster will stop. These procedures bring both sites back in sync.

Warning

Transferring the full state may impact the Data Grid cluster performance by increasing the response time and/or resources usage.

The first procedure is to delete the stale data from the secondary site.

  1. Login into your secondary site.
  2. Shutdown Red Hat build of Keycloak. This will clear all Red Hat build of Keycloak caches, and it prevents the state of Red Hat build of Keycloak from being out-of-sync with Data Grid.

    When deploying Red Hat build of Keycloak using the Red Hat build of Keycloak Operator, change the number of Red Hat build of Keycloak instances in the Red Hat build of Keycloak Custom Resource to 0.

  3. Connect into Data Grid Cluster using the Data Grid CLI tool:

    Command:

    oc -n keycloak exec -it pods/infinispan-0 -- ./bin/cli.sh --trustall --connect https://127.0.0.1:11222
    Copy to Clipboard Toggle word wrap

    It asks for the username and password for the Data Grid cluster. Those credentials are the one set in the Deploy Data Grid for HA with the Data Grid Operator chapter in the configuring credentials section.

    Output:

    Username: developer
    Password:
    [infinispan-0-29897@ISPN//containers/default]>
    Copy to Clipboard Toggle word wrap

    Note

    The pod name depends on the cluster name defined in the Data Grid CR. The connection can be done with any pod in the Data Grid cluster.

  4. Disable the replication from secondary site to the primary site by running the following command. It prevents the clear request to reach the primary site and delete all the correct cached data.

    Command:

    site take-offline --all-caches --site=site-a
    Copy to Clipboard Toggle word wrap

    Output:

    {
      "offlineClientSessions" : "ok",
      "authenticationSessions" : "ok",
      "sessions" : "ok",
      "clientSessions" : "ok",
      "work" : "ok",
      "offlineSessions" : "ok",
      "loginFailures" : "ok",
      "actionTokens" : "ok"
    }
    Copy to Clipboard Toggle word wrap

  5. Check the replication status is offline.

    Command:

    site status --all-caches --site=site-a
    Copy to Clipboard Toggle word wrap

    Output:

    {
      "status" : "offline"
    }
    Copy to Clipboard Toggle word wrap

    If the status is not offline, repeat the previous step.

    Warning

    Make sure the replication is offline otherwise the clear data will clear both sites.

  6. Clear all the cached data in secondary site using the following commands:

    Command:

    clearcache actionTokens
    clearcache authenticationSessions
    clearcache clientSessions
    clearcache loginFailures
    clearcache offlineClientSessions
    clearcache offlineSessions
    clearcache sessions
    clearcache work
    Copy to Clipboard Toggle word wrap

    These commands do not print any output.

  7. Re-enable the cross-site replication from secondary site to the primary site.

    Command:

    site bring-online --all-caches --site=site-a
    Copy to Clipboard Toggle word wrap

    Output:

    {
      "offlineClientSessions" : "ok",
      "authenticationSessions" : "ok",
      "sessions" : "ok",
      "clientSessions" : "ok",
      "work" : "ok",
      "offlineSessions" : "ok",
      "loginFailures" : "ok",
      "actionTokens" : "ok"
    }
    Copy to Clipboard Toggle word wrap

  8. Check the replication status is online.

    Command:

    site status --all-caches --site=site-a
    Copy to Clipboard Toggle word wrap

    Output:

    {
      "status" : "online"
    }
    Copy to Clipboard Toggle word wrap

Now we are ready to transfer the state from the primary site to the secondary site.

  1. Login into your primary site
  2. Connect into Data Grid Cluster using the Data Grid CLI tool:

    Command:

    oc -n keycloak exec -it pods/infinispan-0 -- ./bin/cli.sh --trustall --connect https://127.0.0.1:11222
    Copy to Clipboard Toggle word wrap

    It asks for the username and password for the Data Grid cluster. Those credentials are the one set in the Deploy Data Grid for HA with the Data Grid Operator chapter in the configuring credentials section.

    Output:

    Username: developer
    Password:
    [infinispan-0-29897@ISPN//containers/default]>
    Copy to Clipboard Toggle word wrap

    Note

    The pod name depends on the cluster name defined in the Data Grid CR. The connection can be done with any pod in the Data Grid cluster.

  3. Trigger the state transfer from the primary site to the secondary site.

    Command:

    site push-site-state --all-caches --site=site-b
    Copy to Clipboard Toggle word wrap

    Output:

    {
      "offlineClientSessions" : "ok",
      "authenticationSessions" : "ok",
      "sessions" : "ok",
      "clientSessions" : "ok",
      "work" : "ok",
      "offlineSessions" : "ok",
      "loginFailures" : "ok",
      "actionTokens" : "ok"
    }
    Copy to Clipboard Toggle word wrap

  4. Check the replication status is online for all caches.

    Command:

    site status --all-caches --site=site-b
    Copy to Clipboard Toggle word wrap

    Output:

    {
      "status" : "online"
    }
    Copy to Clipboard Toggle word wrap

  5. Wait for the state transfer to complete by checking the output of push-site-status command for all caches.

    Command:

    site push-site-status --cache=actionTokens
    site push-site-status --cache=authenticationSessions
    site push-site-status --cache=clientSessions
    site push-site-status --cache=loginFailures
    site push-site-status --cache=offlineClientSessions
    site push-site-status --cache=offlineSessions
    site push-site-status --cache=sessions
    site push-site-status --cache=work
    Copy to Clipboard Toggle word wrap

    Output:

    {
      "site-b" : "OK"
    }
    {
      "site-b" : "OK"
    }
    {
      "site-b" : "OK"
    }
    {
      "site-b" : "OK"
    }
    {
      "site-b" : "OK"
    }
    {
      "site-b" : "OK"
    }
    {
      "site-b" : "OK"
    }
    {
      "site-b" : "OK"
    }
    Copy to Clipboard Toggle word wrap

    Check the table in this section for the Cross-Site Documentation for the possible status values.

    If an error is reported, repeat the state transfer for that specific cache.

    Command:

    site push-site-state --cache=<cache-name> --site=site-b
    Copy to Clipboard Toggle word wrap

  6. Clear/reset the state transfer status with the following command

    Command:

    site clear-push-site-status --cache=actionTokens
    site clear-push-site-status --cache=authenticationSessions
    site clear-push-site-status --cache=clientSessions
    site clear-push-site-status --cache=loginFailures
    site clear-push-site-status --cache=offlineClientSessions
    site clear-push-site-status --cache=offlineSessions
    site clear-push-site-status --cache=sessions
    site clear-push-site-status --cache=work
    Copy to Clipboard Toggle word wrap

    Output:

    "ok"
    "ok"
    "ok"
    "ok"
    "ok"
    "ok"
    "ok"
    "ok"
    Copy to Clipboard Toggle word wrap

As now the state is available in the secondary site, Red Hat build of Keycloak can be started again:

  1. Login into your secondary site.
  2. Startup Red Hat build of Keycloak.

    When deploying Red Hat build of Keycloak using the Red Hat build of Keycloak Operator, change the number of Red Hat build of Keycloak instances in the Red Hat build of Keycloak Custom Resource to the original value.

11.2.2. AWS Aurora Database

No action required.

11.2.3. Route53

No action required.

11.3. Further reading

See Concepts to automate Data Grid CLI commands on how to automate Infinispan CLI commands.

맨 위로 이동
Red Hat logoGithubredditYoutubeTwitter

자세한 정보

평가판, 구매 및 판매

커뮤니티

Red Hat 문서 정보

Red Hat을 사용하는 고객은 신뢰할 수 있는 콘텐츠가 포함된 제품과 서비스를 통해 혁신하고 목표를 달성할 수 있습니다. 최신 업데이트를 확인하세요.

보다 포괄적 수용을 위한 오픈 소스 용어 교체

Red Hat은 코드, 문서, 웹 속성에서 문제가 있는 언어를 교체하기 위해 최선을 다하고 있습니다. 자세한 내용은 다음을 참조하세요.Red Hat 블로그.

Red Hat 소개

Red Hat은 기업이 핵심 데이터 센터에서 네트워크 에지에 이르기까지 플랫폼과 환경 전반에서 더 쉽게 작업할 수 있도록 강화된 솔루션을 제공합니다.

Theme

© 2025 Red Hat