红帽全球峰会此内容没有您所选择的语言版本。
27.2. S3_PING Configuration Options
Red Hat JBoss Data Grid works with Amazon Web Services in two ways:
- In Library mode, use JGroups'
jgroups-ec2.xmlfile (see Section 26.2.2.3, “jgroups-ec2.xml” for details) or use theS3_PINGprotocol. - In Remote Client-Server mode, use JGroups'
S3_PINGprotocol.
In Library and Remote Client-Server mode, there are three ways to configure the
S3_PING protocol for clustering to work in Amazon AWS:
- Use Private S3 Buckets. These buckets use Amazon AWS credentials.
- Use Pre-Signed URLs. These pre-assigned URLs are assigned to buckets with private write and public read rights.
- Use Public S3 Buckets. These buckets do not have any credentials.
27.2.1. Using Private S3 Buckets
复制链接链接已复制到粘贴板!
This configuration requires access to a private bucket that can only be accessed with the appropriate AWS credentials. To confirm that the appropriate permissions are available, confirm that the user has the following permissions for the bucket:
- List
- Upload/Delete
- View Permissions
- Edit Permissions
Ensure that the
S3_PING configuration includes the following properties:
- either the
locationor theprefixproperty to specify the bucket, but not both. If theprefixproperty is set,S3_PINGsearches for a bucket with a name that starts with the prefix value. If a bucket with the prefix at the beginning of the name is found,S3_PINGuses that bucket. If a bucket with the prefix is not found,S3_PINGcreates a bucket using the AWS credentials and names it based on the prefix and a UUID (the naming format is {prefix value}-{UUID}). - the
access_keyandsecret_access_keyproperties for the AWS user.
Note
If a
403 error displays when using this configuration, verify that the properties have the correct values. If the problem persists, confirm that the system time in the EC2 node is correct. Amazon S3 rejects requests with a time stamp that is more than 15 minutes old compared to their server's times for security purposes.
Example 27.1. Start the Red Hat JBoss Data Grid Server with a Private Bucket
Run the following command from the top level of the server directory to start the Red Hat JBoss Data Grid server using a private S3 bucket:
bin/clustered.sh -Djboss.bind.address={server_ip_address} -Djboss.bind.address.management={server_ip_address} -Djboss.default.jgroups.stack=s3 -Djgroups.s3.bucket={s3_bucket_name} -Djgroups.s3.access_key={access_key} -Djgroups.s3.secret_access_key={secret_access_key}
bin/clustered.sh -Djboss.bind.address={server_ip_address} -Djboss.bind.address.management={server_ip_address} -Djboss.default.jgroups.stack=s3 -Djgroups.s3.bucket={s3_bucket_name} -Djgroups.s3.access_key={access_key} -Djgroups.s3.secret_access_key={secret_access_key}- Replace {server_ip_address} with the server's IP address.
- Replace {s3_bucket_name} with the appropriate bucket name.
- Replace {access_key} with the user's access key.
- Replace {secret_access_key} with the user's secret access key.
27.2.2. Using Pre-Signed URLs
复制链接链接已复制到粘贴板!
For this configuration, create a publically readable bucket in S3 by setting the List permissions to Everyone to allow public read access. Each node in the cluster generates a pre-signed URL for put and delete operations, as required by the
S3_PING protocol. This URL points to a unique file and can include a folder path within the bucket.
Note
Longer paths will cause errors in
S3_PING. For example, a path such as my_bucket/DemoCluster/node1 works while a longer path such as my_bucket/Demo/Cluster/node1 will not.
27.2.2.1. Generating Pre-Signed URLs
复制链接链接已复制到粘贴板!
JGroup's
S3_PING class includes a utility method to generate pre-signed URLs. The last argument for this method is the time when the URL expires expressed in the number of seconds since the Unix epoch (January 1, 1970).
The syntax to generate a pre-signed URL is as follows:
String Url = S3_PING.generatePreSignedUrl("{access_key}", "{secret_access_key}", "{operation}", "{bucket_name}", "{path}", {seconds});
String Url = S3_PING.generatePreSignedUrl("{access_key}", "{secret_access_key}", "{operation}", "{bucket_name}", "{path}", {seconds});- Replace {operation} with either
PUTorDELETE. - Replace {access_key} with the user's access key.
- Replace {secret_access_key} with the user's secret access key.
- Replace {bucket_name} with the name of the bucket.
- Replace {path} with the desired path to the file within the bucket.
- Replace {seconds} with the number of seconds since the Unix epoch (January 1, 1970) that the path remains valid.
Example 27.2. Generate a Pre-Signed URL
String putUrl = S3_PING.generatePreSignedUrl("access_key", "secret_access_key", "put", "my_bucket", "DemoCluster/node1", 1234567890);
String putUrl = S3_PING.generatePreSignedUrl("access_key", "secret_access_key", "put", "my_bucket", "DemoCluster/node1", 1234567890);
Ensure that the
S3_PING configuration includes the pre_signed_put_url and pre_signed_delete_url properties generated by the call to S3_PING.generatePreSignedUrl(). This configuration is more secure than one using private S3 buckets, because the AWS credentials are not stored on each node in the cluster
Note
If a pre-signed URL is entered into an XML file, then the
& characters in the URL must be replaced with its XML entity (&).
27.2.2.2. Set Pre-Signed URLs Using the Command Line
复制链接链接已复制到粘贴板!
To set the pre-signed URLs using the command line, use the following guidelines:
- Enclose the URL in double quotation marks (
""). - In the URL, each occurrence of the ampersand (
&) character must be escaped with a backslash (\)
Example 27.3. Start a JBoss Data Grid Server with a Pre-Signed URL
bin/clustered.sh -Djboss.bind.address={server_ip_address} -Djboss.bind.address.management={server_ip_address} -Djboss.default.jgroups.stack=s3 -Djgroups.s3.pre_signed_put_url="http://{s3_bucket_name}.s3.amazonaws.com/ node1?AWSAccessKeyId={access_key}\&Expires={expiration_time}\&Signature={signature}"-Djgroups.s3.pre_signed_delete_url="http://{s3_bucket_name}.s3.amazonaws.com/ node1?AWSAccessKeyId={access_key}\&Expires={expiration_time}\&Signature={signature}"
bin/clustered.sh -Djboss.bind.address={server_ip_address} -Djboss.bind.address.management={server_ip_address} -Djboss.default.jgroups.stack=s3 -Djgroups.s3.pre_signed_put_url="http://{s3_bucket_name}.s3.amazonaws.com/ node1?AWSAccessKeyId={access_key}\&Expires={expiration_time}\&Signature={signature}"-Djgroups.s3.pre_signed_delete_url="http://{s3_bucket_name}.s3.amazonaws.com/ node1?AWSAccessKeyId={access_key}\&Expires={expiration_time}\&Signature={signature}"
In the provided example, the
{signatures} values are generated by the S3_PING.generatePreSignedUrl() method. Additionally, the {expiration_time} values are the expiration time for the URL that are passed into the S3_PING.generatePreSignedUrl() method.
27.2.3. Using Public S3 Buckets
复制链接链接已复制到粘贴板!
This configuration involves an S3 bucket that has public read and write permissions, which means that Everyone has permissions to List, Upload/Delete, View Permissions, and Edit Permissions for the bucket.
The
location property must be specified with the bucket name for this configuration. This configuration method is the least secure because any user who knows the name of the bucket can upload and store data in the bucket and the bucket creator's account is charged for this data.
To start the Red Hat JBoss Data Grid server, use the following command:
bin/clustered.sh -Djboss.bind.address={server_ip_address} -Djboss.bind.address.management={server_ip_address} -Djboss.default.jgroups.stack=s3 -Djgroups.s3.bucket={s3_bucket_name}
bin/clustered.sh -Djboss.bind.address={server_ip_address} -Djboss.bind.address.management={server_ip_address} -Djboss.default.jgroups.stack=s3 -Djgroups.s3.bucket={s3_bucket_name}27.2.4. Troubleshooting S3_PING Warnings
复制链接链接已复制到粘贴板!
Depending on the
S3_PING configuration type used, the following warnings may appear when starting the JBoss Data Grid Server:
15:46:03,468 WARN [org.jgroups.conf.ProtocolConfiguration] (MSC service thread 1-7) variable "${jgroups.s3.pre_signed_put_url}" in S3_PING could not be substituted; pre_signed_put_url is removed from properties
15:46:03,468 WARN [org.jgroups.conf.ProtocolConfiguration] (MSC service thread 1-7) variable "${jgroups.s3.pre_signed_put_url}" in S3_PING could not be substituted; pre_signed_put_url is removed from properties15:46:03,469 WARN [org.jgroups.conf.ProtocolConfiguration] (MSC service thread 1-7) variable "${jgroups.s3.prefix}" in S3_PING could not be substituted; prefix is removed from properties
15:46:03,469 WARN [org.jgroups.conf.ProtocolConfiguration] (MSC service thread 1-7) variable "${jgroups.s3.prefix}" in S3_PING could not be substituted; prefix is removed from properties15:46:03,469 WARN [org.jgroups.conf.ProtocolConfiguration] (MSC service thread 1-7) variable "${jgroups.s3.pre_signed_delete_url}" in S3_PING could not be substituted; pre_signed_delete_url is removed from properties
15:46:03,469 WARN [org.jgroups.conf.ProtocolConfiguration] (MSC service thread 1-7) variable "${jgroups.s3.pre_signed_delete_url}" in S3_PING could not be substituted; pre_signed_delete_url is removed from properties
In each case, ensure that the property listed as missing in the warning is not needed by the
S3_PING configuration.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}