Configuring JBoss ON Servers, Agents, and Storage Nodes
JBoss Operations Network 3.2
tuning server and agent configuration
December 4, 2013
Abstract
Both servers and agents in JBoss ON can be configured to improve performance within your specific environment. High availability, affinity, and failover settings all improve performance large JBoss ON deployments. Other settings related to tasks like discovery and monitoring can be tuned to provide better performance and quality within your specific environment.
This guide provides information to understand and edit JBoss ON server and agent configuration.
1. About JBoss Operations Network
The primary use for JBoss ON is to give administrators a single point of access to view their systems. Functionally, that means that JBoss ON provides a means to develop and monitor a system's inventory. Every managed resource – from platforms to applications to services – is contained and organized in the inventory, no matter how complex the IT environment is.
JBoss ON centralizes all of its operations in an installed server. The JBoss ON server communicates with locally installed JBoss ON agents, which interact directly with the platform and services to carry out local tasks such as monitoring. The types of resources that can be managed by JBoss ON and the operations that can be carried out are determined by the server and agent plug-ins which are loaded in JBoss ON.
The relationships between servers, agents, plug-ins, and resources are what define JBoss ON.
1.1. About JBoss ON Agents
JBoss ON agents are deployed on every machine that JBoss ON manages. The agent is an intermediary between the resource itself and the central JBoss ON server.
The agents receive updates like configuration changes, updated packages, new settings for alerts, and operations from the JBoss ON server and then it carries out those tasks on the resource. The agent also collects information from the resource which it forwards to the server. This allows the server to process alerts, metrics, and availability information for the resource.
Because the agent is independent of the server, it can continue with its monitoring tasks and gather information about the resource even if the server is down or the resource loses network connectivity.
Each resource is arranged in a hierarchy, showing relationships between platforms, servers, and services. Only one agent is required per machine; once the platform is managed as a resource, all or a subset of installed applications or services can be added as resources, all using the same local agent.
1.2. About JBoss ON Servers
JBoss ON is built around a central server. The server performs two vital functions:
- Stores the configuration for both resources and resource groups.
- Organizes and responds to the data collect by the agents.
The JBoss ON server is the central location for administrators to manage an operating environment. The server is used to set baseline configuration and provision applications, to define alerts and notifications, and to initiate operations. As agents send information back to the server from the resource, then the server can also perform monitoring tasks (by providing metrics and reporting) and can also respond to events by sending alerts or launching operations.
The data used by the JBoss ON server is stored in a backend SQL database. These data include:
- The inventory of resources
- The configured groups
- Monitoring data
- Configuration data
- Content available to resources
- User and access control information
The JBoss ON server hosts the graphical user interface which is used to interact with JBoss ON.
One very important aspect of JBoss ON servers is this: they only communicate with the backend database and the JBoss ON agent. As long as JBoss ON servers use the same backend database, they are automatically included in a server cloud that allows for failover and scalability, without additional configuration in the servers or the database. JBoss ON agents can be configured to use a list of preferred JBoss ON servers, which naturally distributes the agent load among the servers and provides agent-server failover without detailed configuration.
1.3. About Metrics Storage Nodes
Collecting metrics and generating alerts can be resource-intensive. It results in near-constant write operations on the backend database. That creates a natural threshold for the number of metrics that can be collected (30,000 per minute) before encountering performance degradation.
Note
The natural threshold of 30,000 metrics per minute is based on internal performance testing. This threshold varies depending on system resources and overall load.
JBoss ON uses two databases to store its information. One is a central relational database (PostgreSQL or Oracle) which stores all configuration about the JBoss ON servers and agents, all resource inventory data, resource configuration, and other data. The other database is a distributes database (a cluster of storage nodes) which stores all numeric monitoring data — in other words, all collected metrics.
The metrics storage node can be installed on its own dedicated machine, which can significantly improve write performance to the database (and, therefore, improve monitoring performance):
- Dedicated CPU
- More available physical memory
- Faster disks
- More disk space
These are the same performance considerations as installing the relational database on a separate machine from the JBoss ON server. By using two databases, it is also possible to move write-intensive and resource-intensive metrics storage away from resource-intensive configuration data, such as drift snapshots and bundle configuration.
Additionally, the distributed database can be expanded, with multiple nodes in a cluster. This ability to add additional nodes according to load is a crucial management tool for administrators. Rather than encountering that hardware-driven limit of 30,000 metrics collected per minute, additional nodes can be added to improve performance.
The storage node cluster is created and managed by JBoss ON (which also minimizes the metrics storage node management overhead). A storage node is installed on a system, using the
rhqctl install --storage command. The storage node always requires a companion agent.
