5. Known Issues in JBoss Operations Network
This section contains a subset of the known issues at the time that JBoss Operations Network 3.1 was released. When possible, workarounds are provided.
5.1. Upgrade and Installation
Warning
Upgrade from JBoss Operations Network 3.0 to JBoss Operations Network 3.1 is fully supported.
However, upgrade from JBoss Operations Network 3.1 Beta to JBoss Operations Network 3.1 is not supported. It is not possible to upgrade a beta version of JBoss Operations Network to the release version. If you install a beta server, then you must install the server, and all its resource configuration, fresh for JBoss ON 3.1 GA.
Bugzilla 821093: "HTTP Status 404 - /coregui/" error after upgrade
When the JBoss ON server is upgraded from 3.0 to 3.1, then the UI can return a 404 error when you try to open the installer page to re-install the server.
Even when opening the proper URL to access the installer, such as
server.example.com:7080, the browser has a cached redirect to the old UI, server.example.com:7080/coregui. Because the coregui/ path does not exist yet, because the new server has not yet been configured, this redirect returns a 404 error.
There are a few ways to work around this:
- Enter the full URL to the installer page:
http://server.example.com:7080/installer/welcome.jsf
- Delete the cache and reload the page.
- Use Ctrl+F to force the page to reload.
Some information about the server cannot be changed during upgrade. Even though the fields are editable in the upgrade form, changing these values from the original settings causes upgrade problems. This includes information like the server name. If any of that information changes, then one of the results is NPEs in the server logs.
For example, the process to register the upgraded server plug-ins fails with a null pointer exception. For example:
2012-03-08 20:33:34,523 ERROR [org.rhq.enterprise.server.core.plugin.ServerPluginScanner] Failed to register server plugin file [/home/hudson/jon-server-3.0.1.GA/jbossas/server/default/deploy/rhq.ear/rhq-serverplugins/rhq-serverplugin-ant-bundle-4.2.0.JON.3.0.1.GA.jar] java.lang.NullPointerException at org.rhq.enterprise.server.core.plugin.ServerPluginScanner.registerServerPlugin(ServerPluginScanner.java:212) ...
This error only occurs if a different server name was entered in the configuration page when the server was upgraded. Changing the Server Name field is not supported for upgrades.
If the installer is not bound to 0.0.0.0 when setting up a server, then it does not set all of the required connection properties. Specifically, the installer does not set the
java.rmi.server.hostname parameter to the real value, and it uses the default of 0.0.0.0. This parameter must be set to the real IP address of the server by manually editing the rhq-server.properties file.
Automatic upgrades of agents on Windows appear to fail. Two conditions must be true for automatic upgrade of agents to work on Windows:
- The agent must be installed and running as a Windows service.
- No Windows Explorer window can be open to any agent subdirectory. Windows sets a lock on directories when they are open, which blocks the upgrade process.
When starting a server while agents are running, the server may log servlet errors in the logs. This is because the remoting (communications) classes are loaded early in the startup sequence, before the server is completely started. This causes some communications interruptions until the server is completely started. These errors can be ignored.
If a JBoss ON server was added to the resource inventory, the old JBoss ON server resource must be deleted from the inventory after upgrade. The old resource entry is unusable after upgrade.
Null pointer exceptions for the
org.apache.catalina.connector.CoyoteAdapter service are returned when the JBoss ON 3.1 server is first installed. These errors are harmless and can be ignored. Installation will complete successfully, and both the server and the GUI will start and run properly.
The installer shows non-replaced {} properties.
5.2. JBoss EAP 6 Resources
The tracking bug for all resolved and open issues related to the EAP 6 plug-in for JBoss ON 3.1 is Bugzilla 707223.
- An edit icon appears next to module names in the Configuration > Current > Extensions area of the EAP 6 server configuration. These properties are read-only and cannot, in fact, be edited even though an edit box pops up.
- If an EAP 6 server is reset to run insecurely, then the "Install RHQ User" operation for the EAP 6 server fails because it has no way to connect to the (secure) management realm.
- Bundle provisioning is only available for EAP 6 standalone servers. Standalone servers use a filesystem-based deployment directory, which is used by the bundle system. Domain resources use a different, distributed style of deployment. The domain style of content distribution is not supported for bundles.
- When editing a network interface for a JBoss EAP 6 server, the network configuration requires either a specific IP address or a setting to use any IP address for IPv4, for IPv6, or any address. However, the UI does not enforce that an IP address selection is made, which can cause the interface to be unreachable.
- Attempting to deploy a child resource such as a WAR application fails with a malformed URL exception if the EAP 6 server is listening on IPv6.
- All child elements for an embedded HornetQ resource — connection factories, JMS queues, and JMS topics — all require a JNDI name in the configuration. Optionally, multiple JNDI names can be set, in a comma-separated list, to support failover.However, the UI does not force the JNDI name to be given when creating the connection factory, JMS queue, and JMS topic children. This means that the creation operation fails if the JNDI name is not included.
- The JBossWeb connector configuration has a section for SSL parameters. Any information in those fields is only read if SSL is enabled (the secure option is selected). However, these fields are still active even if SSL is not enabled. They should be grayed out.Also, if SSL is enabled and then disabled, all of the values for the SSL configuration fields are blanked and unset.
- When manually adding a standalone server in the JBoss ON UI, rather than importing it from the discovery queue, there is an intermittent error:
The original product type discovered for this AS7 server was null, but the server is now reporting its product type is [JBoss EAP 6]
This is shown in a red bar at the top of the UI. Despite the error, the resource is successfully imported and available. This error can be ignored. - Some path elements defined in the configuration of a domain controller or standalone server are computed by the server itself. These computed paths are all prefixed with
java.*,jboss.*, anduser.*.While the configuration values are viewable, they are not editable. However, the Configuration tab for the server shows both edit and delete options for the path elements. If a user attempts to change or delete the path, the path simply reverts to its previous state. - A JVM definition can be created at the host, server group, and managed server levels. There can be multiple JVM definitions at the host level; these are used as templates or base definitions for the server group and managed server definitions. However, server groups and managed servers can have only a single JVM definition.The UI allows a user to attempt to create a second JVM definition as a child to a managed server or server group. If a JVM definition already exists for a managed server or server group, then an error is returned if a user attempts to create another JVM definition. The server throws an exception and there is an error message that the JVM definition is a singleton and that the first resource should be deleted before creating another child resource of that type.
- When a child JVM definition is created for a managed resource, the name of the JVM definition must be the same as the host JVM definition which it is based on. However, the JBoss ON UI creation wizard does not detect or enforce a pre-computed name, so it allows a user to enter any name.
- The UI will not let users enter a driver name when manually adding a datasource, so it is not possible to deploy a module-loaded driver.
- The agent properly detects an EAP 6 server running over an IPv6 address, but it incorrectly reports it as offline.
- It can take several minutes to be able to detect a child resource on a JBoss EAP 6 server.
- If both an EAP 6 server and a JDG server are running on the same system, the EAP 6 child resources are detected and added as (unavailable) child resources of the JDG instance.
- A user can edit configuration in JBoss ON that requires a reboot of EAP 6 to take effect, but there is no indication that the reboot is required.
5.3. Server Configuration
The password mechanism in the server installer doesn't properly hash Unicode characters. The Unicode characters are translated into the HTML entities, and then the HTML characters are hashed and stored as the password.
For example, the string
你好 is treated as the string 你好 in the installer, and the second string is the one that is hashed and stored in the rhq-server.properties file.
After configuring the server, but before logging in, manually update the password in the
rhq-server.properties file:
- Manually generate a hashed password based on the Unicode password:
serverRoot/jon-server-3.1.0.GA/bin/generate-db-password.sh 你好 Encoded password: 68f725778bb36d3b
- Replace the database password value with the new hashed password.
rhq.server.database.password=68f725778bb36d3b
5.4. Agents
When a plug-in, effectively a resource type, is deleted, the agent's plug-in container is automatically rebooted, to reload the new configuration. If the agent itself is running as a daemon, then rebooting the plug-in container also kills the agent thread. That means that if a plug-in (with resources of that type in the inventory) is deleted while the agent is running in the background, the agent process is stopped.
The agent can be restarted without any errors, and the plug-in is successfully removed.
Attempting to use the
config import command in the agent command prompt fails with an InvalidPreferencesFormatException exception.
java.util.prefs.InvalidPreferencesFormatException: org.xml.sax.SAXParseException; lineNumber: 2; columnNumber: 41; Document root element "preferences", must match DOCTYPE root "null". ...
The JVM can sometimes cause the agent to crash when Augeas is running.
A fatal error has been detected by the Java Runtime Environment ...
Attempting to ignore a resource in the discovery queue returns a ConcurrentModificationException on the agent.
5.5. Databases
The agent assumes that PostgreSQL uses a default password. If a different password is used, the PostgreSQL resource is shown as unavailable in the UI even if it is running.
5.6. Groups
A dynagroup definition with a short recalculation interval (less than two minutes) can cause duplicate groups to be created.
Changing the recursive setting on a group disassociates the group from any roles it was (previously) associated with.
5.7. Provisioning
Hot deployment fails if the file changes during provisioning.
If a bundle is being deployed in a hot deployment and one of the underlying bundle files, like a WAR or JAR, is changed while the bundle is being uploaded, then provisioning will fail.
Using the Recipe option to upload a recipe file separately than the bundle archive files requires that the XML in the recipe be well-formed and that all tags are explicitly closed, using the format
<tag></tag>, not the abbreviated format <tag />.
Additionally, the Recipe option's upload button does not work on Internet Explorer. To add a recipe file using this option with Internet Explorer, copy the entire recipe file and paste it directly into the text box.
The
<rhq:system-service> Ant tag only works on Red Hat Enterprise Linux systems. Using the tag and then deploying the bundle to other operating systems causes the Ant task to fail to operate and can cause the deployment to fail.
5.8. Content and Repositories
The installation data for content deployed through JBoss ON can be very inaccurate, off by thousands of days.
A repository can only be used for one resource, the one it was initially defined for.
Attempting to delete a repository with a space in the name throws an exception and the deletion fails.
Deploying WARs from a content repository is not supported. WARs and EARs should be deployed using bundles.
Deploying an exploded archive on JBoss EAP 4 results in deploying a zipped archive, which makes the deployed application unavailable.
5.9. Remote API and the CLI
The JBoss ON remote API cannot be used to connect to the JBoss ON server when the client is running inside an application server. For example, the remote API cannot be run from a client inside an EAP instance; it fails with errors like the following:
Caused by: java.lang.IllegalArgumentException: interface org.rhq.enterprise.server.auth.SubjectManagerRemote is not visible from class loader at java.lang.reflect.Proxy.getProxyClass(Proxy.java:353) at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:581) at org.rhq.enterprise.client.RemoteClientProxy.getProcessor(RemoteClientProxy.java:69)
ResourceFactoryManager does not throw any exceptions when attempting to create duplicate or invalid resources in the CLI.
Running a search with the *Criteria API using unlimited PageControl, as well as one or more fetches, returns duplicate results.
5.10. UI
Problems with Sessions and Multiple Tabs
If there are multiple tabs open to the JBoss ON UI and one of those sessions times out, then the other tabs will register errors. This can also occur if multiple people are logged into JBoss ON as the same user.
If there are more than 200 children for a resource, not all of the children are displayed in the resource tree. Additionally, refreshing the tree may display a different subset of child resources.
Searching for a resource with a backslash (\) in the name returned no results, even if the resource exists.
Bugzilla 736732: Bookmarked UI Pages May Display the Old UI
The page names used for different pages in the JBoss ON UI for version 3.0 are different than the ones used in JBoss ON 2.4. However, because JBoss ON 3.0 retains some of the elements used in the older JSF-based UI, these pages still exist. This means that bookmarks will go to the old style pages.
For example, the Dashboard page in the old JSF UI had the URL http://localhost:7080/Dashboard.do/. In the new UI, the location is http://localhost:7080/coregui/#Dashboards.
All of the new pages have the
coregui/ element in them. Remove any old bookmarks and replace them with the new GWT-based pages in the coregui/ area.
When the results for a dynamic search are long, the list is truncated but no scrollbar is presented, so it is not possible to scroll to the end of the list.
5.11. Measurements and Monitoring
A user-defined baseline value for a metric is ignored and overwritten by the server-calculated baseline.
5.12. Alerts
Attempting to upload a CLI script directly to a repository to use for alerting fails because it says the SHA-256 checksum for the script is not an OSGi string. Uploading a CLI script through the alert configuration UI works successfully.
In Oracle, the alert notification table is created with the NOLOGGING option. This means that users are required to review the alert notification configuration manually for every alert to verify that the configuration is correct.
Alerts with compound AND conditions can incorrectly fire when one of the conditions goes from true to false within 30 seconds of the other condition going from false to true.
Alert conditions and alert notifications in an alert definition cannot be edited. The condition or notification must be deleted and a new item added.
Recovery alerts are not triggered by an availability condition.
5.13. Drift
Drift definitions can be created for resources which do not support drift, using the remote API to create a script.
Drift monitoring is not supported for embedded web applications, only for standalone applications.
5.14. Platform Resources
The inventory for a Linux platform may discover non-existent network adapters, which are added and then shown as unavailable in the inventory.
5.15. Tomcat Resources
If a WAR file is configured to be deployed to a non-standard location on a Tomcat application server, an invalid deployment name is added to the inventory. The
unpackWARs option extracts the web app to a second, invalid directory. For example, if it should be extracted to /myapps/helloworld, the web app is extracted both there and to /myapps#helloworld.
The agent cannot discover a Tomcat instance if its Java process is running as a user other than the system user that the agent is running as.
5.16. Apache Resources
Apache directives are case-insensitive, but the Apache plug-in treats them as if they were case-sensitive. This makes the plug-in disregard directives with non-default case.
If the
/tmp directory is configured a noexec in the fstab file, the agent throws exceptions because it cannot properly initialize the Augeas lens. In that case, the Configuration tab is unavailable for the Apache resource.
To work around this, make sure that the
/tmp directory does not have noexec set as an option.
# # /etc/fstab # tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0
noexec is one security method available to that directory to prevent unauthorized or malicious scripts from running. Disabling noexec for the directory is not ideal, but it is also not a security issue by itself because other filesystem security settings can be used.
5.17. JBoss and JVM Resources
If EAR, WAR, or script resources are deleted outside of JBoss ON, the agent throws an exception during discovery, and the server is not informed that the resource was deleted.
Discovering a JBoss AS instance fails if the configuration directory is a URL.
The discovery process fails to determine the JNP URL if the data directory is not located at ${jboss.server.home.dir}/data.
Shutting down a JBoss AS 5 resource can fail with java.rmi.NoSuchObjectException: no such object in table.
It is not possible to add some system properties for a JBoss AS resource.
The JBoss AS plug-in is unable to load resource configuration for JMS queues that are deployed using a SAR file.
The discovery process attempts to discover an embedded JVM process through its JMX remoting rather than using the EMSConnection of its parent Tomcat or JBoss resource.
5.19. Performance
Attempting to update the members in a compatible group with thousands of members times out and fails with 'Failed to fetch Resource Group.'
Viewing configuration, alerts, or other data for groups with more than 1000 members can fail with the SQL exception maximum number of expressions in a list is 1000.
5.20. Plug-in Writing
If an embedded plug-in extension is defined and the parent resource type has subcategories, those subcategories cannot be changed or it causes a database deadline.
