Cloudera Navigator:
We need Cloudera Navigator is a fully integrated data management tool and a security system that is installed on the Hadoop Platform. Cloudera provides a software platform for Machine Learning & Big Data analytics Services that keeps running in the cloud. Cloudera Navigator is the main complete data administration solution for Hadoop, offering basic abilities, such as data disclosure, metadata management, audit, constant optimization, and policy enforcement.
Scenario: While doing the Cloudera Manager upgrade we have to upgrade Cloudera Navigator too. The below steps will guide us on how to do configuration changes and Cloudera Navigator upgrades.
If you are running a version of Cloudera Manager 5.4.x or below and are planning to upgrade to Cloudera Manager 5.5.x or higher, use the following instructions to determine.
Applying Configurations for Navigator Metadata Server Upgrade
From Cloudera Manager:
In Cloudera Manager, go to Cloudera Management Services > Configuration > Navigator Metadata Server > Resource Management > Java Heap Size of Navigator Metadata Server in Bytes
The recommended amount of heap to allocate for the Navigator Metadata Server is ~30 GB.
Alter the value to the newly-calculated heap size for Upgrade Operation
Save the change
Change the Navigator Upgrade Batch size from its default of 100K to a temporary value of 10K. To set this advanced configuration property and value:
- Cloudera Management Services > Configuration > Navigator Metadata Server Advanced Configuration Snippet (Safety Valve) for Cloudera-navigator.properties
- In the text field enter: nav.upgrade.batch.size=10000
Save the change
Restart Navigator Metadata Server for these configuration changes to take effect
After these values are set and all other preparations made, perform the Cloudera Manager upgrade
After Cloudera Manager is upgraded, load the Navigator Metadata Server UI and log in one time to invoke the schema upgrade for this role.
Monitor progress of Navigator Metadata Server schema upgrade and make any adjustments as needed
- Using the NMS UI: The upgrade may take several hours depending on the size of the data set. Progress will be noted in the UI. It is not required to keep a browser session open once the schema upgrade is invoked.
- Using Cloudera Manager Charts: Track the Navigator Metadata Server heap usage during the upgrade by referencing the live chart for JVM Heap Memory Usage on the Clusters > Cloudera Management Service > Navigator Metadata Server role page
- In the rare case an OutOfMemory error is seen during the schema upgrade, allocate an additional heap to the Navigator Metadata Server for the duration of schema upgrade based on the growth seen from the JVM Heap Memory Usage chart.
Once the Navigator Metadata Server upgrade is complete and the Navigator Metadata Server UI is operational, it can be used. The total heap requirement will go back down and should be set to the Normal Operation Post-Upgrade value.
Normal Operation Post-Upgrade
Applying the Right-Sized Heap Configuration for Normal Operation
From Cloudera Manager
Navigate to Cloudera Management Services > Configuration > Navigator Metadata Server > Resource Management > Java Heap Size of Navigator Metadata Server in Bytes
The recommended amount of heap to allocate for the Navigator Metadata Server is ~20 GB.
Alter the value to the newly-calculated heap size
Save the change
Revert the prior change to the Navigator Upgrade Batch size from its upgrade setting of 10K: To set this advanced configuration property and value:
- Cloudera Management Services > Configuration > Navigator Metadata Server Advanced Configuration Snippet (Safety Valve) for Cloudera-navigator.properties
- In the text field find and remove: nav.upgrade.batch.size=10000
Save the change
Restart Navigator Metadata Server for these configuration changes to take effect
Continue to monitor the Navigator Metadata Server JVM Heap Memory Usage periodically and adjust as necessary. Memory usage could increase over time as new elements and relations are established within-cluster data stores.
Cloudera has a similar approach to deal with next-generation data analytics management and big data solutions for hybrid deployments. Cloudera, like Hortonworks, believes that data can drive high-speed business model changes and has improved in a manner that advantage the market by creating new revenue possibilities.
If you have followed all the above instructions and the Navigator Metadata Server is not successfully upgrading its schema, is still presenting Out of Memory exceptions or is not starting after an upgrade, see this related knowledge article for further instruction and additional options.
Recent Blogs
Categories
