Skip to main content
Skip table of contents

3512 - Troubleshoot Grid Master Candidate Promotion in NIOS

Scenario

As part of the Disaster Recovery exercise, you and your fellow grid administrators need to provide a report on NIOS Disaster Recovery readiness. Verify the GM redundancy, by making sure that the GM and GMC will be able to switch over without impacting production too severely. There is a tool in NIOS that will allow you to test GMC promotion, which will provide insight on any potential issues.

Estimate Completion Time

  • 25 to 35 Minutes

Credentials

Description

Username

Password

URL or IP

Grid Manager UI

admin

infoblox

https://10.100.0.100/

Course References

  • 3001: Setting up the NIOS Grid

  • 3002: Managing NIOS Grid and Grid Members

Lab Initiation

Access jump-desktop

Once the lab is deployed, you can access the virtual machines required to complete this lab activity. To initiate the lab, click on the jump-desktop tile and login to the Linux UI:

Username: training

Password: infoblox

Initiate lab

To initiate the lab, double-click the Launch Lab icon on the Desktop.

Launch Lab

Launch Lab

Choose the lab number from the list and click OK.

After clicking OK, you will see a pop-up message with a brief description of the lab task. If the description looks correct, click Yes to continue lab initiation.

Lab initiation will take a couple of minutes to finish.

Once complete, you will see another pop-up message with the login credentials and the URL for the Grid Manager’s User Interface. Note that the credentials may differ from those from prior labs.

Screenshot 2024-05-06 at 3.16.57 PM.png

Tasks

Task 1: Perform a GMC Promotion Test

Before promoting, use the ‘GMC Promote Test’ tool from Grid Manager and record the results.

Task 2: Perform Grid Master Candidate Promotion

Despite the results, you decide to promote the Grid Master Candidate (ibgmc) anyway. Observe the behavior.

Task 3: Perform Grid Master Candidate Promotion

Once you have identified any issues, fail back the Grid so that member (nios-1) is the Grid Master again.


Solutions

Task 1: Perform a GMC Promotion Test

Before promoting, use the ‘GMC Promote Test’ tool from Grid Manager and record the results.

  1. In the Grid Manager UI, navigate to Grid Grid Manager, from the Toolbar, select GMC Promote Test

  2. Select the GMC ibgmc.techblue.net, change the timeout to 10 seconds (so we do not wait as long), check the member ibns1.techblue.net, and click Start.

  3. After a while, we see the results, clearly indicating that port 2114 connectivity has failed.

In this exercise, the NIOS admin is only responsible for identifying any potential issues with GMC failover readiness. If any concerns are found during the assessment, the NIOS admin will report them to the appropriate contact for further investigation, involving the network team and/or firewall team if necessary.

Task 2: Perform Grid Master Candidate Promotion

Despite the results, you decide to promote the Grid Master Candidate (ibgmc) anyway. Observe the behavior.

  1. Connect to member (nios-5) using SSH or the console.

  2. Type the command:
    set promote_master
    Follow the prompts to confirm

  3. After typing in the command, in the original (nios-1) Grid Manager UI, you will see ibgmc being marked as Offline, and after a while, the web UI becomes unresponsive. Use the web browser refresh button, and you will be redirected to login to https://10.200.0.100/, the newly promoted Grid Master (nios-5)

  4. After logging in to the new Grid Master (nios-5), you will see that HA member (ibns1) is offline, and the original Grid Master (nios-1) is reconnected as Grid Master Candidate

  5. Member (ibns1) will fail to establish VPN to (nios-5) due to the blocked port 2114

Task 3: Perform Grid Master Candidate Promotion

Once you have identified any issues, fail back the Grid so that member (nios-1) is the Grid Master again.

  1. Connect to member (nios-1) using SSH or the console.

  2. Type the command:
    set promote_master

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.