3617 - Troubleshooting DNS in NIOS
Scenario
You are the DNS administrator of a coffee company, with the internal zone coffee.corp on the server NS1 (10.100.0.111). This zone is delegated to a branch office, who is maintaining the child zone kona.coffee.corp on server NS2 (10.100.0.222). Users do not query NS1 or NS2 directly, they query the recursive resolver ADA (10.100.0.100), which has the correct forwarding setup so users can correctly resolve all names in the coffee.corp name space.
However, someone recently made a change to the environment, and users are reporting they are unable to resolve names in kona.coffee.corp. Please troubleshoot, locate the root cause, and fix the misconfiguration.
Below is a diagram to help you visualize the current DNS setup.
Estimate Completion Time
40 to 55 Minutes
Credentials
Description | Username | Password | URL or IP |
|---|---|---|---|
Grid Manager UI | admin | infoblox |
Course References
2009: Configuring NIOS DNS Services
2023: Configuring NIOS DNS Zones
3011: DNS Troubleshooting Methodology
Lab Initiation
Access jump-desktop
Once the lab is deployed, you can access the virtual machines required to complete this lab activity. To initiate the lab, click on the jump-desktop tile and login to the Linux UI:
Username: training
Password: infoblox
Initiate lab
To initiate the lab, double-click the Launch Lab icon on the Desktop.
Launch Lab
Choose the lab number from the list and click OK.
After clicking OK, you will see a pop-up message with a brief description of the lab task. If the description looks correct, click Yes to continue lab initiation.
Lab initiation will take a couple of minutes to finish.
Once complete, you will see another pop-up message with the login credentials and the URL for the Grid Manager’s User Interface. Note that the credentials may differ from those from prior labs.
Tasks
Task 1: Troubleshooting delegated name resolution
The delegated name sweet.kona.coffee.corp is reported not working. We start by verifying that it really is not resolving, analyze the failure code, and investigate server configurations as needed. For comparison, you may also query for the name bitter.coffee.corp.
On the Jump-Desktop, open a terminal window and execute the following command:
dig @10.100.0.100 sweet.kona.coffee.corp. AInvestigate why the lookup fails.
Review configurations on all DNS servers involved, and deduce what changes were made to the environment.
Based on the query results, choose a DNS server to analyze further. Study log messages on the DNS server. If necessary, perform packet captures. If you are not sure which DNS server to analyze first, start with the one closest to the client.
Solutions
Task 1 solution: Troubleshooting Forwarding
Root Cause
Forward-only is enabled on the conditional forward zone on ADA (10.100.0.100). This causes the recursive resolver ADA (10.100.0.100) unable to chase down delegation.
Detailed Analysis
The query for sweet.kona.coffee.corp results in SERVFAIL from the server ADA (10.100.0.100). Even when we execute the same query with the +cd option to disable any DNSSEC checking, we still get SERVFAIL. On the recursive server ADA (10.100.0.100), we see the following message when we navigate to Administration → Logs → syslog:
The error message FORMERR is somewhat misleading, as the more relevant error message is in the NOTICE message, shown below:
DNS format error from 10.100.0.111#53 resolving sweet.kona.coffee.corp/A for client 10.35.22.10#41548: non-improving referral
If you performed a traffic capture on ADA (10.100.0.100), you will end up with a file very similar to the one shown here:
In packet #236, authoritative server NS1 (10.100.0.111) responded with A record and glue record, referring ADA (10.100.0.100) to ns2.example.com with the IP address 10.100.0.222; however, in the following traffic we see ADA (10.100.0.100) continues to only query NS1 (10.100.0.111), thus ending in a logical loop that eventually ended with SERVFAIL. The forward-only option forces the recursive name server to only communicate with the list of servers identified in the forwarders list, and the list does not include 10.100.0.222, thus the recursive server is unable to chase the delegation.
Lessons Learned
The lesson we hope you take away from this exercise is, delegation was designed to be used for external-facing or public-facing zones. When setting up internal zones, it is very rare that you need to use delegation. For most internal name spaces, a sub zone is sufficient. Since many internal DNS servers perform recursion, it is also possible to set it up using conditional forwarding. In our example, if NS1 is allowing recursion, then you can setup a forward zone named kona.coffee.corp on NS1, and foward to NS2. With this setup, you do not need to worry about the forward-only setting on ADA breaking internal name resolution.