3525 - Troubleshooting DDNS in NIOS

Scenario

Someone on your team has setup DDNS for the network 172.31.101.0/24 to work with the zone techblue.net. However, while the testing-linux client can receive a DHCP lease, it is unable to create the corresponding DDNS records. Please investigate and locate the root cause.

You may use these commands to help you on the testing-linux:

To enable DHCP on the testing-linux, type: sudo set-network-dhcp
For sudo password, type: infoblox
To disable DHCP on the testing-linux, type: sudo set-network-disable
To clear existing DHCP lease on testing-linux, type: sudo clear-dhcp-leases

Hint: If you are not sure where to start, look at syslog on the DHCP server.

Estimate Completion Time

25 to 30 minutes

Credentials

Description	Username	Password	URL or IP
Grid Manager UI	admin	infoblox	https://10.100.0.100/

Requirements

Administrative access to the Grid

Course References

3011: DNS Troubleshooting Methodology
3014: NIOS DNS Zones Expert Topics
3015: NIOS Dynamic DNS Expert Topics

Lab Initiation

Access jump-desktop

Once the lab is deployed, you can access the virtual machines required to complete this lab activity. To initiate the lab, click on the jump-desktop tile and login to the Linux UI:

Username: training

Password: infoblox

Initiate lab

To initiate the lab, double-click the Launch Lab icon on the Desktop.

Choose the lab number from the list and click OK.

After clicking OK, you will see a pop-up message with a brief description of the lab task. If the description looks correct, click Yes to continue lab initiation.

Lab initiation will take a couple of minutes to finish.

Once complete, you will see another pop-up message with the login credentials and the URL for the Grid Manager’s User Interface. Note that the credentials may differ from those from prior labs.

Tasks

To complete this lab, you need to perform the following tasks.

Investigate DDNS issue
Correct configuration to fix the issue

Task 1: Analyze and troubleshoot DDNS Configuration errors

Use your analytical skills to investigate why DDNS is not working as expected and correct configuration errors to ensure proper DDNS updates.

Solutions

Task 1 Solution: Analyze and troubleshoot DDNS Configuration errors

Below is the syslog from (ibns2) relevant to identify this problem:

CODE

2023-06-15T11:52:11+00:00 daemon ibns2.techblue.net dhcpd[29852]: info DHCPDISCOVER from 00:50:56:00:1a:2b via 172.31.101.1 TransID 620fbcba
2023-06-15T11:52:12+00:00 daemon ibns2.techblue.net dhcpd[29852]: info DHCPOFFER on 172.31.101.253 to 00:50:56:00:1a:2b (testing-linux) via eth1 relay 172.31.101.1 lease-duration 119 offered-duration 43200
2023-06-15T11:52:12+00:00 daemon ibns2.techblue.net dhcpd[29852]: info Option 82: received a REQUEST DHCP packet from relay-agent 172.31.101.1 with a circuit-id of "65:74:68:33" for 172.31.101.253 (00:50:56:00:1a:2b) lease time is undefined seconds. (NEW)
2023-06-15T11:52:12+00:00 daemon ibns2.techblue.net dhcpd[29852]: info DHCPREQUEST for 172.31.101.253 (10.100.0.105) from 00:50:56:00:1a:2b (testing-linux) via 172.31.101.1 TransID 620fbcba
2023-06-15T11:52:12+00:00 daemon ibns2.techblue.net dhcpd[29852]: info DHCPACK on 172.31.101.253 to 00:50:56:00:1a:2b (testing-linux) via eth1 relay 172.31.101.1 lease-duration 43200
2023-06-15T11:52:24+00:00 daemon ibns2.techblue.net dhcpd[29852]: err Unable to add forward map from testing-linux.techblue.net to 172.31.101.253: timed out
2023-06-15T11:52:24+00:00 daemon ibns2.techblue.net dhcpd[29852]: info Addition of forward map for 172.31.101.253 deferred
2023-06-15T11:52:24+00:00 daemon ibns2.techblue.net dhcpd[29852]: info Addition of reverse map for 172.31.101.253 deferred

We see a complete DHCP handshake, and immediately after the DHCP messages, we see an error reporting that DHCP daemon is unable to add the forward-mapping entry.

The IPv4 address here is slightly misleading, as the IPv4 address is part of the DNS record to be added, not the IP address that is unreachable.

During the DDNS update process, the MNAME is contacted, this is the primary authoritative server. This is most likely what the “timed out” message is referring to, that the DHCP server is unable to contact the primary authoritative server.

To find the primary authoritative server, we can analyze the SOA record:

CODE

$ dig @10.200.0.105 techblue.net. SOA +multi
; <<>> DiG 9.11.12 <<>> @10.200.0.105 techblue.net. SOA +multi
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48392
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; ANSWER SECTION:
techblue.net.      28800 IN SOA ibns1.techblue.net. please_set_email.absolutely.nowhere. (
                              7          ; serial
                              10800      ; refresh (3 hours)
                              3600       ; retry (1 hour)
                              2419200    ; expire (4 weeks)
                              900        ; minimum (15 minutes)
                              )

Using DNS lookup, or looking at the Grid configuration, we can find out the IPv4 address for ibns1.techblue.net is 10.100.0.105. We can ping it, but it is not responding to DNS queries:

CODE

$ ping 10.100.0.105 -c3
PING 10.100.0.105 (10.100.0.105) 56(84) bytes of data.
64 bytes from 10.100.0.105: icmp_seq=1 ttl=63 time=4.86 ms
64 bytes from 10.100.0.105: icmp_seq=2 ttl=63 time=0.823 ms
64 bytes from 10.100.0.105: icmp_seq=3 ttl=63 time=0.519 ms

--- 10.100.0.105 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.519/2.068/4.864/1.981 ms
$ dig @10.100.0.105 techblue.net.

; <<>> DiG 9.11.12 <<>> @10.100.0.105 techblue.net.
;; connection timed out; no servers could be reached

We can ping the ibns1, but not lookup DNS, indicating that the service is turned off. We can enable it in Data Management → DNS →

Once DNS Service is enabled, we can check the logs again:

CODE

2023-06-15T12:22:00+00:00 daemon ibns2.techblue.net dhcpd[29852]: info DHCPREQUEST for 172.31.101.253 from 00:50:56:00:1a:2b (testing-linux) via 172.31.101.1 TransID 1c182aae (RENEW)
2023-06-15T12:22:00+00:00 daemon ibns2.techblue.net dhcpd[29852]: info DHCPACK on 172.31.101.253 to 00:50:56:00:1a:2b (testing-linux) via eth1 relay 172.31.101.1 lease-duration 43200 (RENEW)
2023-06-15T12:22:00+00:00 daemon ibns2.techblue.net dhcpd[29852]: err Forward map update for 172.31.101.253 abandoned because of non-retryable failure: NXRRSET
2023-06-15T12:22:00+00:00 daemon ibns2.techblue.net dhcpd[29852]: err Reverse map update for 172.31.101.253 abandoned because of non-retryable failure: NXRRSET
2023-06-15T12:22:00+00:00 daemon ibns2.techblue.net dhcpd[29852]: err Forward map from testing-linux.techblue.net to 172.31.101.253 FAILED: Has an address record but no DHCID, not mine.

The last 2 lines of the log message reads: “no DHCID, not mine”. This indicates that DHCP server tried to add the name testing-linux.techblue.net, but there is already an existing identical name. And there is no DHCID (or TXT) record with the same name, meaning this record was NOT added by DHCP.

DHCP server is configured to NOT override this record, this is why it said “not mine” and stops. We can verify this configuration in Data Management → DHCP → Grid DHCP Properties, IPv4 DDNS → Advanced → TXT (DHCID) Record Handling. The mode is set to “ISC”. Under this mode, we are telling DHCP servers that if there is an existing record with the same name, do not override it.

We can change the behavior by changing the mode to “ISC Transitional”, which tells the DHCP server: ignore the fact this name was manually added here for a reason, overwrite it with the new information from DHCP.

The “ISC Transitional” mode is often used during migration, to move from a legacy environment where there are no TXT or DHCID records.

After a period of time (usually a few weeks), the mode should be changed from “ISC Transitional” to either “ISC” (strict) or “Check-Only” (less strict).

Another alternative is to manually remove the existing AAAA record testing-linux.techblue.net, then re-try the DDNS update.

After we fixed the previous problem, either by removing the existing record or modifying the update mode (from “ISC” to “ISC Transitional”), if we try to obtain a DHCP address again, we get a slightly different set of messages. Below is the syslog from (ibns2) relevant to identify the last problem:

CODE

2023-06-15T13:44:39+00:00 daemon ibns2.techblue.net dhcpd[22155]: info DHCPDISCOVER from 00:50:56:00:1a:2b via 172.31.101.1 TransID 657dfacc
2023-06-15T13:44:40+00:00 daemon ibns2.techblue.net dhcpd[22155]: info DHCPOFFER on 172.31.101.253 to 00:50:56:00:1a:2b (testing-linux) via eth1 relay 172.31.101.1 lease-duration 119 offered-duration 43200
2023-06-15T13:44:40+00:00 daemon ibns2.techblue.net dhcpd[22155]: info Option 82: received a REQUEST DHCP packet from relay-agent 172.31.101.1 with a circuit-id of "65:74:68:33" for 172.31.101.253 (00:50:56:00:1a:2b) lease time is undefined seconds. (NEW)
2023-06-15T13:44:40+00:00 daemon ibns2.techblue.net dhcpd[22155]: info DHCPREQUEST for 172.31.101.253 (10.200.0.105) from 00:50:56:00:1a:2b (testing-linux) via 172.31.101.1 TransID 657dfacc
2023-06-15T13:44:40+00:00 daemon ibns2.techblue.net dhcpd[22155]: info DHCPACK on 172.31.101.253 to 00:50:56:00:1a:2b (testing-linux) via eth1 relay 172.31.101.1 lease-duration 43200
2023-06-15T13:44:40+00:00 daemon ibns2.techblue.net dhcpd[22155]: info Added new forward map from testing-linux.techblue.net to 172.31.101.253
2023-06-15T13:45:13+00:00 daemon ibns2.techblue.net dhcpd[22155]: err Unable to add reverse map from 253.101.31.172.in-addr.arpa. to testing-linux.techblue.net: SERVFAIL
2023-06-15T13:45:13+00:00 daemon ibns2.techblue.net dhcpd[22155]: info Addition of reverse map for 172.31.101.253 deferred

This time we receive an error message (SERVFAIL) when attempting to add the reverse record 253.101.31.172.in-addr.arpa.

Look at the DNS configuration on the Grid, and we will discover that there is no reverse-mapping zone for 172.31.101.253.

To fix this problem, we need to create a reverse-mapping zone to hold the PTR record. The size of the reverse-mapping zone is not important, it can be 172.0.0.0/8, it can be 172.31.0.0/16, or 17.31.101.0/24, as long as there is a reverse-mapping zone to hold the PTR record.

Once done, below are the messages showing a successful DDNS update with all of the problems addressed: a) DNS service on ibns1 started; b) the conflicting AAAA record deleted; c) reverse-mapping zone 172.31.101.0/24 is added.

CODE

2023-06-15T14:26:29+00:00 daemon ibns2.techblue.net dhcpd[6094]: info DHCPREQUEST for 172.31.101.253 from 00:50:56:00:1a:2b via 172.31.101.1 TransID 4cd2aea8
2023-06-15T14:26:29+00:00 daemon ibns2.techblue.net dhcpd[6094]: info DHCPACK on 172.31.101.253 to 00:50:56:00:1a:2b (testing-linux) via eth1 relay 172.31.101.1 lease-duration 43200
2023-06-15T14:26:29+00:00 daemon ibns2.techblue.net dhcpd[6094]: info Added new forward map from testing-linux.techblue.net to 172.31.101.253
2023-06-15T14:26:29+00:00 daemon ibns2.techblue.net dhcpd[6094]: info Added reverse map from 253.101.31.172.in-addr.arpa. to testing-linux.techblue.net

We should be able to see all 3 DNS records created successfully:

A record: testing-linux.techblue.net →31.101.253
DHCID record: testing-linux.techblue.net → [hash value]
PTR record: 172.31.101.253 → testing-linux.techblue.net