Requirements:

  • Alibaba Cloud cluster can resolve internal domain names
  • Office network resolves internal domain names + internet access resolution

Solution:

  • For the first requirement, directly use Alibaba Cloud PrivateZone for resolution.
  • For the second requirement, configure internal domain zones in PrivateZone, then synchronize them to the office network’s bind9 server using Alibaba Cloud’s synchronization tool. Use Dnsmasq as the DNS entry point for the office network: forward public queries to public DNS servers, and forward internal domain queries to the bind9 server.

Some may wonder: Why not use bind9 alone to handle all internal resolutions? The main reason is that in practice, bind9 exhibits performance issues when forwarding to multiple DNS servers simultaneously—occasional timeouts occur. In contrast, Dnsmasq handles this scenario significantly better.

1. Configure Alibaba Cloud PrivateZone

Reference: https://help.aliyun.com/document_detail/64627.html

2. Synchronize Alibaba Cloud Zones to bind9

1. Deploy bind9 via docker-compose

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

version: '2'

services:
  bind:
    restart: always
    image: sameersbn/bind:9.16.1-20200524
    environment:
    - ROOT_PASSWORD=DNS2021#
    - WEBMIN_ENABLED=true
    - WEBMIN_INIT_SSL_ENABLED=false
    ports:
    - "15353:53/tcp"
    - "15353:53/udp"
    - "11953:953/tcp"
    - "10000:10000/tcp" # webmin management
    volumes:
    - ./data:/data
    networks:
      - bind9

networks:
  bind9:
    ipam:
      config:
      - subnet: 10.220.0.0/16
        gateway: 10.220.0.1

2. Modify bind configuration files

data/bind/etc/named.conf

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

include "/etc/bind/named.conf.options";
include "/etc/bind/named.conf.local";
include "/etc/bind/named.conf.default-zones";
key "rndc-key" {
    algorithm hmac-sha256;
    secret "tREasaE2Jal1GfwfL5iii3a88eRGKWui41l5h3v89OM=";
};
controls {
	inet 127.0.0.1 port 953 allow { 127.0.0.1; } keys { rndc-key; };
	};
logging {
    channel query_log {
        file "query.log" versions 10 size 50M;
        severity info;
        print-category yes;
        print-severity yes;
        print-time yes;
    };
    category queries {
        query_log;
    };
};

data/bind/etc/named.conf.options

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

options {
	directory "/var/cache/bind";
    dnssec-validation no;
    dnssec-enable no;
    recursion yes;
    allow-recursion { any;};
    allow-transfer { any; };
    allow-query-cache { any; };
    listen-on-v6 { any; };
    listen-on port 53 { any; };
    forward first;
    forwarders {
        192.168.1.211 port 53;
        192.168.1.212 port 53;
        };
    transfer-format many-answers;
    transfers-per-ns 500;
    recursive-clients 100000;
    max-transfer-time-in 5;
    transfers-in 300;
    transfers-out 300;
    querylog yes;
};

data/bind/etc/named.conf.local

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

zone "sd.com" {
	type master;
	file "/etc/bind/zones/sd.com.zone";
    allow-update { 127.0.0.1; };
	};
zone "bgt.sdi" {
	type master;
	file "/etc/bind/zones/bgt.sdi.zone";
    allow-update { 127.0.0.1; };
	};
zone "con.sdi" {
	type master;
	file "/etc/bind/zones/con.sdi.zone";
    allow-update { 127.0.0.1; };
	};

3. Synchronize zone configurations from Alibaba Cloud to bind9

Reference: https://help.aliyun.com/document_detail/102718.html

Create a shell script and schedule it via cron for batch execution.

1

*/5 * * * * /bin/bash /opt/bind9/update.sh

update.sh

1
2
3
4
5

#!/bin/bash
cd /opt/bind9/tools
./Zone_file_sync config.json
chown 101.101 /opt/bind9/data/bind/etc/zones/*
docker exec bind9_bind_1 bash -c "rndc -c /etc/bind/rndc.conf freeze;rndc -c /etc/bind/rndc.conf reload;rndc -c /etc/bind/rndc.conf thaw"

tools/config.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

{
  "accessKeyId": "LIDD5ssssmGExzGsdfsY6sJrSqo",
  "accessKeySecret": "C5N1TTESt74KhSTsswSSSWiz2",
  "zone": [
    {
      "zoneName": "sd.com",
      "zoneId": "2a4dc4e0sdsfa5d36a3b88ab6482saf",
      "filePath": "/opt/bind9/data/bind/etc/zones/sd.com.zone"
    },
    {
      "zoneName": "bgt.sdi",
      "zoneId": "f842ca07ccsd6f35d9e294d55a0c900",
      "filePath": "/opt/bind9/data/bind/etc/zones/bgt.sdi.zone"
    },
    {
      "zoneName": "con.sdi",
      "zoneId": "beb4d911addsf2bd86425ds280e7bbf2",
      "filePath": "/opt/bind9/data/bind/etc/zones/con.sdi.zone"
    }
  ]
}

3. Deploy dnsmasq

1. Install and configure dnsmasq

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# yum -y install dnsmasq
# cat >> /etc/dnsmasq.conf << Tag
port=53
proxy-dnssec
no-hosts  # do not load local /etc/hosts
no-negcache
dns-forward-max=2000
server=114.114.114.114 # upstream DNS server
server=223.5.5.5 # upstream DNS server
server=/sd.com/127.0.0.1#15353  # forward to specific DNS on port
server=/bgt.sdi/127.0.0.1#15353
server=/con.sdi/127.0.0.1#15353
log-queries  # log DNS queries
log-facility=/var/log/dnsmasq/dnsmasq.log # specify log path
log-async=50
cache-size=100000
Tag
# systemctl start dnsmasq
# systemctl enable dnsmasq

2. Configure log rotation policy for dnsmasq:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# cat >> /etc/logrotate.d/dnsmasq Tag
/var/log/dnsmasq/dnsmasq.log {
    notifempty
    daily
    dateext
    rotate 15
    sharedscripts
    postrotate
        [ ! -f /var/run/dnsmasq.pid ] || kill -USR2 `cat /var/run/dnsmasq.pid`
    endscript
}
Tag

Test execution:

1

logrotate -vf /etc/logrotate.conf

4. Adding a New Zone: What Operations Are Needed?

Both bind9 and dnsmasq require updates.

Example: Add test.com

  1. Add to /etc/dnsmasq.conf:

    server=/test.com/127.0.0.1#15353

  2. Add to /opt/bind9/data/bind/etc/named.conf.local:

    1
2
3
4
5
6

    zone "test.com" {
    type master;
    file "/etc/bind/zones/test.com.zone";
    allow-update { 127.0.0.1; };
    forwarders {};
};

5. How to Integrate Internal DNS into K8s Cluster

  1. Update /etc/resolv.conf on cluster nodes to use internal DNS:

    1
2
3

    options timeout:1 attempts:1 rotate
nameserver 192.168.1.211
nameserver 192.168.1.212

  2. Configure CoreDNS to forward unresolved queries:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

    Corefile: |
  .:53 {
      errors
      health
      kubernetes cluster.local in-addr.arpa ip6.arpa {
         pods insecure
         upstream
         fallthrough in-addr.arpa ip6.arpa
      }
      prometheus :9153
      forward . /etc/resolv.conf
      cache 30
      loop
      reload
      loadbalance
  }

With these steps, the complete internal DNS resolution system is now implemented.