k8s中有个重要机制：服务发现，该功能提供调用方通过服务名来进行调用。该功能便是依赖于集群中dns服务，k8s目前使用coreDNS，之前版本用过SkyDNS，KubeDNS
CoreDNS组件 组件官方地址：https://coredns.io/
看一下当前集群中coredns
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 [root@k8s-master ~]# kubectl get cm -n kube-system NAME DATA AGE calico-config 4 84d coredns 1 84d extension-apiserver-authentication 6 84d kube-proxy 2 84d kubeadm-config 2 84d kubelet-config-1.18 1 84d [root@k8s-master ~]# kubectl get cm coredns -n kube-system NAME DATA AGE coredns 1 84d [root@k8s-master ~]# kubectl get cm coredns -n kube-system -o yaml apiVersion: v1 data: Corefile: .:53 { errors health { lameduck 5s } ready kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure fallthrough in-addr.arpa ip6.arpa ttl 30 } prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance } kind: ConfigMap metadata: creationTimestamp: "2020-08-29T04:46:18Z" managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:data: .: {} f:Corefile: {} manager: kubeadm operation: Update time: "2020-08-29T04:46:18Z" name: coredns namespace: kube-system resourceVersion: "185" selfLink: /api/v1/namespaces/kube-system/configmaps/coredns uid: 8482bd6d-f1f6-4b07-a0c7-f1478739caf7
CoreDNS的主要功能是通过插件系统实现的。它实现了一种链式插件的结构，将dns的逻辑抽象成了一个个插件。常见的插件如下：
loadbalance：提供基于dns的负载均衡功能
loop：检测在dns解析过程中出现的简单循环问题
cache：提供前端缓存功能
health：对Endpoint进行健康检查
kubernetes：从kubernetes中读取zone数据
etcd：从etcd读取zone数据，可以用于自定义域名记录
file：从文件中读取zone数据
hosts：使用/etc/hosts文件或者其他文件读取zone数据，可以用于自定义域名记录
auto：从磁盘中自动加载区域文件
reload：定时自动重新加载Corefile配置文件的内容
forward：转发域名查询到上游dns服务器。
proxy：转发特定的域名查询到多个其他dns服务器，同时提供到多个dns服务器的负载均衡功能
prometheus：为prometheus系统提供采集性能指标数据的URL
pprof：在URL路径/debug/pprof下提供运行是的性能数据
log：对dns查询进行日志记录
errors：对错误信息镜像日志记录
coredns配置自定义dns服务器 通过forward转发配置 查看上面的配置中，博主这里是配置在.:53 默认域下方
1 forward . /etc/resolv.conf
这个意思就是dns请求在coredns 内.:53 默认域内没有找到记录时，边会将dns请求转发到/etc/resolv.conf中配置的dns server。注意该/etc/resolv.conf是coredns pod 所有的主机的/etc/resolv.conf
使用forward指定dns 服务器可以用如下方式
通过添加指定根域解析配置 指定一个特定的域名后缀的解析域，例如 xadocker.cn。将所有后缀为 xadocker.cn 的请求都在此处解析，此时我们的configmap配置如下：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 apiVersion: v1 kind: ConfigMap metadata: name: coredns namespace: kube-system data: Corefile: .:53 { log errors health kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure fallthrough in-addr.arpa ip6.arpa } prometheus :9153 cache 30 loop reload loadbalance } xadocker.cn:53 { log errors cache 30 forward . 192.168.44.145 }
上面配置中，去掉了forward . 223.5.5.5 配置，同时我们开启了coredns的log日志，方便我们测试，重启coredns配置后就可以看到以下日志
1 2 3 4 5 .:53 xadocker.cn.:53 [INFO] plugin/reload: Running configuration MD5 = d811d21984480b26f691d81b4e1617e8 CoreDNS-1.6.7 linux/amd64, go1.13.6, da7f65b
在有dig工具的pod内测试解析情况 测试集群外部域名
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # 可以发现此时我们去掉了forward . 223.5.5.5配置后，外网解析均失败 root@dnsutils:/# dig www.baidu.com ; <<>> DiG 9.9.5-9+deb8u19-Debian <<>> www.baidu.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 37226 ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;www.baidu.com. IN A ;; Query time: 1 msec ;; SERVER: 10.96.0.10#53(10.96.0.10) ;; WHEN: Sat Dec 05 11:25:08 UTC 2020 ;; MSG SIZE rcvd: 42 root@dnsutils:/# ping www.baidu.com ping: unknown host www.baidu.com
查看coredns日志
1 2 [INFO] 10.100.235.228:58331 - 31536 "A IN www.baidu.com. udp 42 false 4096" NOERROR - 0 0.000132927s [ERROR] plugin/errors: 2 www.baidu.com. A: plugin/loop: no next plugin found
测试集群内部域名
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 root@dnsutils:/# dig kubernetes.default.svc.cluster.local ; <<>> DiG 9.9.5-9+deb8u19-Debian <<>> kubernetes.default.svc.cluster.local ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52000 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;kubernetes.default.svc.cluster.local. IN A ;; ANSWER SECTION: kubernetes.default.svc.cluster.local. 16 IN A 10.96.0.1 ;; Query time: 0 msec ;; SERVER: 10.96.0.10#53(10.96.0.10) ;; WHEN: Sat Dec 05 11:30:19 UTC 2020 ;; MSG SIZE rcvd: 117 root@dnsutils:/# ping kubernetes PING kubernetes.default.svc.cluster.local (10.96.0.1) 56(84) bytes of data. 64 bytes from kubernetes.default.svc.cluster.local (10.96.0.1): icmp_seq=1 ttl=127 time=1.11 ms 64 bytes from kubernetes.default.svc.cluster.local (10.96.0.1): icmp_seq=2 ttl=127 time=1.29 ms
coredns对应日志为
1 2 [INFO] 10.100.235.228:58004 - 1846 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 106 0.000181724s [INFO] 10.100.235.228:45913 - 2392 "PTR IN 1.0.96.10.in-addr.arpa. udp 40 false 512" NOERROR qr,aa,rd 112 0.000153302s
测试私有zone: xadocker.cn 博主的私有dns服务是用dnsmasq部署的，可以跳转到此篇：用dnsmasq做私有dns解析
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 root@dnsutils:/# dig apitest1.xadocker.cn ; <<>> DiG 9.9.5-9+deb8u19-Debian <<>> apitest1.xadocker.cn ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32326 ;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;apitest1.xadocker.cn. IN A ;; ANSWER SECTION: apitest1.xadocker.cn. 5 IN A 192.168.44.145 ;; Query time: 2 msec ;; SERVER: 10.96.0.10#53(10.96.0.10) ;; WHEN: Sat Dec 05 11:39:25 UTC 2020 ;; MSG SIZE rcvd: 85 root@dnsutils:/# ping apitest1.xadocker.cn PING apitest1.xadocker.cn (192.168.44.145) 56(84) bytes of data. 64 bytes from 192.168.44.145: icmp_seq=1 ttl=63 time=0.575 ms 64 bytes from 192.168.44.145: icmp_seq=2 ttl=63 time=0.972 ms
对应coredns日志
1 2 3 4 [INFO] 10.100.235.228:50876 - 39995 "A IN apitest1.xadocker.cn.default.svc.cluster.local. udp 64 false 512" NXDOMAIN qr,aa,rd 157 0.000167575s [INFO] 10.100.235.228:43162 - 51676 "A IN apitest1.xado cker.cn.svc.cluster.local. udp 56 false 512" NXDOMAIN qr,aa,rd 149 0.000161292s [INFO] 10.100.235.228:42107 - 43651 "A IN apitest1.xadocker.cn.cluster.local. udp 52 false 512" NXDOMAIN qr,aa,rd 145 0.000152128s [INFO] 10.100.235.228:53563 - 39760 "A IN apitest1.xadocker.cn. udp 38 false 512" NOERROR qr,aa,rd,ra 74 0.001193359s
hosts插件 该功能可以实现pod中的硬解析
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 apiVersion: v1 kind: ConfigMap metadata: name: coredns namespace: kube-system data: Corefile: .:53 { log errors health kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure fallthrough in-addr.arpa ip6.arpa } hosts { 192.168.44 178 nexus.xadocker.cn 192.168.44 179 harbor.xadocker.cn fallthrough } forward . 114.114.114.114 prometheus :9153 cache 30 loop reload loadbalance }
配置完后重起coredns，在pod中测试
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 root@dnsutils:/# nslookup nexus.xadocker.cn Server: 10.96.0.10 Address: 10.96.0.10#53 Name: nexus.xadocker.cn Address: 192.168.44.178 # 查看pod中hosts也没有上述记录，看来是从coredns中获取的，coredns pod中hosts也没有相应记录 root@dnsutils:/# cat /etc/hosts # Kubernetes-managed hosts file. 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet fe00::0 ip6-mcastprefix fe00::1 ip6-allnodes fe00::2 ip6-allrouters 10.100.235.218 dnsutils
其实还存在另外一种方式在pod中配置dns解析，那就是在pod里配置HostAliases
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 [root@k8s-master ~]# kubectl explain deploy.spec.template.spec.hostAliases KIND: Deployment VERSION: apps/v1 RESOURCE: hostAliases <[]Object> DESCRIPTION: HostAliases is an optional list of hosts and IPs that will be injected into the pod's hosts file if specified. This is only valid for non-hostNetwork pods. HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod's hosts file. FIELDS: hostnames <[]string> Hostnames for the above IP address. ip <string> IP address of the host file entry.
样例demo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 [root@k8s-master ~]# cat dnsutils.yaml apiVersion: v1 kind: Pod metadata: name: dnsutils namespace: default spec: containers: - name: dnsutils image: registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 command: - sleep - "infinity" imagePullPolicy: IfNotPresent restartPolicy: Always hostAliases: - ip: "192.168.44.168" hostnames: - "nexus.xadocker.cn" - "mvn.xadocker.cn" - ip: "192.168.44.169" hostnames: - "harbor.xadocker.cn" [root@k8s-master ~]# kubectl delete -f dnsutils.yaml [root@k8s-master ~]# kubectl apply -f dnsutils.yaml [root@k8s-master ~]# kubectl exec -it dnsutils -- /bin/bash root@dnsutils:/# cat /etc/hosts # Kubernetes-managed hosts file. 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet fe00::0 ip6-mcastprefix fe00::1 ip6-allnodes fe00::2 ip6-allrouters 10.100.235.208 dnsutils # Entries added by HostAliases. 192.168.44.178 nexus.xadocker.cn mvn.xadocker.cn 192.168.44.179 harbor.xadocker.cn root@dnsutils:/# ping harbor.xadocker.cn PING harbor.xadocker.cn (192.168.44.169) 56(84) bytes of data.
