kubernetes中的pod不能访问域名问题排查
<h3>一、进入pod可以访问IP，不能访问域名</h3><div class="cnblogs_code">
<pre>bash-<span style="color: rgba(128, 0, 128, 1)">5.0</span># <span style="color: rgba(0, 0, 255, 1)">ping</span><span style="color: rgba(0, 0, 0, 1)"> www.baidu.com
</span><span style="color: rgba(0, 0, 255, 1)">ping</span>: bad address <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">www.baidu.com</span></pre>
</div>
<h3>二、进入目标pod容器，查看/etc/resolv.conf</h3>
<div class="cnblogs_code">
<pre>bash-<span style="color: rgba(128, 0, 128, 1)">5.0</span># <span style="color: rgba(0, 0, 255, 1)">cat</span> /etc/<span style="color: rgba(0, 0, 0, 1)">resolv.conf
</span><span style="color: rgba(0, 0, 0, 1)">nameserver </span><span style="color: rgba(128, 0, 128, 1)">10.96</span>.<span style="color: rgba(128, 0, 128, 1)">0.10</span><span style="color: rgba(0, 0, 0, 1)">
search hl95</span>-<span style="color: rgba(0, 0, 0, 1)">notary.svc.master69.kubernetes.blockchain.hl95.com svc.master69.kubernetes.blockchain.hl95.com master69.kubernetes.blockchain.hl95.com hlqxt
options ndots:</span><span style="color: rgba(128, 0, 128, 1)">5</span></pre>
</div>
<p>可以看到dns服务器IP为0.96.0.10,我们查看下系统的coredns pod容器信息</p>
<div class="cnblogs_code">
<pre># kubectl get pods -n kube-system -o wide |<span style="color: rgba(0, 0, 255, 1)">grep</span><span style="color: rgba(0, 0, 0, 1)"> coredns
coredns</span>-66bff467f8-6w5j5 <span style="color: rgba(128, 0, 128, 1)">1</span>/<span style="color: rgba(128, 0, 128, 1)">1</span> Running <span style="color: rgba(128, 0, 128, 1)">0</span> 3d20h <span style="color: rgba(128, 0, 128, 1)">10.244</span>.<span style="color: rgba(128, 0, 128, 1)">3.9</span> redis-<span style="color: rgba(128, 0, 128, 1)">03</span>.hlqxt <none> <none><span style="color: rgba(0, 0, 0, 1)">
coredns</span>-66bff467f8-h2zgp <span style="color: rgba(128, 0, 128, 1)">1</span>/<span style="color: rgba(128, 0, 128, 1)">1</span> Running <span style="color: rgba(128, 0, 128, 1)">0</span> 3d20h <span style="color: rgba(128, 0, 128, 1)">10.244</span>.<span style="color: rgba(128, 0, 128, 1)">4.9</span> redis-<span style="color: rgba(128, 0, 128, 1)">02</span>.hlqxt <none> <none></pre>
</div>
<p>可以看到两个coredns pod位于两个node节点上，并且状态是running，正常</p>
<p>我们再进一步查看dns service信息</p>
<div class="cnblogs_code">
<pre># kubectl get svc -n kube-system -<span style="color: rgba(0, 0, 0, 1)">o wide
NAME TYPE CLUSTER</span>-IP EXTERNAL-<span style="color: rgba(0, 0, 0, 1)">IP PORT(S) AGE SELECTOR
kube</span>-dns ClusterIP <span style="color: rgba(128, 0, 128, 1)">10.96</span>.<span style="color: rgba(128, 0, 128, 1)">0.10</span> <none> <span style="color: rgba(128, 0, 128, 1)">53</span>/UDP,<span style="color: rgba(128, 0, 128, 1)">53</span>/TCP,<span style="color: rgba(128, 0, 128, 1)">9153</span>/TCP 5d2h k8s-app=kube-dns</pre>
</div>
<p>kube-dns服务的IP正是10.96.0.10，这样我们知道了pod是通过kube-dns 服务来解析域名的，现在的问题是POD无法与kube-dns通信呢？还是coredns本身域名解析有问题呢，我们需要进一步来确认kube-dns 服务后端正确绑定了coredns容器，我们查看endpoint来确认</p>
<div class="cnblogs_code">
<pre># kubectl get endpoints -n kube-system -o wide|<span style="color: rgba(0, 0, 255, 1)">grep</span> kube-<span style="color: rgba(0, 0, 0, 1)">dns
kube</span>-dns <span style="color: rgba(128, 0, 128, 1)">10.244</span>.<span style="color: rgba(128, 0, 128, 1)">3.9</span>:<span style="color: rgba(128, 0, 128, 1)">53</span>,<span style="color: rgba(128, 0, 128, 1)">10.244</span>.<span style="color: rgba(128, 0, 128, 1)">4.9</span>:<span style="color: rgba(128, 0, 128, 1)">53</span>,<span style="color: rgba(128, 0, 128, 1)">10.244</span>.<span style="color: rgba(128, 0, 128, 1)">3.9</span>:<span style="color: rgba(128, 0, 128, 1)">9153</span> + <span style="color: rgba(128, 0, 128, 1)">3</span> <span style="color: rgba(0, 0, 255, 1)">more</span><span style="color: rgba(0, 0, 0, 1)">... 5d2h
# </pre>
</div>
<p>可以看到kube-dns后端正确的绑定了两个coredns pod的IP。</p>
<p>我们再将目标pod中的nameserver 的ip地址改为coredns pod的IP地址，绕过kube-dns服务，直接与coredns pod通信</p>
<div class="cnblogs_code">
<pre>bash-<span style="color: rgba(128, 0, 128, 1)">5.0</span># <span style="color: rgba(0, 0, 255, 1)">cat</span> /etc/<span style="color: rgba(0, 0, 0, 1)">resolv.conf
nameserver </span><span style="color: rgba(128, 0, 128, 1)">10.244</span>.<span style="color: rgba(128, 0, 128, 1)">3.9</span><span style="color: rgba(0, 0, 0, 1)">
#nameserver </span><span style="color: rgba(128, 0, 128, 1)">10.96</span>.<span style="color: rgba(128, 0, 128, 1)">0.10</span><span style="color: rgba(0, 0, 0, 1)">
search hl95</span>-<span style="color: rgba(0, 0, 0, 1)">notary.svc.master69.kubernetes.blockchain.hl95.com svc.master69.kubernetes.blockchain.hl95.com master69.kubernetes.blockchain.hl95.com hlqxt
options ndots:</span><span style="color: rgba(128, 0, 128, 1)">5</span></pre>
</div>
<p>10.244.3.9：为coredns pod其中一个的IP</p>
<p>再执行ping </p>
<div class="cnblogs_code">
<pre>bash-<span style="color: rgba(128, 0, 128, 1)">5.0</span># <span style="color: rgba(0, 0, 255, 1)">ping</span><span style="color: rgba(0, 0, 0, 1)"> www.baidu.com
PING www.baidu.com (</span><span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>): <span style="color: rgba(128, 0, 128, 1)">56</span><span style="color: rgba(0, 0, 0, 1)"> data bytes
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">0</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.281</span><span style="color: rgba(0, 0, 0, 1)"> ms
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">1</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.296</span><span style="color: rgba(0, 0, 0, 1)"> ms
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">2</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.203</span><span style="color: rgba(0, 0, 0, 1)"> ms
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">3</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.233</span><span style="color: rgba(0, 0, 0, 1)"> ms
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">4</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.241</span><span style="color: rgba(0, 0, 0, 1)"> ms
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">5</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.259</span><span style="color: rgba(0, 0, 0, 1)"> ms
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">6</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.270</span><span style="color: rgba(0, 0, 0, 1)"> ms
</span><span style="color: rgba(128, 0, 128, 1)">64</span> bytes from <span style="color: rgba(128, 0, 128, 1)">110.242</span>.<span style="color: rgba(128, 0, 128, 1)">68.3</span>: <span style="color: rgba(0, 0, 255, 1)">seq</span>=<span style="color: rgba(128, 0, 128, 1)">7</span> ttl=<span style="color: rgba(128, 0, 128, 1)">50</span> <span style="color: rgba(0, 0, 255, 1)">time</span>=<span style="color: rgba(128, 0, 128, 1)">9.342</span> ms</pre>
</div>
<p>可以看到域名解析成功</p>
<p>说明coredns pod工作是正常的，应用目标pod也是工作正常的问题出在kube-dns服务与coredns节点通信上，服务与pod之间通信是通过kube-proxy实现</p>
<p> </p><br><br>
来源：https://www.cnblogs.com/sky-cheng/p/14254871.html
頁:
[1]