Kubernetes 部署 Nginx Ingress Controller 之 nginxinc/kubernetes-ingress
<p>更新：这里用的是 nginxinc/kubernetes-ingress ，还有个 kubernetes/ingress-nginx ，它们的区别见 Differences Between nginxinc/kubernetes-ingress and kubernetes/ingress-nginx Ingress Controllers ，后来我们选用了 <code>kubernetes/ingress-nginx</code> ，详见博文。</p><p>开始天真地以为只要写一个 ingress 配置文件并部署好就行了。</p>
<pre><code class="language-yaml">apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cnblogs-ingress
spec:
rules:
- host: q.cnblogs.com
http:
paths:
- backend:
serviceName: q-web
servicePort: 80
</code></pre>
<pre><code class="language-bash"># kubectl apply -f cnblogs-ingress.yaml
# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
cnblogs-ingress q.cnblogs.com 80 6h18
</code></pre>
<p>但部署后发现所有 node 服务器上没有任何进程监听 80 端口，显然不对。</p>
<p>从 k8s 帮助文档中知道了答案：</p>
<blockquote>
<p>You must have an ingress controller to satisfy an Ingress. Only creating an Ingress resource has no effect.<br>
In order for the Ingress resource to work, the cluster must have an ingress controller running.<br>
Unlike other types of controllers which run as part of the kube-controller-manager binary, Ingress controllers are not started automatically with a cluster. Use this page to choose the ingress controller implementation that best fits your cluster.</p>
</blockquote>
<p>原来 k8s 没有内置 ingress controller ，需要安装第三方的 ingress controller ，比如 nginx ingress controller ，上面通过 cnblogs-ingress.yaml 只是创建了 ingress 资源。那为什么通过 deployment.yaml 创建了 deployment 资源就能正常部署 pod ？那是因为 kube-controller-manager 中内置了 deployment controller 。</p>
<p>我们选用 nginx ingress controller ，部署操作步骤如下（参考文档）：<br>
1）从 github 上签出 kubernetes-ingress 仓库</p>
<pre><code class="language-bash">$ git clone https://github.com/nginxinc/kubernetes-ingress/
$ cd kubernetes-ingress
$ git checkout v1.6.1 -f
$ cd deployments
</code></pre>
<p>2）创建 namespace 与 ServiceAccount ，都叫 nginx-ingress</p>
<pre><code class="language-bash">kubectl apply -f common/ns-and-sa.yaml
</code></pre>
<p>3）创建cluster role 与 cluster role binding</p>
<pre><code class="language-bash">kubectl apply -f rbac/rbac.yaml
</code></pre>
<p>4）创建 secret<br>
使用自己的证书文件创建 secret</p>
<pre><code class="language-bash">kubectl create secret tls default-server-secret --cert=path/to/cert.pem --key=path/to/key.pem
</code></pre>
<p>或者使用 nginx-ingress 自带的证书创建 sescret</p>
<pre><code class="language-bash">kubectl apply -f common/default-server-secret.yaml
</code></pre>
<p>5）创建 ConfigMap</p>
<pre><code class="language-bash">kubectl apply -f common/nginx-config.yaml
</code></pre>
<p>6）创建custom resource definitions</p>
<pre><code class="language-bash">kubectl apply -f common/custom-resource-definitions.yaml
</code></pre>
<p>7）创建 DaemonSet</p>
<pre><code class="language-bash">kubectl apply -f daemon-set/nginx-ingress.yaml
</code></pre>
<p>8）查看 pod 是否部署成功</p>
<pre><code class="language-bash">$ kubectl get pods --namespace=nginx-ingress 1 ↵
NAME READY STATUS RESTARTS AGE
nginx-ingress-7xdzp 1/1 Running 5 12m
nginx-ingress-rs4th 1/1 Running 0 114s
nginx-ingress-w2fnh 1/1 Running 0 12m
nginx-ingress-z54r6 1/1 Running 5 12m
</code></pre>
<p>9）创建监听 31080 端口的 NodePort 类型的 service</p>
<p>配置文件 nodeport.yaml （去掉了443端口）<br>
注0：nodePort 只能使用 30000-32767 范围的端口。<br>
注1：去掉了443端口，我们在最前端使用了阿里云负载均衡，请求都通过 http 转发。</p>
<pre><code class="language-yaml">apiVersion: v1
kind: Service
metadata:
name: nginx-ingress
namespace: nginx-ingress
spec:
type: NodePort
ports:
- nodePort: 31080
port: 80
targetPort: 80
protocol: TCP
name: http
selector:
app: nginx-ingress
</code></pre>
<p>部署命令</p>
<pre><code class="language-bash">kubectl apply -f service/nodeport.yaml
</code></pre>
<p>10）检查 nginx-ingress 部署成功</p>
<p>进入 nginx-ingress 容器</p>
<pre><code class="language-bash">kubectl exec -it daemonset/nginx-ingress -n nginx-ingress /bin/bash
</code></pre>
<p>查看 nginx 配置</p>
<pre><code class="language-bash">cat /etc/nginx/conf.d/production-cnblogs-ingress.conf
</code></pre>
<p>确认 ingress 中添加的转发规则已被导入</p>
<pre><code class="language-conf">upstream production-cnblogs-ingress-q.cnblogs.com-q-web-80 {
zone production-cnblogs-ingress-q.cnblogs.com-q-web-80 256k;
random two least_conn;
server 192.168.107.211:80 max_fails=1 fail_timeout=10s max_conns=0;
server 192.168.186.72:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
listen 80;
server_tokens on;
server_name q.cnblogs.com;
location / {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://production-cnblogs-ingress-q.cnblogs.com-q-web-80;
}
}
</code></pre>
<p>至此 nginx-ingress 部署成功。</p>
<h4 id="解决转发-x-forwarded-proto-请求头问题">解决转发 X-Forwarded-Proto 请求头问题</h4>
<p>解决方法：在 ingress 配置文件中添加 <code>nginx.org/redirect-to-https: "true"</code>，详见博问 K8s Nginx Ingress Controller 转发 X-Forwarded-Proto 请求头的问题</p>
<pre><code class="language-yaml">apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cnblogs-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.org/redirect-to-https: "true"
</code></pre>
<h4 id="添加-proxy_set_header-配置">添加 proxy_set_header 配置</h4>
<p>通过 ingress 的 nginx.org/location-snippets 注解添加如下的配置：</p>
<pre><code class="language-yaml">apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cnblogs-ingress
annotations:
nginx.org/location-snippets: |
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
</code></pre>
<h4 id="七层负载均衡对-nginx-ingress-进行健康检查问题">七层负载均衡对 nginx-ingress 进行健康检查问题</h4>
<p>详见博问 阿里云负载均衡对 K8s Nginx Ingress 的健康检查问题</p>
<h4 id="基于二级域名自动转发到-service-的实现方法">基于二级域名自动转发到 service 的实现方法</h4>
<p>详见博问 K8s Ingress 如何自动根据主机名中的二级域名匹配 service</p>
<h4 id="应用获取不到客户端真实-ip-地址的问题">应用获取不到客户端真实 IP 地址的问题</h4>
<p>详见博问 K8s 中 ASP.NET Core 应用获取不到客户端真实 IP 地址</p><br><br>
来源：https://www.cnblogs.com/dudu/p/12236483.html
頁:
[1]