kubernetes安装ingress-nginx

干千翻 發表於 2021-11-2 21:16:00

版本说明
网上的资料一般是基于v0.30.0来安装，但是对于kubernetes@1.22来说要安装ingress-nginx@v1.0.0以上版本（目前最新版本是v1.0.4，本文采用v1.0.0），原因是 kubectl@v1.22版本不再支持v1beta1
如果安装ingress-nginx@v0.30.0版本后启动pod有如下问题
<div class="cnblogs_code">
<pre>Failed to list *v1beta1.Ingress: the server could not find the requested resource</pre>
</div>
有一个版本的支持情况（https://github.com/kubernetes/ingress-nginx/）
<img src="https://img2020.cnblogs.com/blog/490358/202111/490358-20211106214928621-1210857692.png">
我的node环境
<div class="cnblogs_code">
<pre>$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
kbs1 Ready <none> 21d v1.22.2 192.168.0.106 <none> Ubuntu 20.04.3 LTS 5.4.0-88-generic docker://20.10.9
kbs2 Ready <none> 21d v1.22.2 192.168.0.107 <none> Ubuntu 20.04.3 LTS 5.4.0-88-generic docker://20.10.9
kbsm Ready control-plane,master 25d v1.22.2 192.168.0.105 <none> Ubuntu 20.04.3 LTS 5.4.0-88-generic docker://20.10.</pre>
</div>
 
官方的yaml中使用的镜像不能下载，所以要改成dockerhub上的镜像
<div class="cnblogs_code">
<pre># controller镜像对应地址，所有的node都要docker pull
image: k8s.gcr.io/ingress-nginx/controller:v1.0.0-> https://hub.docker.com/r/willdockerhub/ingress-nginx-controller
docker pull willdockerhub/ingress-nginx-controller:v1.0.0

image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 -> https://hub.docker.com/r/jettech/kube-webhook-certgen/tags
docker pull jettech/kube-webhook-certgen:v1.0.0</pre>
</div>
 原yaml文件
<div class="cnblogs_Highlighter">
<pre class="brush:bash;gutter:true;">wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/cloud/deploy.yaml
</pre>
</div>
国内访问这个地址不稳定，我把他复制下来
<div class="cnblogs_code"><img id="code_img_closed_98648cf6-83e8-46cb-a59f-e4c81d968fbe" class="code_img_closed lazyload" data-src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_98648cf6-83e8-46cb-a59f-e4c81d968fbe" class="code_img_opened lazyload" style="display: none" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_98648cf6-83e8-46cb-a59f-e4c81d968fbe" class="cnblogs_code_hide">
<pre>apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx

---
# Source: ingress-nginx/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
automountServiceAccountToken: true
---
# Source: ingress-nginx/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
data:
---
# Source: ingress-nginx/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
name: ingress-nginx
rules:
- apiGroups:
 - ''
resources:
 - configmaps
 - endpoints
 - nodes
 - pods
 - secrets
verbs:
 - list
 - watch
- apiGroups:
 - ''
resources:
 - nodes
verbs:
 - get
- apiGroups:
 - ''
resources:
 - services
verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses
verbs:
 - get
 - list
 - watch
- apiGroups:
 - ''
resources:
 - events
verbs:
 - create
 - patch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses/status
verbs:
 - update
- apiGroups:
 - networking.k8s.io
resources:
 - ingressclasses
verbs:
 - get
 - list
 - watch
---
# Source: ingress-nginx/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
 - ''
resources:
 - namespaces
verbs:
 - get
- apiGroups:
 - ''
resources:
 - configmaps
 - pods
 - secrets
 - endpoints
verbs:
 - get
 - list
 - watch
- apiGroups:
 - ''
resources:
 - services
verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses
verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses/status
verbs:
 - update
- apiGroups:
 - networking.k8s.io
resources:
 - ingressclasses
verbs:
 - get
 - list
 - watch
- apiGroups:
 - ''
resources:
 - configmaps
resourceNames:
 - ingress-controller-leader
verbs:
 - get
 - update
- apiGroups:
 - ''
resources:
 - configmaps
verbs:
 - create
- apiGroups:
 - ''
resources:
 - events
verbs:
 - create
 - patch
---
# Source: ingress-nginx/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-service-webhook.yaml
apiVersion: v1
kind: Service
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
type: ClusterIP
ports:
- name: https-webhook
 port: 443
 targetPort: webhook
 appProtocol: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
annotations:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- name: http
 port: 80
 protocol: TCP
 targetPort: http
 appProtocol: http
- name: https
 port: 443
 protocol: TCP
 targetPort: https
 appProtocol: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
selector:
matchLabels:
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/component: controller
revisionHistoryLimit: 10
minReadySeconds: 0
template:
metadata:
 labels:
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/component: controller
spec:
 dnsPolicy: ClusterFirst
 containers:
 - name: controller
 image: k8s.gcr.io/ingress-nginx/controller:v1.0.0@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6
 imagePullPolicy: IfNotPresent
 lifecycle:
 preStop:
 exec:
 command:
 - /wait-shutdown
 args:
 - /nginx-ingress-controller
 - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
 - --election-id=ingress-controller-leader
 - --controller-class=k8s.io/ingress-nginx
 - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
 - --validating-webhook=:8443
 - --validating-webhook-certificate=/usr/local/certificates/cert
 - --validating-webhook-key=/usr/local/certificates/key
 securityContext:
 capabilities:
 drop:
 - ALL
 add:
 - NET_BIND_SERVICE
 runAsUser: 101
 allowPrivilegeEscalation: true
 env:
 - name: POD_NAME
 valueFrom:
 fieldRef:
 fieldPath: metadata.name
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 - name: LD_PRELOAD
 value: /usr/local/lib/libmimalloc.so
 livenessProbe:
 failureThreshold: 5
 httpGet:
 path: /healthz
 port: 10254
 scheme: HTTP
 initialDelaySeconds: 10
 periodSeconds: 10
 successThreshold: 1
 timeoutSeconds: 1
 readinessProbe:
 failureThreshold: 3
 httpGet:
 path: /healthz
 port: 10254
 scheme: HTTP
 initialDelaySeconds: 10
 periodSeconds: 10
 successThreshold: 1
 timeoutSeconds: 1
 ports:
 - name: http
 containerPort: 80
 protocol: TCP
 - name: https
 containerPort: 443
 protocol: TCP
 - name: webhook
 containerPort: 8443
 protocol: TCP
 volumeMounts:
 - name: webhook-cert
 mountPath: /usr/local/certificates/
 readOnly: true
 resources:
 requests:
 cpu: 100m
 memory: 90Mi
 nodeSelector:
 kubernetes.io/os: linux
 serviceAccountName: ingress-nginx
 terminationGracePeriodSeconds: 300
 volumes:
 - name: webhook-cert
 secret:
 secretName: ingress-nginx-admission
---
# Source: ingress-nginx/templates/controller-ingressclass.yaml
# We don't support namespaced ingressClass yet
# So a ClusterRole and a ClusterRoleBinding is required
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: nginx
namespace: ingress-nginx
spec:
controller: k8s.io/ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
# before changing this value, check the required kubernetes version
# https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
name: ingress-nginx-admission
webhooks:
- name: validate.nginx.ingress.kubernetes.io
matchPolicy: Equivalent
rules:
 - apiGroups:
 - networking.k8s.io
 apiVersions:
 - v1
 operations:
 - CREATE
 - UPDATE
 resources:
 - ingresses
failurePolicy: Fail
sideEffects: None
admissionReviewVersions:
 - v1
clientConfig:
 service:
 namespace: ingress-nginx
 name: ingress-nginx-controller-admission
 path: /networking/v1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: ingress-nginx-admission
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: ingress-nginx-admission
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
rules:
- apiGroups:
 - admissionregistration.k8s.io
resources:
 - validatingwebhookconfigurations
verbs:
 - get
 - update
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: ingress-nginx-admission
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: ingress-nginx-admission
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
rules:
- apiGroups:
 - ''
resources:
 - secrets
verbs:
 - get
 - create
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: ingress-nginx-admission
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: ingress-nginx-admission-create
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
spec:
template:
metadata:
 name: ingress-nginx-admission-create
 labels:
 helm.sh/chart: ingress-nginx-4.0.1
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/version: 1.0.0
 app.kubernetes.io/managed-by: Helm
100 19023100 19023 0 015218 00:00:010:00:01 --:--:-- 15230-webhook
spec:
 containers:
 - name: create
 image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068
 imagePullPolicy: IfNotPresent
 args:
 - create
 - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
 - --namespace=$(POD_NAMESPACE)
 - --secret-name=ingress-nginx-admission
 env:
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 restartPolicy: OnFailure
 serviceAccountName: ingress-nginx-admission
 nodeSelector:
 kubernetes.io/os: linux
 securityContext:
 runAsNonRoot: true
 runAsUser: 2000
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: ingress-nginx-admission-patch
namespace: ingress-nginx
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
spec:
template:
metadata:
 name: ingress-nginx-admission-patch
 labels:
 helm.sh/chart: ingress-nginx-4.0.1
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/version: 1.0.0
 app.kubernetes.io/managed-by: Helm
 app.kubernetes.io/component: admission-webhook
spec:
 containers:
 - name: patch
 image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068
 imagePullPolicy: IfNotPresent
 args:
 - patch
 - --webhook-name=ingress-nginx-admission
 - --namespace=$(POD_NAMESPACE)
 - --patch-mutating=false
 - --secret-name=ingress-nginx-admission
 - --patch-failure-policy=Fail
 env:
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 restartPolicy: OnFailure
 serviceAccountName: ingress-nginx-admission
 nodeSelector:
 kubernetes.io/os: linux
 securityContext:
 runAsNonRoot: true
 runAsUser: 2000</pre>
</div>
View Code</div>
需要在原deploy.yaml文件上修改修改后再部署
<div class="cnblogs_code">
<pre># 修改点如下
1：k8s.gcr.io/ingress-nginx/controller:v1.0.0@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6 改为：willdockerhub/ingress-nginx-controller:v1.0.0

2：k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068 改为：jettech/kube-webhook-certgen:v1.0.0

3：Deployment修改点
template:
metadata:
 labels:
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/component: controller
spec:
 dnsPolicy: ClusterFirstWithHostNet#既能使用宿主机DNS，又能使用集群DNS原文line：319
 hostNetwork: true #与宿主机共享网络
 nodeName: k8s-master-1 #设置只能在k8s-master-1节点运行
 tolerations: #设置能容忍master污点
 - key: node-role.kubernetes.io/master
 operator: Exists
 containers:
 - name: controller
 image: willdockerhub/ingress-nginx-controller:v1.0.0
 imagePullPolicy: IfNotPresent </pre>
</div>
修改后的yaml文件
<div class="cnblogs_code"><img id="code_img_closed_b9a4d598-2fd9-419c-ba7c-b6ea243946ef" class="code_img_closed lazyload" data-src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_b9a4d598-2fd9-419c-ba7c-b6ea243946ef" class="code_img_opened lazyload" style="display: none" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_b9a4d598-2fd9-419c-ba7c-b6ea243946ef" class="cnblogs_code_hide">
<pre>apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx

---
# Source: ingress-nginx/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
automountServiceAccountToken: true
---
# Source: ingress-nginx/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
data:
---
# Source: ingress-nginx/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
name: ingress-nginx
rules:
- apiGroups:
 - ''
resources:
 - configmaps
 - endpoints
 - nodes
 - pods
 - secrets
verbs:
 - list
 - watch
- apiGroups:
 - ''
resources:
 - nodes
verbs:
 - get
- apiGroups:
 - ''
resources:
 - services
verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses
verbs:
 - get
 - list
 - watch
- apiGroups:
 - ''
resources:
 - events
verbs:
 - create
 - patch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses/status
verbs:
 - update
- apiGroups:
 - networking.k8s.io
resources:
 - ingressclasses
verbs:
 - get
 - list
 - watch
---
# Source: ingress-nginx/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
 - ''
resources:
 - namespaces
verbs:
 - get
- apiGroups:
 - ''
resources:
 - configmaps
 - pods
 - secrets
 - endpoints
verbs:
 - get
 - list
 - watch
- apiGroups:
 - ''
resources:
 - services
verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses
verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
resources:
 - ingresses/status
verbs:
 - update
- apiGroups:
 - networking.k8s.io
resources:
 - ingressclasses
verbs:
 - get
 - list
 - watch
- apiGroups:
 - ''
resources:
 - configmaps
resourceNames:
 - ingress-controller-leader
verbs:
 - get
 - update
- apiGroups:
 - ''
resources:
 - configmaps
verbs:
 - create
- apiGroups:
 - ''
resources:
 - events
verbs:
 - create
 - patch
---
# Source: ingress-nginx/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-service-webhook.yaml
apiVersion: v1
kind: Service
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
type: ClusterIP
ports:
- name: https-webhook
 port: 443
 targetPort: webhook
 appProtocol: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
selector:
matchLabels:
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/component: controller
revisionHistoryLimit: 10
minReadySeconds: 0
template:
metadata:
 labels:
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/component: controller
spec:
 hostNetwork: true
 dnsPolicy: ClusterFirst
 containers:
 - name: controller
 image: registry.cn-beijing.aliyuncs.com/kole_chang/controller:v1.0.0
 imagePullPolicy: IfNotPresent
 lifecycle:
 preStop:
 exec:
 command:
 - /wait-shutdown
 args:
 - /nginx-ingress-controller
 - --election-id=ingress-controller-leader
 - --controller-class=k8s.io/ingress-nginx
 - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
 - --validating-webhook=:8443
 - --validating-webhook-certificate=/usr/local/certificates/cert
 - --validating-webhook-key=/usr/local/certificates/key
 - --watch-ingress-without-class=true
 securityContext:
 capabilities:
 drop:
 - ALL
 add:
 - NET_BIND_SERVICE
 runAsUser: 101
 allowPrivilegeEscalation: true
 env:
 - name: POD_NAME
 valueFrom:
 fieldRef:
 fieldPath: metadata.name
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 - name: LD_PRELOAD
 value: /usr/local/lib/libmimalloc.so
 livenessProbe:
 failureThreshold: 5
 httpGet:
 path: /healthz
 port: 10254
 scheme: HTTP
 initialDelaySeconds: 10
 periodSeconds: 10
 successThreshold: 1
 timeoutSeconds: 1
 readinessProbe:
 failureThreshold: 3
 httpGet:
 path: /healthz
 port: 10254
 scheme: HTTP
 initialDelaySeconds: 10
 periodSeconds: 10
 successThreshold: 1
 timeoutSeconds: 1
 ports:
 - name: http
 containerPort: 80
 protocol: TCP
 - name: https
 containerPort: 443
 protocol: TCP
 - name: webhook
 containerPort: 8443
 protocol: TCP
 volumeMounts:
 - name: webhook-cert
 mountPath: /usr/local/certificates/
 readOnly: true
 resources:
 requests:
 cpu: 100m
 memory: 90Mi
 nodeSelector:
 kubernetes.io/os: linux
 serviceAccountName: ingress-nginx
 terminationGracePeriodSeconds: 300
 volumes:
 - name: webhook-cert
 secret:
 secretName: ingress-nginx-admission
---
# Source: ingress-nginx/templates/controller-ingressclass.yaml
# We don't support namespaced ingressClass yet
# So a ClusterRole and a ClusterRoleBinding is required
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: nginx
namespace: ingress-nginx
spec:
controller: k8s.io/ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
# before changing this value, check the required kubernetes version
# https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
name: ingress-nginx-admission
webhooks:
- name: validate.nginx.ingress.kubernetes.io
matchPolicy: Equivalent
rules:
 - apiGroups:
 - networking.k8s.io
 apiVersions:
 - v1
 operations:
 - CREATE
 - UPDATE
 resources:
 - ingresses
failurePolicy: Fail
sideEffects: None
admissionReviewVersions:
 - v1
clientConfig:
 service:
 namespace: ingress-nginx
 name: ingress-nginx-controller-admission
 path: /networking/v1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: ingress-nginx-admission
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: ingress-nginx-admission
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
rules:
- apiGroups:
 - admissionregistration.k8s.io
resources:
 - validatingwebhookconfigurations
verbs:
 - get
 - update
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: ingress-nginx-admission
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: ingress-nginx-admission
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
rules:
- apiGroups:
 - ''
resources:
 - secrets
verbs:
 - get
 - create
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: ingress-nginx-admission
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: ingress-nginx-admission-create
namespace: ingress-nginx
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
spec:
template:
metadata:
 name: ingress-nginx-admission-create
 labels:
 helm.sh/chart: ingress-nginx-4.0.1
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/version: 1.0.0
 app.kubernetes.io/managed-by: Helm
 app.kubernetes.io/component: admission-webhook
spec:
 containers:
 - name: create
 image: registry.cn-beijing.aliyuncs.com/kole_chang/kube-webhook-certgen:v1.0
 imagePullPolicy: IfNotPresent
 args:
 - create
 - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
 - --namespace=$(POD_NAMESPACE)
 - --secret-name=ingress-nginx-admission
 env:
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 restartPolicy: OnFailure
 serviceAccountName: ingress-nginx-admission
 nodeSelector:
 kubernetes.io/os: linux
 securityContext:
 runAsNonRoot: true
 runAsUser: 2000
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: ingress-nginx-admission-patch
namespace: ingress-nginx
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
helm.sh/chart: ingress-nginx-4.0.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: admission-webhook
spec:
template:
metadata:
 name: ingress-nginx-admission-patch
 labels:
 helm.sh/chart: ingress-nginx-4.0.1
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/version: 1.0.0
 app.kubernetes.io/managed-by: Helm
 app.kubernetes.io/component: admission-webhook
spec:
 containers:
 - name: patch
 image: registry.cn-beijing.aliyuncs.com/kole_chang/kube-webhook-certgen:v1.0
 imagePullPolicy: IfNotPresent
 args:
 - patch
 - --webhook-name=ingress-nginx-admission
 - --namespace=$(POD_NAMESPACE)
 - --patch-mutating=false
 - --secret-name=ingress-nginx-admission
 - --patch-failure-policy=Fail
 env:
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 restartPolicy: OnFailure
 serviceAccountName: ingress-nginx-admission
 nodeSelector:
 kubernetes.io/os: linux
 securityContext:
 runAsNonRoot: true
 runAsUser: 2000</pre>
</div>
View Code</div>
开始执行
<div class="cnblogs_code">
<pre>$ kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created</pre>
</div>
 查看生成状态
<div class="cnblogs_code">
<pre>$ kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create--1-ncg9g 0/1 Completed 0 4h58m
pod/ingress-nginx-admission-patch--1-dmtp2 0/1 Completed 0 4h58m
pod/ingress-nginx-controller-7cv4l 1/1 Running 0 4h58m
pod/ingress-nginx-controller-dbz2b 1/1 Running 0 4h58m
pod/ingress-nginx-controller-ddl5c 1/1 Running 0 4h58m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller-admission ClusterIP 10.109.220.110 <none> 443/TCP 4h58m

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/ingress-nginx-controller 3 3 3 3 3 kubernetes.io/os=linux 4h58m

NAME COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create 1/1 6s 4h58m
job.batch/ingress-nginx-admission-patch 1/1 6s 4h58m</pre>
</div>
这个时候ingress-nginx就安装完了，在kubernets之外的机器上访问每个节点，nginx都可以访问了，先不用管404的错误
<div class="cnblogs_code">
<pre>$ curl http://kbsm
% Total % Received % XferdAverage Speed Time Time TimeCurrent
 DloadUpload Total Spent LeftSpeed
100 146100 146 0 0 7684 0 --:--:-- --:--:-- --:--:--8588<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

$ curl http://kbs1
% Total % Received % XferdAverage Speed Time Time TimeCurrent
 DloadUpload Total Spent LeftSpeed
100 146100 146 0 0 481 0 --:--:-- --:--:-- --:--:-- 481<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

$ curl http://kbs2
% Total % Received % XferdAverage Speed Time Time TimeCurrent
 DloadUpload Total Spent LeftSpeed
100 146100 146 0 0 1505 0 --:--:-- --:--:-- --:--:--1536<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html></pre>
</div>
 下面生成一个自己写的ingress，关联到一个我自己写的叫gostarter-svc的服务上，参见本文系列的kubernets创建Service
<div class="cnblogs_code">
<pre># ingress-gostarter.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-gostarter
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules: # 要在kubernets的node之外的节点上，一般是你的笔记本配置，windows下是Hosts配置域名解析，192.168.0.105 gostarter，因为使用的是DamonSet所以IP使用任意一个node的ip都可以
- host: gostarter.com
http:
 paths:
 - path: /
 pathType: Prefix
 backend:
 service:
 name: gostarter-svc
 port:
 # gostarter-svc的端口是8000
 number: 8000 </pre>
</div>
 根据上面红色字体配置host之后，在笔记本上访问
<div class="cnblogs_code">
<pre>$ curl http://gostarter.com/hostname
% Total % Received % XferdAverage Speed Time Time TimeCurrent
 DloadUpload Total Spent LeftSpeed
100 30100 30 0 0 1111 0 --:--:-- --:--:-- --:--:--1153
gostarter-dep-df898587f-wq2cz</pre>
</div>
完美，至此ingress-nginx安装完成
dockerhub也可以使用下面两个镜像地址
docker pull serenade11/ingress-nginx-controller:v1.0.0 docker pull serenade11/kube-webhook-certgen:v1.0
 
参考资料
<ol>
<li>ingress-nginx官方网站</li>
<li>k8s之ingress-nginx部署(v1.0.0)</li>
<li>nginx ingress最后的倔强: admission webhook</li>
<li>k8s ingress原理及ingress-nginx部署测试</li>
<li>K8S_1.22部署ingress （目前最新ingress-nginx-controller-v1.0.0）</li>

</ol>

</div>
<div id="MySignature" role="contentinfo">

作者    ：秋时 
本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接。
 
来源：https://www.cnblogs.com/Netsharp/p/15501138.html

頁: [1]

琼殿技术论坛's Archiver

kubernetes安装ingress-nginx