在k8s上部署ingress-nginx并使用

[TOC]

部署nginx-ingress

环境准备

准备一套k8s环境

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# kubectl get cs
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health":"true"}
# kubectl cluster-info
Kubernetes master is running at https://10.122.17.200:6443
KubeDNS is running at https://10.122.17.200:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 4d1h v1.15.3
k8s-node1 Ready <none> 4d1h v1.15.3
k8s-node2 Ready <none> 4d v1.15.3
k8s-node3 Ready <none> 4d v1.15.3

下载安装nginx-ingressyaml文件

1
# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

设置nginx-ingress-controller调度节点

1
2
3
4
# kubectl label nodes k8s-node1 hosts=nginx-ingress-controller
# kubectl get node -l hosts=nginx-ingress-controller --show-labels
NAME STATUS ROLES AGE VERSION LABELS
k8s-node1 Ready <none> 4d6h v1.15.3 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,hosts=nginx-ingress-controller,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node1,kubernetes.io/os=linux

修改mandatory.yaml文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# vim mandatory.yaml
......
apiVersion: apps/v1
kind: Deployment
......
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
......
spec:
hostNetwork: true # 使用hostNetwork 模式
nodeSelector: # 使用nodeSelector
hosts: nginx-ingress-controller # 选择标签为hosts=nginx-ingress-controller节点
serviceAccountName: nginx-ingress-serviceaccount
containers:
- name: nginx-ingress-controller
image: lijiawang/nginx-ingress-controller:0.25.1 # 更换镜像源
......
......

执行

1
2
3
4
5
# kubectl apply -f mandatory.yaml
# kubectl get pod -n ingress-nginx
# kubectl get pod -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-6956498fcf-jkzbs 1/1 Running 0 55s 10.122.17.204 k8s-node1 <none> <none>

可以看到nginx-ingress-controller-6956498fcf-jkzbs POD 调度到了打好标记的k8s-node1

登录nginx-ingress-controller节点验证

因为k8s-node1nginx-ingress-controller节点,所有登录k8s-node1节点即可

1
2
3
4
5
6
7
8
9
10
# ssh k8s-node1
[[email protected] ~]# netstat -lntp|grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 5901/nginx: master
tcp6 0 0 :::80 :::* LISTEN 5901/nginx: master
[[email protected] ~]# netstat -lntp|grep 443
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 5901/nginx: master
tcp6 0 0 :::443 :::* LISTEN 5901/nginx: master
[[email protected] ~]# exit
logout
Connection to k8s-node1 closed.

部署应用

部署测试应用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# cat deploy-demon.yaml 
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: default
spec:
selector:
app: myapp
release: canary
ports:
- name: http
port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-deploy
spec:
replicas: 5
selector:
matchLabels:
app: myapp
release: canary
template:
metadata:
labels:
app: myapp
release: canary
spec:
containers:
- name: myapp
image: lijiawang/myapp:v2
ports:
- name: httpd
containerPort: 80
# kubectl apply -f deploy-demon.yaml
service/myapp created
deployment.apps/myapp-deploy created
# kubectl get pod
NAME READY STATUS RESTARTS AGE
curl-6bf6db5c4f-dqw9x 1/1 Running 1 16d
myapp-deploy-c69757d67-9bv4w 1/1 Running 0 38s
myapp-deploy-c69757d67-j2kxc 1/1 Running 0 38s
myapp-deploy-c69757d67-jgm5v 1/1 Running 0 38s
myapp-deploy-c69757d67-n5rxw 1/1 Running 0 38s
myapp-deploy-c69757d67-t4nxr 1/1 Running 0 38s
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17d
myapp ClusterIP 10.104.246.234 <none> 80/TCP 2m58s

ingress方式暴露应用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# cat ingress-myapp.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.lijw19.com # 定义域名
http:
paths:
- path:
backend:
serviceName: myapp # 跟你要爆了的服务的svc名字相同
servicePort: 80 # 要暴露的端口
# kubectl apply -f ingress-myapp.yaml
ingress.extensions/ingress-myapp created
# kubectl get ingresses.
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.lijw19.com 80 10s

本地host解析
windowshosts文件在c:\windows\system32\drivers\etc
修改host文件,增加以下内容

1
10.122.17.204 myapp.lijw19.com 

访问应用

1
2
3
4
5
6
7
8
9
10
11
# for i in `seq 10`; do curl http://myapp.lijw19.com; done
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

使用浏览器访问http://myapp.lijw19.com


这里我们是使用的http访问的,那如果要使用

使用https访问

创建证书

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
......................+++
.....................................+++
e is 65537 (0x10001)
# ls -l tls.key
-rw-r--r-- 1 root root 1675 Sep 16 08:43 tls.key
# openssl req -new -x509 -key tls.key -out tls.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:lijw19
Locality Name (eg, city) [Default City]:lijw19
Organization Name (eg, company) [Default Company Ltd]:test
Organizational Unit Name (eg, section) []:test
Common Name (eg, your name or your server's hostname) []:test.lijw19.com
Email Address []:
[[email protected] ~]# ls -l tls.*
-rw-r--r-- 1 root root 1318 Sep 16 08:46 tls.crt
-rw-r--r-- 1 root root 1675 Sep 16 08:43 tls.key

证书转成secret

将创建好的证书转成secret

1
2
3
4
5
6
7
8
# kubectl create secret tls lijw-ingress-secret --cert=tls.crt --key=tls.key
secret/lijw-ingress-secret created
# kubectl get secrets
NAME TYPE DATA AGE
default-token-vmfwt kubernetes.io/service-account-token 3 17d
lijw-ingress-secret kubernetes.io/tls 2 10s


创建https ingress

修改下ingress-myapp-https.yaml加入刚刚添加的secret,修改后的文件如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# cat ingress-myapp-https.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp-https
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls: # 添加了tls这一段
- hosts:
- test.lijw19.com
secretName: lijw-ingress-secret # 到这结束
rules:
- host: test.lijw19.com
http:
paths:
- path:
backend:
serviceName: myapp
servicePort: 80




# kubectl apply -f ingress-myapp-https.yaml
ingress.extensions/ingress-myapp-https created
# kubectl get ingresses ingress-myapp-https
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp-https test.lijw19.com 80, 443 13s

本地host解析

windowshosts文件在c:\windows\system32\drivers\etc
修改host文件,增加以下内容

1
10.122.17.204 myapp.lijw19.com  test.lijw19.com

在浏览器访问https://test.lijw19.com