基于 Kubernetes v1.16.2 之 metrics-server 部署

简单介绍

如果使用kubernetes的自动扩容功能的话，那首先得有一个插件，然后该插件将收集到的信息（cpu、memory..）与自动扩容的设置的值进行比对，自动调整pod数量。关于该插件，在kubernetes的早些版本中采用的是heapster，1.13版本正式发布后，丢弃了heapster，官方推荐采用metrics-sever。

测试环境

这是之前部署的环境现在我升级到了Kubernetes v1.16.2，升级方案可以参考

展开全文 >>

openstack instance vm_power_state 状态置为 NOSTATE

openstack 上的instance运行正常，但是power_state状态为NOSTATE，导致instance热迁移失败

现象

1、再一次热迁移时候发现迁移时候报以下错误

1
2
# openstack server migrate --live node-53 86487ef4-cc12-4be6-995e-46f5ac093901
Cannot 'os-migrateLive' instance 86487ef4-cc12-4be6-995e-46f5ac093901 while it is in power_state 0 (HTTP 409) (Request-ID: req-6c14e0ee-c3df-42de-873d-9ecc8ad215cc)

2、查看实例，发现虚拟机运行正常，但是 power_state 为 NOSTATE

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# openstack server show 86487ef4-cc12-4be6-995e-46f5ac093901
+-------------------------------------+----------------------------------------------------------------+
| Field                               | Value                                                          |
+-------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                         |
| OS-EXT-AZ:availability_zone         | m_cpu+san                                                      |
| OS-EXT-SRV-ATTR:host                | node-27                                                        |
| OS-EXT-SRV-ATTR:hypervisor_hostname | node-27                                                        |
| OS-EXT-SRV-ATTR:instance_name       | instance-000071aa                                              |
| OS-EXT-STS:power_state              | NOSTATE                                                        |
| OS-EXT-STS:task_state               | None                                                           |
| OS-EXT-STS:vm_state                 | active                                                         |
| OS-SRV-USG:launched_at              | 2018-09-21T05:40:36.000000                                     |
| OS-SRV-USG:terminated_at            | None                                                           |
| accessIPv4                          |                                                                |
| accessIPv6                          |                                                                |
| addresses                           | vlan_10.122.44.0/23=10.122.45.53                               |
| config_drive                        | True                                                           |
| created                             | 2018-09-21T05:40:31Z                                           |
| flavor                              | 16-64-100 (1cbe4ea1-8a67-4027-afd4-8f31a8b94851)               |
| hostId                              | 2730fa3d62e347ddb67e155f6eed973787a868c82316f7e6ba641b10       |
| id                                  | 86487ef4-cc12-4be6-995e-46f5ac093901                           |
| image                               | lenovo-centos-7-release (b9f8f864-4217-4ac6-a116-62ecfa0fc074) |
| key_name                            | None                                                           |
| name                                | SLP3YM7HRCX                                                    |
| progress                            | 100                                                            |
| project_id                          | e992715df18a417997c068e5f9834b0f                               |
| properties                          |                                                                |
| security_groups                     | name='default'                                                 |
| status                              | ACTIVE                                                         |
| updated                             | 2019-10-09T07:58:58Z                                           |
| user_id                             | af518aeb935c4d258f8cb7d302c83797                               |
| volumes_attached                    | id='a821856b-409f-4e6e-becd-eb4b2344c7d8'                      |
+-------------------------------------+----------------------------------------------------------------+

查找原因

列出该虚拟机的历史操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# nova instance-action-list 86487ef4-cc12-4be6-995e-46f5ac093901
+----------------+------------------------------------------+---------+----------------------------+
| Action         | Request_ID                               | Message | Start_Time                 |
+----------------+------------------------------------------+---------+----------------------------+
| create         | req-4d01a466-ce7a-48a9-b575-e129b61bcc30 | -       | 2018-09-21T05:40:30.000000 |
| live-migration | req-2d93bc73-e43f-4ca6-88fc-a6ad1a4021a6 | -       | 2018-09-21T05:44:47.000000 |
| live-migration | req-7fcbfb15-7e3c-471d-8ea5-020d438d23c3 | -       | 2019-03-24T13:00:30.000000 |
| live-migration | req-2b009c4f-caef-4d90-81fd-d480c1b9efad | Error   | 2019-10-09T07:00:13.000000 |
| live-migration | req-b364724b-6b28-4d5f-8f14-05d1c4abeceb | Error   | 2019-10-09T07:07:37.000000 |
| live-migration | req-8669d3b1-e57e-4b35-8d40-a110b64f47c8 | Error   | 2019-10-09T07:29:13.000000 |
| live-migration | req-ecec98ca-3777-4ca3-afcd-83e36381e038 | Error   | 2019-10-09T07:55:12.000000 |
| live-migration | req-6378f839-89c0-43c0-8a65-40e5b775283c | Error   | 2019-10-09T07:58:14.000000 |
| live-migration | req-e966caa7-06e5-4a29-a371-517d98c37121 | Error   | 2019-10-09T07:58:58.000000 |
| live-migration | req-6c14e0ee-c3df-42de-873d-9ecc8ad215cc | Error   | 2019-10-09T08:31:47.000000 |
+----------------+------------------------------------------+---------+----------------------------+

列出该实例的迁移记录

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
# nova  migration-list|grep 86487ef4-cc12-4be6-995e-46f5ac093901
| 84892 | -           | -         | node-27        | node-29            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-10-09T07:07:37.000000 | 2019-10-09T07:07:37.000000 | live-migration |
| 54307 | -           | -         | node-27        | node-53            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-03-24T13:00:30.000000 | 2019-03-24T17:10:10.000000 | live-migration |
| 84904 | -           | -         | node-27        | node-53            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-10-09T07:29:13.000000 | 2019-10-09T07:29:13.000000 | live-migration |
| 84913 | -           | -         | node-27        | node-53            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-10-09T07:55:12.000000 | 2019-10-09T07:55:12.000000 | live-migration |
| 84919 | -           | -         | node-27        | node-53            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-10-09T07:58:14.000000 | 2019-10-09T07:58:14.000000 | live-migration |
| 84922 | -           | -         | node-27        | node-53            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-10-09T07:58:58.000000 | 2019-10-09T07:58:58.000000 | live-migration |
| 84931 | -           | -         | node-27        | node-53            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-10-09T08:31:47.000000 | 2019-10-09T08:31:47.000000 | live-migration |
| 84889 | -           | -         | node-27        | node-89            | -            | error         | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2019-10-09T07:00:13.000000 | 2019-10-09T07:00:13.000000 | live-migration |
| 42241 | -           | -         | node-33        | node-27            | -            | completed     | 86487ef4-cc12-4be6-995e-46f5ac093901 | 189        | 189        | 2018-09-21T05:44:48.000000 | 2018-09-21T05:45:13.000000 | live-migration |
可以看到该实例从node-27节点迁移到node-85、node-53、node-29节点均为迁移成功
查看instance所在的节点
# ssh node-27
[email protected]:~# virsh list --all|grep instance-000071aa 
[email protected]:~# exit
logout
Connection to node-27 closed.
# ssh node-53
[email protected]:~#  virsh list --all|grep instance-000071aa
 91    instance-000071aa              running
[email protected]:~# exit
logout
Connection to node-53 closed.
# openstack server show 86487ef4-cc12-4be6-995e-46f5ac093901
+-------------------------------------+----------------------------------------------------------------+
| Field                               | Value                                                          |
+-------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                         |
| OS-EXT-AZ:availability_zone         | m_cpu+san                                                      |
| OS-EXT-SRV-ATTR:host                | node-27                                                        |
| OS-EXT-SRV-ATTR:hypervisor_hostname | node-27                                                        |
| OS-EXT-SRV-ATTR:instance_name       | instance-000071aa                                              |
| OS-EXT-STS:power_state              | NOSTATE                                                        |
| OS-EXT-STS:task_state               | None                                                           |
| OS-EXT-STS:vm_state                 | active                                                         |
| OS-SRV-USG:launched_at              | 2018-09-21T05:40:36.000000                                     |
| OS-SRV-USG:terminated_at            | None                                                           |
| accessIPv4                          |                                                                |
| accessIPv6                          |                                                                |
| addresses                           | vlan_10.122.44.0/23=10.122.45.53                               |
| config_drive                        | True                                                           |
| created                             | 2018-09-21T05:40:31Z                                           |
| flavor                              | 16-64-100 (1cbe4ea1-8a67-4027-afd4-8f31a8b94851)               |
| hostId                              | 2730fa3d62e347ddb67e155f6eed973787a868c82316f7e6ba641b10       |
| id                                  | 86487ef4-cc12-4be6-995e-46f5ac093901                           |
| image                               | lenovo-centos-7-release (b9f8f864-4217-4ac6-a116-62ecfa0fc074) |
| key_name                            | None                                                           |
| name                                | SLP3YM7HRCX                                                    |
| progress                            | 100                                                            |
| project_id                          | e992715df18a417997c068e5f9834b0f                               |
| properties                          |                                                                |
| security_groups                     | name='default'                                                 |
| status                              | ACTIVE                                                         |
| updated                             | 2019-10-09T08:31:47Z                                           |
| user_id                             | af518aeb935c4d258f8cb7d302c83797                               |
| volumes_attached                    | id='a821856b-409f-4e6e-becd-eb4b2344c7d8'                      |
+-------------------------------------+----------------------------------------------------------------+
[email protected]:~# mysql -unova -pnova
MariaDB [(none)]> select * from nova.instances where uuid='86487ef4-cc12-4be6-995e-46f5ac093901'\G;
*************************** 1. row ***************************
              created_at: 2018-09-21 05:40:31
              updated_at: 2019-10-09 08:31:47
              deleted_at: NULL
                      id: 29098
             internal_id: NULL
                 user_id: af518aeb935c4d258f8cb7d302c83797
              project_id: e992715df18a417997c068e5f9834b0f
               image_ref: b9f8f864-4217-4ac6-a116-62ecfa0fc074
               kernel_id: 
              ramdisk_id: 
            launch_index: 0
                key_name: NULL
                key_data: NULL
             power_state: 0
                vm_state: active
               memory_mb: 65536
                   vcpus: 16
                hostname: slp3ym7hrcx
                    host: node-27
               user_data: NULL
          reservation_id: r-o8dr8ibn
             launched_at: 2018-09-21 05:40:36
           terminated_at: NULL
            display_name: SLP3YM7HRCX
     display_description: SLP3YM7HRCX
       availability_zone: no_san
                  locked: 0
                 os_type: NULL
             launched_on: node-33
        instance_type_id: 189
                 vm_mode: NULL
                    uuid: 86487ef4-cc12-4be6-995e-46f5ac093901
            architecture: NULL
        root_device_name: /dev/vda
            access_ip_v4: NULL
            access_ip_v6: NULL
            config_drive: True
              task_state: NULL
default_ephemeral_device: NULL
     default_swap_device: NULL
                progress: 100
        auto_disk_config: 0
      shutdown_terminate: 0
       disable_terminate: 0
                 root_gb: 100
            ephemeral_gb: 0
               cell_name: NULL
                    node: node-27
                 deleted: 0
               locked_by: NULL
                 cleaned: 1
      ephemeral_key_uuid: NULL
1 row in set (0.00 sec)
ERROR: No query specified

发现实际instance启动在node-53节点上，但是数据库记录在node-27节点上

manager.py 捕获到 InstanceNotFound 设置 vm_power_state=NOSTATE

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
/usr/lib/python2.7/dist-packages/nova/compute/manager.py
    def _query_driver_power_state_and_sync(self, context, db_instance):
        if db_instance.task_state is not None:
            LOG.info(_LI("During sync_power_state the instance has a "
                         "pending task (%(task)s). Skip."),
                     {'task': db_instance.task_state}, instance=db_instance)
            return
        # No pending tasks. Now try to figure out the real vm_power_state.
        try:
            vm_instance = self.driver.get_info(db_instance)
            vm_power_state = vm_instance.state
        except exception.InstanceNotFound:         # 可以看到如果InstanceNotFound
            vm_power_state = power_state.NOSTATE   # power_state设置为NOSTATE
        # Note(maoy): the above get_info call might take a long time,
        # for example, because of a broken libvirt driver.
        try:
            self._sync_instance_power_state(context,
                                            db_instance,
                                            vm_power_state,
                                            use_slave=True)
        except exception.InstanceNotFound:
            # NOTE(hanlind): If the instance gets deleted during sync,
            # silently ignore.
            pass

得到原因

由于之前 migrate ERROR 导致 instance 实际启动的节点和数据库记录的节点不一致，comoute 通过 manager.py 捕获到 InstanceNotFound， 所以把 vm_power_state 状态置为 NOSTATE

解决方案

更新数据库，将数据库中的instance对应的host、node更新成实际instance启动的节点信息。硬重启虚拟机

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
[email protected]:~# mysql -unova -pnova
MariaDB [(none)]> update nova.instances set host='node-53' where uuid='86487ef4-cc12-4be6-995e-46f5ac093901'\G;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0
ERROR: No query specified
MariaDB [(none)]> update nova.instances set node='node-53' where uuid='86487ef4-cc12-4be6-995e-46f5ac093901'\G;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0
ERROR: No query specified
MariaDB [(none)]> select * from nova.instances where uuid='86487ef4-cc12-4be6-995e-46f5ac093901'\G;
*************************** 1. row ***************************
              created_at: 2018-09-21 05:40:31
              updated_at: 2019-10-09 08:31:47
              deleted_at: NULL
                      id: 29098
             internal_id: NULL
                 user_id: af518aeb935c4d258f8cb7d302c83797
              project_id: e992715df18a417997c068e5f9834b0f
               image_ref: b9f8f864-4217-4ac6-a116-62ecfa0fc074
               kernel_id: 
              ramdisk_id: 
            launch_index: 0
                key_name: NULL
                key_data: NULL
             power_state: 0
                vm_state: active
               memory_mb: 65536
                   vcpus: 16
                hostname: slp3ym7hrcx
                    host: node-53
               user_data: NULL
          reservation_id: r-o8dr8ibn
             launched_at: 2018-09-21 05:40:36
           terminated_at: NULL
            display_name: SLP3YM7HRCX
     display_description: SLP3YM7HRCX
       availability_zone: no_san
                  locked: 0
                 os_type: NULL
             launched_on: node-33
        instance_type_id: 189
                 vm_mode: NULL
                    uuid: 86487ef4-cc12-4be6-995e-46f5ac093901
            architecture: NULL
        root_device_name: /dev/vda
            access_ip_v4: NULL
            access_ip_v6: NULL
            config_drive: True
              task_state: NULL
default_ephemeral_device: NULL
     default_swap_device: NULL
                progress: 100
        auto_disk_config: 0
      shutdown_terminate: 0
       disable_terminate: 0
                 root_gb: 100
            ephemeral_gb: 0
               cell_name: NULL
                    node: node-53
                 deleted: 0
               locked_by: NULL
                 cleaned: 1
      ephemeral_key_uuid: NULL
1 row in set (0.00 sec)
ERROR: No query specified
MariaDB [(none)]> exit
Bye
# openstack server show 86487ef4-cc12-4be6-995e-46f5ac093901
+-------------------------------------+----------------------------------------------------------------+
| Field                               | Value                                                          |
+-------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                         |
| OS-EXT-AZ:availability_zone         | no_san                                                         |
| OS-EXT-SRV-ATTR:host                | node-53                                                        |
| OS-EXT-SRV-ATTR:hypervisor_hostname | node-53                                                        |
| OS-EXT-SRV-ATTR:instance_name       | instance-000071aa                                              |
| OS-EXT-STS:power_state              | NOSTATE                                                        |
| OS-EXT-STS:task_state               | None                                                           |
| OS-EXT-STS:vm_state                 | active                                                         |
| OS-SRV-USG:launched_at              | 2018-09-21T05:40:36.000000                                     |
| OS-SRV-USG:terminated_at            | None                                                           |
| accessIPv4                          |                                                                |
| accessIPv6                          |                                                                |
| addresses                           | vlan_10.122.44.0/23=10.122.45.53                               |
| config_drive                        | True                                                           |
| created                             | 2018-09-21T05:40:31Z                                           |
| flavor                              | 16-64-100 (1cbe4ea1-8a67-4027-afd4-8f31a8b94851)               |
| hostId                              | f5274ebe433297bc376a4ce09151591735a04d2b6762280866de2729       |
| id                                  | 86487ef4-cc12-4be6-995e-46f5ac093901                           |
| image                               | lenovo-centos-7-release (b9f8f864-4217-4ac6-a116-62ecfa0fc074) |
| key_name                            | None                                                           |
| name                                | SLP3YM7HRCX                                                    |
| progress                            | 100                                                            |
| project_id                          | e992715df18a417997c068e5f9834b0f                               |
| properties                          |                                                                |
| security_groups                     | name='default'                                                 |
| status                              | ACTIVE                                                         |
| updated                             | 2019-10-09T08:31:47Z                                           |
| user_id                             | af518aeb935c4d258f8cb7d302c83797                               |
| volumes_attached                    | id='a821856b-409f-4e6e-becd-eb4b2344c7d8'                      |
+-------------------------------------+----------------------------------------------------------------+
# nova reboot --hard 86487ef4-cc12-4be6-995e-46f5ac093901
# openstack  server show 86487ef4-cc12-4be6-995e-46f5ac093901
+-------------------------------------+----------------------------------------------------------------+
| Field                               | Value                                                          |
+-------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                         |
| OS-EXT-AZ:availability_zone         | no_san                                                         |
| OS-EXT-SRV-ATTR:host                | node-53                                                        |
| OS-EXT-SRV-ATTR:hypervisor_hostname | node-53                                                        |
| OS-EXT-SRV-ATTR:instance_name       | instance-000071aa                                              |
| OS-EXT-STS:power_state              | Running                                                        |
| OS-EXT-STS:task_state               | None                                                           |
| OS-EXT-STS:vm_state                 | active                                                         |
| OS-SRV-USG:launched_at              | 2018-09-21T05:40:36.000000                                     |
| OS-SRV-USG:terminated_at            | None                                                           |
| accessIPv4                          |                                                                |
| accessIPv6                          |                                                                |
| addresses                           | vlan_10.122.44.0/23=10.122.45.53                               |
| config_drive                        | True                                                           |
| created                             | 2018-09-21T05:40:31Z                                           |
| flavor                              | 16-64-100 (1cbe4ea1-8a67-4027-afd4-8f31a8b94851)               |
| hostId                              | 4bfe001f6fa4d443f283121b56139ef8bfbff47be8106387cc19edc6       |
| id                                  | 86487ef4-cc12-4be6-995e-46f5ac093901                           |
| image                               | lenovo-centos-7-release (b9f8f864-4217-4ac6-a116-62ecfa0fc074) |
| key_name                            | None                                                           |
| name                                | SLP3YM7HRCX                                                    |
| progress                            | 0                                                              |
| project_id                          | e992715df18a417997c068e5f9834b0f                               |
| properties                          |                                                                |
| security_groups                     | name='default'                                                 |
| status                              | ACTIVE                                                         |
| updated                             | 2019-10-09T10:04:19Z                                           |
| user_id                             | af518aeb935c4d258f8cb7d302c83797                               |
| volumes_attached                    | id='a821856b-409f-4e6e-becd-eb4b2344c7d8'                      |
|                                     | id='ff7d2dad-1954-4906-ae18-95c76a09b442'                      |
+-------------------------------------+----------------------------------------------------------------+
# openstack server migrate --live node-89 86487ef4-cc12-4be6-995e-46f5ac093901
# openstack  server show 86487ef4-cc12-4be6-995e-46f5ac093901
+-------------------------------------+----------------------------------------------------------------+
| Field                               | Value                                                          |
+-------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                         |
| OS-EXT-AZ:availability_zone         | Contingency                                                    |
| OS-EXT-SRV-ATTR:host                | node-89                                                        |
| OS-EXT-SRV-ATTR:hypervisor_hostname | node-89                                                        |
| OS-EXT-SRV-ATTR:instance_name       | instance-000071aa                                              |
| OS-EXT-STS:power_state              | Running                                                        |
| OS-EXT-STS:task_state               | None                                                           |
| OS-EXT-STS:vm_state                 | active                                                         |
| OS-SRV-USG:launched_at              | 2018-09-21T05:40:36.000000                                     |
| OS-SRV-USG:terminated_at            | None                                                           |
| accessIPv4                          |                                                                |
| accessIPv6                          |                                                                |
| addresses                           | vlan_10.122.44.0/23=10.122.45.53                               |
| config_drive                        | True                                                           |
| created                             | 2018-09-21T05:40:31Z                                           |
| flavor                              | 16-64-100 (1cbe4ea1-8a67-4027-afd4-8f31a8b94851)               |
| hostId                              | 4bfe001f6fa4d443f283121b56139ef8bfbff47be8106387cc19edc6       |
| id                                  | 86487ef4-cc12-4be6-995e-46f5ac093901                           |
| image                               | lenovo-centos-7-release (b9f8f864-4217-4ac6-a116-62ecfa0fc074) |
| key_name                            | None                                                           |
| name                                | SLP3YM7HRCX                                                    |
| progress                            | 0                                                              |
| project_id                          | e992715df18a417997c068e5f9834b0f                               |
| properties                          |                                                                |
| security_groups                     | name='default'                                                 |
| status                              | ACTIVE                                                         |
| updated                             | 2019-10-09T10:04:19Z                                           |
| user_id                             | af518aeb935c4d258f8cb7d302c83797                               |
| volumes_attached                    | id='a821856b-409f-4e6e-becd-eb4b2344c7d8'                      |
|                                     | id='ff7d2dad-1954-4906-ae18-95c76a09b442'                      |
+-------------------------------------+----------------------------------------------------------------+

展开全文 >>

使用kubeadm升级kubernetes集群

kubeadm升级k8s集群(v1.15.3 to v1.16.0版)

Kubernetes在2019年9月18日发布了年度的第三个版本1.16,本篇文章介绍使用kubeadm升级现在有的集群到v1.16.0。

Kubernetes 1.16版本的发布徽章

Kubernetes 1.16版本的发布徽章

查看k8s集群版本

查看当前的k8s版本

1
2
3
4
5
6
[[email protected] ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   20d   v1.15.3
k8s-node1    Ready    <none>   20d   v1.15.3
k8s-node2    Ready    <none>   20d   v1.15.3
k8s-node3    Ready    <none>   20d   v1.15.3

1
2
[[email protected] ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:11:18Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
1
2
3
4
5
6
[[email protected] ~]# kubelet --version
Kubernetes v1.15.3
[[email protected] ~]# systemctl status kubelet.service |grep Active
   Active: active (running) since Fri 2019-08-30 07:26:33 EDT; 3 weeks 0 days ago
1
2
3
[[email protected] yum.repos.d]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

查看版本支持

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[[email protected] yum.repos.d]# yum list kubelet kubeadm kubectl
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Installed Packages
kubeadm.x86_64                                                        1.15.3-0                                                        @kubernetes
kubectl.x86_64                                                        1.15.3-0                                                        @kubernetes
kubelet.x86_64                                                        1.15.3-0                                                        @kubernetes
Available Packages
kubeadm.x86_64                                                        1.16.0-0                                                        kubernetes 
kubectl.x86_64                                                        1.16.0-0                                                        kubernetes 
kubelet.x86_64                                                        1.16.0-0                                                        kubernetes

升级kubelet kubeadm kubectl版本到1.16.0(每个节点都执行)

1
2
3
4
5
6
7
8
9
10
11
[[email protected] ~]# yum install -y kubelet kubeadm kubectl
# systemctl daemon-reload && systemctl restart kubelet && systemctl status kubelet
[[email protected] ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
[[email protected] ~]# kubelet --version
Kubernetes v1.16.0
[[email protected] ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:34:01Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

升级k8s集群

1
2
3
4
5
6
[[email protected] ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   21d   v1.15.3
k8s-node1    Ready    <none>   20d   v1.15.3
k8s-node2    Ready    <none>   20d   v1.15.3
k8s-node3    Ready    <none>   20d   v1.15.3

升级集群

1
[[email protected] ~]# kubeadm config print init-defaults > kubeadm1.16.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
[[email protected] ~]# cat kubeadm1.16.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.122.17.200
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
networking:
  dnsDomain: cluster.local
  podSubnet: 192.168.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
1
2
3
4
5
6
7
8
9
10
11
12
[[email protected] ~]# kubeadm upgrade apply --config kubeadm1.16.yaml
.......
.......
.......
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.16.0". Enjoy!
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
1
2
3
[[email protected] ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[[email protected] ~]# yum clean all
[[email protected] ~]# yum install -y kubelet kubeadm kubectl
systemctl daemon-reload && systemctl restart kubelet && systemctl status kubelet
[[email protected]ode1 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[[email protected] ~]# kubelet --version
Kubernetes v1.16.0
[[email protected] ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:34:01Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
[[email protected] ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
[[email protected] ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   21d   v1.16.0
k8s-node1    Ready    <none>   21d   v1.16.0
k8s-node2    Ready    <none>   21d   v1.16.0
k8s-node3    Ready    <none>   21d   v1.16.0
[[email protected] ~]# kubectl get pod -A
NAMESPACE       NAME                                        READY   STATUS    RESTARTS   AGE
default         curl-6bf6db5c4f-dqw9x                       1/1     Running   2          20d
ingress-nginx   nginx-ingress-controller-6956498fcf-jkzbs   1/1     Running   1          16d
kube-system     calico-kube-controllers-65b8787765-hx9cr    1/1     Running   1          21d
kube-system     calico-node-2rlsz                           1/1     Running   1          21d
kube-system     calico-node-5vz46                           1/1     Running   1          21d
kube-system     calico-node-6szsd                           1/1     Running   1          21d
kube-system     calico-node-s5nmr                           1/1     Running   1          21d
kube-system     calicoctl                                   1/1     Running   1          20d
kube-system     coredns-58cc8c89f4-cnl45                    1/1     Running   1          26m
kube-system     coredns-58cc8c89f4-nj8hr                    1/1     Running   1          9m4s
kube-system     etcd-k8s-master                             1/1     Running   0          6m35s
kube-system     kube-apiserver-k8s-master                   1/1     Running   0          6m30s
kube-system     kube-controller-manager-k8s-master          1/1     Running   0          6m35s
kube-system     kube-proxy-52mbf                            1/1     Running   1          26m
kube-system     kube-proxy-gf75r                            1/1     Running   1          25m
kube-system     kube-proxy-hbvjf                            1/1     Running   1          26m
kube-system     kube-proxy-zppdf                            1/1     Running   1          26m
kube-system     kube-scheduler-k8s-master                   1/1     Running   0          6m31s
kube-system     kubernetes-dashboard-5dc4c54b55-c7sv2       1/1     Running   1          20d
[[email protected] ~]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
curl-6bf6db5c4f-dqw9x   1/1     Running   2          20d

验证cordns

1
2
3
4
5
6
7
8
9
10
11
12
13
[[email protected] ~]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
curl-6bf6db5c4f-dqw9x   1/1     Running   2          20d
[[email protected] ~]# kubectl exec -it curl-6bf6db5c4f-dqw9x -- /bin/sh
/bin/sh: shopt: not found
[ [email protected]:/ ]$ nslookup kubernetes.default
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name:      kubernetes.default
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local
[ [email protected]:/ ]$ exit
[[email protected] ~]#

未解决的问题

kubectl get cs显示为unknown
错误提示信息示例如下:

1
2
3
4
5
# kubectl get cs
NAME                 AGE
scheduler            <unknown>
controller-manager   <unknown>
etcd-0               <unknown>

这个问题似乎对集群没有太大的影响，暂未解决，后续愿意确认之后会继续更新，如果有大神知道如何解决，请指点。

参考链接

1、 https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-upgrade/
2、 http://www.sohu.com/a/342118551_198222
3、 https://www.linuxidc.com/Linux/2019-09/160728.htm

展开全文 >>

在k8s上部署ingress-nginx并使用

[TOC]

部署nginx-ingress

环境准备

准备一套k8s环境

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-0               Healthy   {"health":"true"}   
# kubectl cluster-info
Kubernetes master is running at https://10.122.17.200:6443
KubeDNS is running at https://10.122.17.200:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
# kubectl get nodes
NAME         STATUS   ROLES    AGE    VERSION
k8s-master   Ready    master   4d1h   v1.15.3
k8s-node1    Ready    <none>   4d1h   v1.15.3
k8s-node2    Ready    <none>   4d     v1.15.3
k8s-node3    Ready    <none>   4d     v1.15.3

下载安装nginx-ingress的yaml文件

1
# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

设置nginx-ingress-controller调度节点

1
2
3
4
# kubectl label nodes k8s-node1 hosts=nginx-ingress-controller
# kubectl get node -l hosts=nginx-ingress-controller --show-labels 
NAME        STATUS   ROLES    AGE    VERSION   LABELS
k8s-node1   Ready    <none>   4d6h   v1.15.3   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,hosts=nginx-ingress-controller,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node1,kubernetes.io/os=linux

修改mandatory.yaml文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# vim mandatory.yaml
......
apiVersion: apps/v1
kind: Deployment
......
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    ......
    spec:
      hostNetwork: true    # 使用hostNetwork 模式
      nodeSelector:        # 使用nodeSelector
        hosts: nginx-ingress-controller   # 选择标签为hosts=nginx-ingress-controller节点
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: lijiawang/nginx-ingress-controller:0.25.1   # 更换镜像源
		  ......
......

执行

1
2
3
4
5
# kubectl apply -f mandatory.yaml
# kubectl get pod -n ingress-nginx
# kubectl get pod -n ingress-nginx -o wide
NAME                                        READY   STATUS    RESTARTS   AGE   IP              NODE        NOMINATED NODE   READINESS GATES
nginx-ingress-controller-6956498fcf-jkzbs   1/1     Running   0          55s   10.122.17.204   k8s-node1   <none>           <none>

可以看到nginx-ingress-controller-6956498fcf-jkzbs POD 调度到了打好标记的k8s-node1。

登录nginx-ingress-controller节点验证

因为k8s-node1为nginx-ingress-controller节点，所有登录k8s-node1节点即可

1
2
3
4
5
6
7
8
9
10
# ssh k8s-node1
[[email protected] ~]# netstat -lntp|grep 80
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      5901/nginx: master  
tcp6       0      0 :::80                   :::*                    LISTEN      5901/nginx: master  
[[email protected] ~]# netstat -lntp|grep 443
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      5901/nginx: master  
tcp6       0      0 :::443                  :::*                    LISTEN      5901/nginx: master  
[[email protected] ~]# exit
logout
Connection to k8s-node1 closed.

部署应用

部署测试应用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# cat deploy-demon.yaml 
apiVersion: v1
kind: Service
metadata:
  name: myapp
  namespace: default
spec:
  selector:
    app: myapp
    release: canary
  ports:
  - name: http
    port: 80
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata: 
  name: myapp-deploy
spec:
  replicas: 5
  selector: 
    matchLabels:
      app: myapp
      release: canary
  template:
    metadata:
      labels:
        app: myapp
        release: canary
    spec:
      containers:
      - name: myapp
        image: lijiawang/myapp:v2
        ports:
        - name: httpd
          containerPort: 80
# kubectl  apply -f deploy-demon.yaml 
service/myapp created
deployment.apps/myapp-deploy created
# kubectl get pod 
NAME                           READY   STATUS    RESTARTS   AGE
curl-6bf6db5c4f-dqw9x          1/1     Running   1          16d
myapp-deploy-c69757d67-9bv4w   1/1     Running   0          38s
myapp-deploy-c69757d67-j2kxc   1/1     Running   0          38s
myapp-deploy-c69757d67-jgm5v   1/1     Running   0          38s
myapp-deploy-c69757d67-n5rxw   1/1     Running   0          38s
myapp-deploy-c69757d67-t4nxr   1/1     Running   0          38s
# kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP   17d
myapp        ClusterIP   10.104.246.234   <none>        80/TCP    2m58s

以ingress方式暴露应用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# cat ingress-myapp.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-myapp
  namespace: default
  annotations: 
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: myapp.lijw19.com   # 定义域名
    http:
      paths:
      - path: 
        backend:
          serviceName: myapp   # 跟你要爆了的服务的svc名字相同
          servicePort: 80     # 要暴露的端口
# kubectl apply -f ingress-myapp.yaml 
ingress.extensions/ingress-myapp created
# kubectl get ingresses.
NAME            HOSTS              ADDRESS   PORTS   AGE
ingress-myapp   myapp.lijw19.com             80      10s

本地host解析
windows的hosts文件在c:\windows\system32\drivers\etc下
修改host文件，增加以下内容

1
10.122.17.204 myapp.lijw19.com

访问应用

1
2
3
4
5
6
7
8
9
10
11
# for i in `seq 10`; do curl http://myapp.lijw19.com; done
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

使用浏览器访问http://myapp.lijw19.com


这里我们是使用的http访问的，那如果要使用

使用https访问

创建证书

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
......................+++
.....................................+++
e is 65537 (0x10001)
# ls -l tls.key 
-rw-r--r-- 1 root root 1675 Sep 16 08:43 tls.key
# openssl req -new -x509 -key tls.key -out tls.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN     
State or Province Name (full name) []:lijw19
Locality Name (eg, city) [Default City]:lijw19
Organization Name (eg, company) [Default Company Ltd]:test
Organizational Unit Name (eg, section) []:test
Common Name (eg, your name or your server's hostname) []:test.lijw19.com
Email Address []:
[[email protected] ~]# ls -l tls.*
-rw-r--r-- 1 root root 1318 Sep 16 08:46 tls.crt
-rw-r--r-- 1 root root 1675 Sep 16 08:43 tls.key

证书转成secret

将创建好的证书转成secret

1
2
3
4
5
6
# kubectl create secret tls lijw-ingress-secret --cert=tls.crt --key=tls.key
secret/lijw-ingress-secret created
# kubectl get secrets 
NAME                  TYPE                                  DATA   AGE
default-token-vmfwt   kubernetes.io/service-account-token   3      17d
lijw-ingress-secret   kubernetes.io/tls                     2      10s

创建https ingress

修改下ingress-myapp-https.yaml加入刚刚添加的secret，修改后的文件如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# cat ingress-myapp-https.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-myapp-https
  namespace: default
  annotations: 
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:    # 添加了tls这一段
  - hosts:
    - test.lijw19.com
    secretName: lijw-ingress-secret  # 到这结束
  rules:
  - host: test.lijw19.com
    http:
      paths:
      - path: 
        backend:
          serviceName: myapp
          servicePort: 80
# kubectl apply -f ingress-myapp-https.yaml 
ingress.extensions/ingress-myapp-https created
# kubectl get ingresses ingress-myapp-https 
NAME                  HOSTS             ADDRESS   PORTS     AGE
ingress-myapp-https   test.lijw19.com             80, 443   13s

本地host解析

windows的hosts文件在c:\windows\system32\drivers\etc下
修改host文件，增加以下内容

1
10.122.17.204 myapp.lijw19.com  test.lijw19.com

在浏览器访问https://test.lijw19.com

展开全文 >>

harbor搭建

harbor介绍

Harbor是一个用于存储和分发Docker镜像的企业级Registry服务器，通过添加一些企业必需的功能特性，例如安全、标识和管理等，扩展了开源Docker Distribution。作为一个企业级私有Registry服务器，Harbor提供了更好的性能和安全。提升用户使用Registry构建和运行环境传输镜像的效率。Harbor支持安装在多个Registry节点的镜像资源复制，镜像全部保存在私有Registry中， 确保数据和知识产权在公司内部网络中管控。另外，Harbor也提供了高级的安全特性，诸如用户管理，访问控制和活动审计等。

more >>

展开全文 >>

docker网络

docker四种单节点网络

Docker在创建容器时有四种网络模式，bridge为默认不需要用-–net去指定，其他三种模式需要在创建容器时使用–-net去指定。
四种单节点网络模式如下：
none模式，使用–-net=none指定；
host模式，使用–-net=host指定；
bridge模式，使用–-net=bridge指定，默认设置；
container模式，使用–-net=container:容器名称或ID指定。
在Docker安装时会自动在host上创建三个网络,我们可以通过docker network ls查看：

more >>

展开全文 >>

ceph-for-openstack-storage

openstack对接ceph存储

简介

  • 截至2018年5月3日，OpenStack也已经伴随着我们走过了8个年头了，并且成为了云计算领域中最火热的项目之一，逐渐成为IaaS的事实标准，私有云项目的部署首选；
  • OpenStack发展如此的迅速，以至于部署规模愈发的庞大，此时就该思量OpenStack集群的部署支持以及持续可扩展性；
  • OpenStack和Ceph的集成更让开源项目锦上添花，Ceph作为优秀的分布式存储系统，实现对OpenStack相关子项目进行集成或替代，目前在OpenStack中扮演者非常重要的角色；
more >>

展开全文 >>

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

不看官网不足以熟练<br>不懂源码不足以精通<br><br>很惭愧<br><br>只做了一点微小的工作<br>谢谢大家