Enabling Router ha:
L3 HA in kilo requires that DVR and L2 Population are disabled, so to enable it in the charms:
juju set neutron-api enable-l3ha=True
juju set neutron-api enable-dvr=False
juju set neutron-api l2-population=False
The number of L3 agents that will run standby routers can also be configured:
juju set neutron-api max-l3-agents-per-router=2 juju set neutron-api min-l3-agents-per-router=2
Creating a HA enabled router.
$ neutron router-create ha-router
Created a new router:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| distributed | False |
| external_gateway_info | |
| ha | True |
| id | 64ff0665-5600-433c-b2d8-33509ce88eb1 |
| name | test2 |
| routes | |
| status | ACTIVE |
| tenant_id | 8e8b1426508f42aeaff783180d7b2ef4 |
+-----------------------+--------------------------------------+
/!\ Currently a router cannot be switched in and out of HA mode$ neutron router-update 64ff0665-5600-433c-b2d8-33509ce88eb1 --ha=False
400-{u'NeutronError': {u'message': u'Cannot update read-only attribute ha', u'type': u'HTTPBadRequest', u'detail': u''}}
Under the hood:
Below is a worked example, following the creation of an HA enabled router showing the components created implicitly by Neutron. In this environment the following networks have already been created:$ neutron net-list
+--------------------------------------+---------+------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+---------+------------------------------------------------------+
| 32ba54bc-804e-489e-8903-b8dc0ed535f7 | private | a3ed1cc4-3451-418f-a412-80ad8cca2ec4 192.168.21.0/24 |
| c9a3bc24-6390-4220-b136-bc0edf1fe2f2 | ext_net | 76098d4d-bfa4-4f96-89e0-78c851d80dac 10.5.0.0/16 |
+--------------------------------------+---------+------------------------------------------------------+
$ neutron subnet-list
+--------------------------------------+----------------+-----------------+----------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+----------------+-----------------+----------------------------------------------------+
| 76098d4d-bfa4-4f96-89e0-78c851d80dac | ext_net_subnet | 10.5.0.0/16 | {"start": "10.5.150.0", "end": "10.5.200.254"} |
| a3ed1cc4-3451-418f-a412-80ad8cca2ec4 | private_subnet | 192.168.21.0/24 | {"start": "192.168.21.2", "end": "192.168.21.254"} |
+--------------------------------------+----------------+-----------------+----------------------------------------------------+
In this environment there are three neutron-gateways:
$ juju status neutron-gateway --format=short
- neutron-gateway/0: 10.5.29.216 (started)
- neutron-gateway/1: 10.5.29.217 (started)
- neutron-gateway/2: 10.5.29.218 (started)
With their corresponding L3 agents:
$ neutron agent-list | grep "L3 agent"
| 28f227d8-e620-4478-ba36-856fb0409393 | L3 agent | juju-lytrusty-machine-7 | :-) | True |
| 8d439f33-e4f8-4784-a617-5b3328bab9e3 | L3 agent | juju-lytrusty-machine-6 | :-) | True |
| bdc00c2a-77c0-45c3-ab8a-ceca3319832d | L3 agent | juju-lytrusty-machine-8 | :-) | True |
There is no router defined yet so the only network namespace present is the dhcp namespace for the private network:
$ juju run --service neutron-gateway --format=yaml "ip netns list"
- MachineId: "6"
Stdout: ""
UnitId: neutron-gateway/0
- MachineId: "7"
Stdout: |
qdhcp-32ba54bc-804e-489e-8903-b8dc0ed535f7
UnitId: neutron-gateway/1
- MachineId: "8"
Stdout: ""
UnitId: neutron-gateway/2
Creating a router will add a qrouter-$ROUTERID netns to two of the gateway nodes (since min-l3-agents-per-router=2 and max-l3-agents-per-router=2)
$ neutron router-create ha-router
Created a new router:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| distributed | False |
| external_gateway_info | |
| ha | True |
| id | 192ba483-c060-4ee2-86ad-fe38ea280c93 |
| name | ha-router |
| routes | |
| status | ACTIVE |
| tenant_id | 8e8b1426508f42aeaff783180d7b2ef4 |
+-----------------------+--------------------------------------+
Neutron has assigned this router to two of the three agents:
$ ROUTER_ID="192ba483-c060-4ee2-86ad-fe38ea280c93"
$ neutron l3-agent-list-hosting-router $ROUTER_ID
+--------------------------------------+-------------------------+----------------+-------+
| id | host | admin_state_up | alive |
+--------------------------------------+-------------------------+----------------+-------+
| 28f227d8-e620-4478-ba36-856fb0409393 | juju-lytrusty-machine-7 | True | :-) |
| bdc00c2a-77c0-45c3-ab8a-ceca3319832d | juju-lytrusty-machine-8 | True | :-) |
+--------------------------------------+-------------------------+----------------+-------+
A netns for the new router will have been created in neutron-gateway/1 and neutron-gateway/2:
$ juju run --service neutron-gateway --format=yaml "ip netns list"
- MachineId: "6"
Stdout: ""
UnitId: neutron-gateway/0
- MachineId: "7"
Stdout: |
qrouter-192ba483-c060-4ee2-86ad-fe38ea280c93
qdhcp-32ba54bc-804e-489e-8903-b8dc0ed535f7
UnitId: neutron-gateway/1
- MachineId: "8"
Stdout: |
qrouter-192ba483-c060-4ee2-86ad-fe38ea280c93
UnitId: neutron-gateway/2
A keepalived process is spawned in each of the qrouter netns and these process communicate over a dedicated network which is created implicitly when the HA enabled router is added.
$ neutron net-list
+--------------------------------------+----------------------------------------------------+-------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+----------------------------------------------------+-------------------------------------------------------+
| 32ba54bc-804e-489e-8903-b8dc0ed535f7 | private | a3ed1cc4-3451-418f-a412-80ad8cca2ec4 192.168.21.0/24 |
| af9cad57-b4fe-465d-b439-b72aaec16309 | HA network tenant 8e8b1426508f42aeaff783180d7b2ef4 | f0cb279b-36fe-43dc-a03b-8eb8b99e7f0b 169.254.192.0/18 |
| c9a3bc24-6390-4220-b136-bc0edf1fe2f2 | ext_net | 76098d4d-bfa4-4f96-89e0-78c851d80dac 10.5.0.0/16 |
+--------------------------------------+----------------------------------------------------+-------------------------------------------------------+
Neutron creates a dedicated interface in the qrouter netns for this traffic.
$ neutron port-list
+--------------------------------------+-------------------------------------------------+-------------------+--------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+-------------------------------------------------+-------------------+--------------------------------------------------------------------------------------+
| 72326e9b-67e8-403a-80c3-4bac9748cdb6 | | fa:16:3e:aa:2c:96 | {"subnet_id": "a3ed1cc4-3451-418fa412-80ad8cca2ec4", "ip_address": "192.168.21.2"} |
| 89c47030-f849-41ed-96e6-a36a3a696eeb | HA port tenant 8e8b1426508f42aeaff783180d7b2ef4 | fa:16:3e:d4:fc:a1 | {"subnet_id": "f0cb279b-36fe-43dca03b-8eb8b99e7f0b", "ip_address": "169.254.192.1"} |
| 9ce2b6ac-9983-4ffd-ae97-6400682021c8 | HA port tenant 8e8b1426508f42aeaff783180d7b2ef4 | fa:16:3e:a5:76:e9 | {"subnet_id": "f0cb279b-36fe-43dca03b-8eb8b99e7f0b", "ip_address": "169.254.192.2"} |
+--------------------------------------+-------------------------------------------------+-------------------+--------------------------------------------------------------------------------------+
$ juju run --unit neutron-gateway/1,neutron-gateway/2 --format=yaml "ip netns exec qrouter-$ROUTER_ID ip addr list | grep ha-"
- MachineId: "7"
Stdout: |
2: ha-89c47030-f8: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 169.254.192.1/18 brd 169.254.255.255 scope global ha-89c47030-f8
UnitId: neutron-gateway/1
- MachineId: "8"
Stdout: |
2: ha-9ce2b6ac-99: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 169.254.192.2/18 brd 169.254.255.255 scope global ha-9ce2b6ac-99
inet 169.254.0.1/24 scope global ha-9ce2b6ac-99
UnitId: neutron-gateway/2
Keepalived writes out state to /var/lib/neutron/ha_confs/$ROUTER_ID/state, this can be queried to find out who is currently the master:
$ juju run --unit neutron-gateway/1,neutron-gateway/2 "cat /var/lib/neutron/ha_confs/$ROUTER_ID/state"
- MachineId: "7"
Stdout: backup
UnitId: neutron-gateway/1
- MachineId: "8"
Stdout: master
UnitId: neutron-gateway/2
Plugging the router into the networks:
$ neutron router-gateway-set $ROUTER_ID c9a3bc24-6390-4220-b136-bc0edf1fe2f2
Set gateway for router 192ba483-c060-4ee2-86ad-fe38ea280c93
$ neutron router-interface-add $ROUTER_ID a3ed1cc4-3451-418f-a412-80ad8cca2ec4
Added interface 4ffe673c-b528-4891-b9ec-3ebdcfc146e2 to router 192ba483-c060-4ee2-86ad-fe38ea280c93.
The router now has an IP on the private subnet which will be managed by keepalived:
$ neutron router-show $ROUTER_ID
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| distributed | False |
| external_gateway_info | {"network_id": "c9a3bc24-6390-4220-b136-bc0edf1fe2f2", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "76098d4d-bfa4-4f96-89e0-78c851d80dac", "ip_address": "10.5.150.0"}]} |
| ha | True |
| id | 192ba483-c060-4ee2-86ad-fe38ea280c93 |
| name | ha-router |
| routes | |
| status | ACTIVE |
| tenant_id | 8e8b1426508f42aeaff783180d7b2ef4 |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Since neutron-gateway/2 has been designated as the leader it will have the router ip (10.5.150.0) in its netns:
$ juju run --unit neutron-gateway/1,neutron-gateway/2 --format=yaml "ip netns exec qrouter-192ba483-c060-4ee2-86ad-fe38ea280c93 ip addr list | grep 10.5.150"
- MachineId: "7"
ReturnCode: 1
Stdout: ""
UnitId: neutron-gateway/1
- MachineId: "8"
Stdout: |2
inet 10.5.150.0/16 scope global qg-288da587-97
UnitId: neutron-gateway/2
$ ping -c2 10.5.150.0
PING 10.5.150.0 (10.5.150.0) 56(84) bytes of data.
64 bytes from 10.5.150.0: icmp_seq=1 ttl=64 time=0.756 ms
64 bytes from 10.5.150.0: icmp_seq=2 ttl=64 time=0.487 ms
--- 10.5.150.0 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.487/0.621/0.756/0.136 ms
Finally, shutting down neutron-gateway/2 will tigger the router ip to flip over to neutron-gateway/1:
$ juju run --unit neutron-gateway/2 "shutdown -h now"
$ juju run --unit neutron-gateway/1 "cat /var/lib/neutron/ha_confs/$ROUTER_ID/state"
master
$ juju run --unit neutron-gateway/1 --format=yaml "ip netns exec qrouter-192ba483-c060-4ee2-86ad-fe38ea280c93 ip addr list | grep 10.5.150"
- MachineId: "7"
Stdout: |2
inet 10.5.150.0/16 scope global qg-288da587-97
UnitId: neutron-gateway/1
$ ping -c2 10.5.150.0
PING 10.5.150.0 (10.5.150.0) 56(84) bytes of data.
64 bytes from 10.5.150.0: icmp_seq=1 ttl=64 time=0.359 ms
64 bytes from 10.5.150.0: icmp_seq=2 ttl=64 time=0.497 ms
--- 10.5.150.0 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.359/0.428/0.497/0.069 ms
No comments:
Post a Comment