Enabling QoS
The charm encapsulates all the work of setting up QoS so all that is needed to enable it is:
$ juju config neutron-api enable-qos=True
Testing QoS On a Guest
In the test below traffic speed between two OpenStack guests is generated with iperf with and without a
QoS bandwidth policy applied to the guest 'qos-test-1'. Below are the two guests used in the test.
$ nova list +--------------------------------------+------------+--------+------------+-------------+------------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------------+--------+------------+-------------+------------------------------------+ | 0b8fcb80-8210-4034-918a-a253f2f810bf | qos-test-1 | ACTIVE | - | Running | private=192.168.21.8, 10.5.150.8 | | ee7d2a15-fb20-448e-81c2-cedd52bc45c3 | qos-test-2 | ACTIVE | - | Running | private=192.168.21.12, 10.5.150.12 | +--------------------------------------+------------+--------+------------+-------------+------------------------------------+
The 192.168 addresses are on the internal project network and the 10.5 addresses are floating IPs which have been assigned to the guests.
The first step is to create a QoS policy that rules can be added to:
Now add a rule to the policy:
qos-test-1's IP address on the internal network is 192.168.21.8 and the corresponding port will have the QoS rule applied to it. List the ports and find the one that corresponds with the guests IP address
$ neutron qos-policy-create bw-limiter Created a new policy: +-----------------+--------------------------------------+ | Field | Value | +-----------------+--------------------------------------+ | created_at | 2017-09-19T16:27:31Z | | description | | | id | 5440ddb6-b93c-44d6-b0a8-b3b36e552db3 | | name | bw-limiter | | project_id | 7a6b7bcc7284483f88dc44f765cad5df | | revision_number | 1 | | rules | | | shared | False | | tenant_id | 7a6b7bcc7284483f88dc44f765cad5df | | updated_at | 2017-09-19T16:27:31Z | +-----------------+--------------------------------------+
Now add a rule to the policy:
$ neutron qos-bandwidth-limit-rule-create bw-limiter --max-kbps 3000 --max-burst-kbps 300 Created a new bandwidth_limit_rule: +----------------+--------------------------------------+ | Field | Value | +----------------+--------------------------------------+ | id | 9d0bd218-d04e-4444-83e5-35f6e9ec8b44 | | max_burst_kbps | 300 | | max_kbps | 3000 | +----------------+--------------------------------------+
$ neutron port-list +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+ | 735b2928-578d-491e-9fff-3e45bd82237e | | fa:16:3e:35:01:2d | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.8"} | | bb97d685-a891-4492-93b7-ea7b6caa367e | | fa:16:3e:03:0a:4c | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.12"} | | bf8aac5f-959f-4382-9b4d-10ce8aded036 | | fa:16:3e:55:60:28 | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.2"} | | c07f336f-72ee-4ed3-849a-7181bd2a0492 | | fa:16:3e:cc:a5:b1 | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.4"} | | c6c04b68-a1e3-48f1-a2af-7cfabf989c97 | | fa:16:3e:7d:4d:d9 | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.1"} | | d75d8ee3-71c1-41f9-90ed-f379f0256f1d | | fa:16:3e:ff:2b:2d | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.12"} | | dd33eb79-aaf7-4135-aa4b-28abfbb9fb50 | | fa:16:3e:f7:6a:4d | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.8"} | | ea056cbc-be5b-4223-a7b3-e548aca8a2cb | | fa:16:3e:6d:dc:94 | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.9"} | +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
Apply the policy to the port:
$ neutron port-update 735b2928-578d-491e-9fff-3e45bd82237e --qos-policy bw-limiter Updated port: 735b2928-578d-491e-9fff-3e45bd82237e
On qos-test-2 start the iperf server:
On qos-test-1 (the guest with the qos rule applied) run iperf connecting to qos-test-2:ubuntu@qos-test-2:~$ iperf -s -p 8080 ------------------------------------------------------------ Server listening on TCP port 8080 TCP window size: 85.3 KByte (default) ------------------------------------------------------------
ubuntu@qos-test-1:~$ iperf -c 192.168.21.12 -p 8080 ------------------------------------------------------------ Client connecting to 192.168.21.12, TCP port 8080 TCP window size: 45.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.21.8 port 49704 connected with 192.168.21.12 port 8080 [ ID] Interval Transfer Bandwidth [ 3] 0.0-11.2 sec 2.12 MBytes 1.60 Mbits/sec
On that test achieved a bandwidth of 1.6 MBits/s. It is worth running the test a few times to validate the rate. Now, rerun test with the QoS policy removed. To remove the policy from the port:
$ neutron port-update 735b2928-578d-491e-9fff-3e45bd82237e --no-qos-policy Updated port: 735b2928-578d-491e-9fff-3e45bd82237e
Rerunning the test:
ubuntu@qos-test-1:~$ iperf -c 192.168.21.12 -p 8080 ------------------------------------------------------------ Client connecting to 192.168.21.12, TCP port 8080 TCP window size: 45.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.21.8 port 49705 connected with 192.168.21.12 port 8080 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 403 MBytes 338 Mbits/sec
This time the test achieved a bandwidth of 388 MBits/s.
Testing QoS on a Network
The QoS policy can also be applied to the router port to affect all guests on a given network. In the previous example the last test removed the QoS policy from qos-test-1's port but just in case double check it is gone:$ neutron port-show 735b2928-578d-491e-9fff-3e45bd82237e | grep qos_policy_id neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. | qos_policy_id | |
In example below there is an iperf server running on an external host at 10.5.0.9. Collect iperf performance figures from both guests to the external host:
ubuntu@qos-test-1:~$ iperf -c 10.5.0.9 -p 8080 ------------------------------------------------------------ Client connecting to 10.5.0.9, TCP port 8080 TCP window size: 325 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.21.8 port 34654 connected with 10.5.0.9 port 8080 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 321 MBytes 269 Mbits/sec ubuntu@qos-test-1:~$
and the second guest:
ubuntu@qos-test-2:~$ iperf -c 10.5.0.9 -p 8080 ------------------------------------------------------------ Client connecting to 10.5.0.9, TCP port 8080 TCP window size: 325 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.21.12 port 59839 connected with 10.5.0.9 port 8080 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 287 MBytes 241 Mbits/sec ubuntu@qos-test-2:~$
Now get the external ip of the project router and apply the QoS policy to it:
$ neutron router-list +--------------------------------------+-----------------+------------------------------------------------------------------------+-------------+-------+ | id | name | external_gateway_info | distributed | ha | +--------------------------------------+-----------------+------------------------------------------------------------------------+-------------+-------+ | 51a15d70-5917-4dc2-9ab4-03c0f03e4fd7 | provider-router | {"network_id": "73c17d70-0298-4a0f-9158-f40bd73ee92e", "enable_snat": | False | False | | | | true, "external_fixed_ips": [{"subnet_id": | | | | | | "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.4"}]} | | | +--------------------------------------+-----------------+------------------------------------------------------------------------+-------------+-------+ $ neutron port-list | grep 10.5.150.4 | c07f336f-72ee-4ed3-849a-7181bd2a0492 | | fa:16:3e:cc:a5:b1 | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.4"} | $ neutron port-update c07f336f-72ee-4ed3-849a-7181bd2a0492 --qos-policy bw-limiter Updated port: c07f336f-72ee-4ed3-849a-7181bd2a0492
Rerunning the benchmarks:
ubuntu@qos-test-1:~$ iperf -c 10.5.0.9 -p 8080 ------------------------------------------------------------ Client connecting to 10.5.0.9, TCP port 8080 TCP window size: 325 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.21.8 port 34655 connected with 10.5.0.9 port 8080 [ ID] Interval Transfer Bandwidth [ 3] 0.0-11.1 sec 2.38 MBytes 1.80 Mbits/sec
ubuntu@qos-test-2:~$ iperf -c 10.5.0.9 -p 8080 ------------------------------------------------------------ Client connecting to 10.5.0.9, TCP port 8080 TCP window size: 325 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.21.12 port 59840 connected with 10.5.0.9 port 8080 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.8 sec 2.25 MBytes 1.75 Mbits/sec
Under the Hood
Connect to the neutron-gateway and find the IP of the ovs port corresponding to the external router port.
$ juju ssh neutron-gateway/0 $ sudo su - # ovs-vsctl show | grep "tapc07f336f" "tapc07f336f-72" │··················· Interface "tapc07f336f-72"
Looking at the interface seeting on the port shows the qos rules:
# ovs-vsctl list interface tapc07f336f-72 _uuid : f2d0313d-a945-493f-a268-b70885a134c3 admin_state : up bfd : {} bfd_status : {} cfm_fault : [] cfm_fault_status : [] cfm_flap_count : [] cfm_health : [] cfm_mpid : [] cfm_remote_mpids : [] cfm_remote_opstate : [] duplex : full error : [] external_ids : {attached-mac="fa:16:3e:cc:a5:b1", iface-id="c07f336f-72ee-4ed3-849a-7181bd2a0492", iface-status=active} ifindex : 9 ingress_policing_burst: 300 ingress_policing_rate: 3000 lacp_current : [] link_resets : 1 link_speed : 10000000000 link_state : up lldp : {} mac : [] mac_in_use : "c2:fd:5c:d0:f1:4b" mtu : 1500 mtu_request : [] name : "tapc07f336f-72" ofport : 3 ofport_request : [] options : {} other_config : {} statistics : {collisions=0, rx_bytes=654378013, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=88605, tx_bytes=6777887, tx_dropped=0, tx_errors=0, tx_packets=98736} status : {driver_name=veth, driver_version="1.0", firmware_version=""}
