Wednesday 29 November 2017

Enabling QoS with OpenStack Charms

The next charm release will include the option to enable the QoS plugin for neutron. QoS can be enabled via the 'enable-qos' setting in the neutron-api charm.

Enabling QoS

The charm encapsulates all the work of setting up QoS so all that is needed to enable it is:

$ juju config neutron-api enable-qos=True

Testing QoS On a Guest

In the test below traffic speed between two OpenStack guests is generated with iperf with and without a QoS bandwidth policy applied to the guest 'qos-test-1'. Below are the two guests used in the test.

$ nova list
+--------------------------------------+------------+--------+------------+-------------+------------------------------------+
| ID                                   | Name       | Status | Task State | Power State | Networks                           |
+--------------------------------------+------------+--------+------------+-------------+------------------------------------+
| 0b8fcb80-8210-4034-918a-a253f2f810bf | qos-test-1 | ACTIVE | -          | Running     | private=192.168.21.8, 10.5.150.8   |
| ee7d2a15-fb20-448e-81c2-cedd52bc45c3 | qos-test-2 | ACTIVE | -          | Running     | private=192.168.21.12, 10.5.150.12 |
+--------------------------------------+------------+--------+------------+-------------+------------------------------------+

The 192.168 addresses are on the internal project network and the 10.5 addresses are floating IPs which have been assigned to the guests.

The first step is to create a QoS policy that rules can be added to:

$ neutron qos-policy-create bw-limiter
Created a new policy:
+-----------------+--------------------------------------+
| Field           | Value                                |
+-----------------+--------------------------------------+
| created_at      | 2017-09-19T16:27:31Z                 |
| description     |                                      |
| id              | 5440ddb6-b93c-44d6-b0a8-b3b36e552db3 |
| name            | bw-limiter                           |
| project_id      | 7a6b7bcc7284483f88dc44f765cad5df     |
| revision_number | 1                                    |
| rules           |                                      |
| shared          | False                                |
| tenant_id       | 7a6b7bcc7284483f88dc44f765cad5df     |
| updated_at      | 2017-09-19T16:27:31Z                 |
+-----------------+--------------------------------------+

Now add a rule to the policy:

$ neutron qos-bandwidth-limit-rule-create bw-limiter --max-kbps 3000 --max-burst-kbps 300
Created a new bandwidth_limit_rule:
+----------------+--------------------------------------+
| Field          | Value                                |
+----------------+--------------------------------------+
| id             | 9d0bd218-d04e-4444-83e5-35f6e9ec8b44 |
| max_burst_kbps | 300                                  |
| max_kbps       | 3000                                 |
+----------------+--------------------------------------+

qos-test-1's IP address on the internal network is 192.168.21.8 and the corresponding port will have the QoS rule applied to it. List the ports and find the one that corresponds with the guests IP address


$ neutron port-list
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                            |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| 735b2928-578d-491e-9fff-3e45bd82237e |      | fa:16:3e:35:01:2d | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.8"}  |
| bb97d685-a891-4492-93b7-ea7b6caa367e |      | fa:16:3e:03:0a:4c | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.12"} |
| bf8aac5f-959f-4382-9b4d-10ce8aded036 |      | fa:16:3e:55:60:28 | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.2"}  |
| c07f336f-72ee-4ed3-849a-7181bd2a0492 |      | fa:16:3e:cc:a5:b1 | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.4"}    |
| c6c04b68-a1e3-48f1-a2af-7cfabf989c97 |      | fa:16:3e:7d:4d:d9 | {"subnet_id": "e76ed2d3-2f2e-4ca5-9c5c-503cd17c0e91", "ip_address": "192.168.21.1"}  |
| d75d8ee3-71c1-41f9-90ed-f379f0256f1d |      | fa:16:3e:ff:2b:2d | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.12"}   |
| dd33eb79-aaf7-4135-aa4b-28abfbb9fb50 |      | fa:16:3e:f7:6a:4d | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.8"}    |
| ea056cbc-be5b-4223-a7b3-e548aca8a2cb |      | fa:16:3e:6d:dc:94 | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.9"}    |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+


Apply the policy to the port:

$ neutron port-update 735b2928-578d-491e-9fff-3e45bd82237e --qos-policy bw-limiter
Updated port: 735b2928-578d-491e-9fff-3e45bd82237e

On qos-test-2 start the iperf server:

ubuntu@qos-test-2:~$ iperf -s -p 8080
------------------------------------------------------------
Server listening on TCP port 8080
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
On qos-test-1 (the guest with the qos rule applied) run iperf connecting to qos-test-2:

ubuntu@qos-test-1:~$ iperf -c 192.168.21.12 -p 8080
------------------------------------------------------------
Client connecting to 192.168.21.12, TCP port 8080
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.21.8 port 49704 connected with 192.168.21.12 port 8080
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-11.2 sec  2.12 MBytes  1.60 Mbits/sec

On that test achieved a bandwidth of 1.6 MBits/s. It is worth running the test a few times to validate the rate. Now, rerun test with the QoS policy removed. To remove the policy from the port:

$ neutron port-update 735b2928-578d-491e-9fff-3e45bd82237e --no-qos-policy
Updated port: 735b2928-578d-491e-9fff-3e45bd82237e

Rerunning the test:

ubuntu@qos-test-1:~$ iperf -c 192.168.21.12 -p 8080
------------------------------------------------------------
Client connecting to 192.168.21.12, TCP port 8080
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.21.8 port 49705 connected with 192.168.21.12 port 8080
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   403 MBytes   338 Mbits/sec

This time the test achieved a bandwidth of 388 MBits/s.

Testing QoS on a Network

The QoS policy can also be applied to the router port to affect all guests on a given network. In the previous example the last test removed the QoS policy from qos-test-1's port but just in case double check it is gone:


$ neutron port-show 735b2928-578d-491e-9fff-3e45bd82237e | grep qos_policy_id
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
| qos_policy_id         |                                                                                     |

In example below there is an iperf server running on an external host at 10.5.0.9. Collect iperf performance figures from both guests to the external host:


ubuntu@qos-test-1:~$ iperf -c 10.5.0.9 -p 8080                                                                                                                                            
------------------------------------------------------------
Client connecting to 10.5.0.9, TCP port 8080
TCP window size:  325 KByte (default)
------------------------------------------------------------
[  3] local 192.168.21.8 port 34654 connected with 10.5.0.9 port 8080
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   321 MBytes   269 Mbits/sec
ubuntu@qos-test-1:~$ 

and the second guest:


ubuntu@qos-test-2:~$ iperf -c 10.5.0.9 -p 8080
------------------------------------------------------------
Client connecting to 10.5.0.9, TCP port 8080
TCP window size:  325 KByte (default)
------------------------------------------------------------
[  3] local 192.168.21.12 port 59839 connected with 10.5.0.9 port 8080
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   287 MBytes   241 Mbits/sec
ubuntu@qos-test-2:~$

Now get the external ip of the project router and apply the QoS policy to it:


$ neutron router-list
+--------------------------------------+-----------------+------------------------------------------------------------------------+-------------+-------+
| id                                   | name            | external_gateway_info                                                  | distributed | ha    |
+--------------------------------------+-----------------+------------------------------------------------------------------------+-------------+-------+
| 51a15d70-5917-4dc2-9ab4-03c0f03e4fd7 | provider-router | {"network_id": "73c17d70-0298-4a0f-9158-f40bd73ee92e", "enable_snat":  | False       | False |
|                                      |                 | true, "external_fixed_ips": [{"subnet_id":                             |             |       |
|                                      |                 | "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.4"}]}  |             |       |
+--------------------------------------+-----------------+------------------------------------------------------------------------+-------------+-------+

$ neutron port-list | grep 10.5.150.4
| c07f336f-72ee-4ed3-849a-7181bd2a0492 |      | fa:16:3e:cc:a5:b1 | {"subnet_id": "3dc53883-8ca9-43f6-bb19-ea4742bd9357", "ip_address": "10.5.150.4"}    |

$ neutron port-update c07f336f-72ee-4ed3-849a-7181bd2a0492 --qos-policy bw-limiter
Updated port: c07f336f-72ee-4ed3-849a-7181bd2a0492

Rerunning the benchmarks:


ubuntu@qos-test-1:~$ iperf -c 10.5.0.9 -p 8080
------------------------------------------------------------
Client connecting to 10.5.0.9, TCP port 8080
TCP window size:  325 KByte (default)
------------------------------------------------------------
[  3] local 192.168.21.8 port 34655 connected with 10.5.0.9 port 8080
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-11.1 sec  2.38 MBytes  1.80 Mbits/sec


ubuntu@qos-test-2:~$ iperf -c 10.5.0.9 -p 8080
------------------------------------------------------------
Client connecting to 10.5.0.9, TCP port 8080
TCP window size:  325 KByte (default)
------------------------------------------------------------
[  3] local 192.168.21.12 port 59840 connected with 10.5.0.9 port 8080
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.8 sec  2.25 MBytes  1.75 Mbits/sec

Under the Hood

Connect to the neutron-gateway and find the IP of the ovs port corresponding to the external router port.

$ juju ssh neutron-gateway/0
$ sudo su -
# ovs-vsctl show | grep "tapc07f336f"                                                                                                                          
"tapc07f336f-72"                                                                                                                                                             │···················
            Interface "tapc07f336f-72"  

Looking at the interface seeting on the port shows the qos rules:

# ovs-vsctl list interface tapc07f336f-72
_uuid               : f2d0313d-a945-493f-a268-b70885a134c3
admin_state         : up
bfd                 : {}
bfd_status          : {}
cfm_fault           : []
cfm_fault_status    : []
cfm_flap_count      : []
cfm_health          : []
cfm_mpid            : []
cfm_remote_mpids    : []
cfm_remote_opstate  : []
duplex              : full
error               : []
external_ids        : {attached-mac="fa:16:3e:cc:a5:b1", iface-id="c07f336f-72ee-4ed3-849a-7181bd2a0492", iface-status=active}
ifindex             : 9
ingress_policing_burst: 300
ingress_policing_rate: 3000
lacp_current        : []
link_resets         : 1
link_speed          : 10000000000
link_state          : up
lldp                : {}
mac                 : []
mac_in_use          : "c2:fd:5c:d0:f1:4b"
mtu                 : 1500
mtu_request         : []
name                : "tapc07f336f-72"
ofport              : 3
ofport_request      : []
options             : {}
other_config        : {}
statistics          : {collisions=0, rx_bytes=654378013, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=88605, tx_bytes=6777887, tx_dropped=0, tx_errors=0, tx_packets=98736}
status              : {driver_name=veth, driver_version="1.0", firmware_version=""}



No comments:

Post a Comment