基於 Open vSwitch 的 OpenFlow 親測實踐

2020-06-16 17:12:48

今天看到朋友推薦的一篇IBM的文章 http://www.linuxidc.com/Linux/2017-06/144770.htm

文章內容很好，如果我只是收藏起來，那就實在是太浪費，還是動手練習一次，好好補一下我的網路。還是用UnitedStack的UOS，建立一個虛擬機器來完成全部的試驗。

文章就一個小筆誤，把ns2寫成ns3。外面很多轉載的文章，都沒註明出處，搞的以為是原創。如果錯誤都一樣，就有點不好意思。

和IBM的文件不一樣的地方是：我使用的是Ubuntu 14.04，減少很多沒必要的麻煩。

建立一個Ubuntu 14.04的最小設定虛擬機器，ssh到虛擬機器上。

檢視Ubuntu版本

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.1 LTS
Release:        14.04
Codename:       trusty

OVS

Ubuntu 14.04的OVS版本，已經是2.02，所以預設安裝就可以。不過不同的發行版，ovs的名字會有點不同。

apt-cache search openvswitch

開始安裝

apt-get install openvswitch-switch

檢視OVS執行情況

# ps -ea | grep ovs
 3007 ?        00:00:00 ovsdb-server
 3017 ?        00:00:00 ovs-vswitchd

檢視OVS版本

# ovs-appctl --version
ovs-appctl (Open vSwitch) 2.0.2
Compiled Aug 15 2014 14:31:01

檢視 OVS 支援的 OpenFlow 協定的版本

# ovs-ofctl --version
ovs-ofctl (Open vSwitch) 2.0.2
Compiled Aug 15 2014 14:31:02
OpenFlow versions 0x1:0x4

OpenFlow 命令

建立一個OVS交換機

ovs-vsctl add-br ovs-switch

建立一個埠 p0，設定埠 p0 的 OpenFlow 埠編號為 100

ovs-vsctl add-port ovs-switch p0 -- set Interface p0 ofport_request=100

設定網路介面裝置型別為”internal”,

ovs-vsctl set Interface p0 type=internal

檢視設定結果

# ethtool -i p0
driver: openvswitch
version: 
firmware-version: 
bus-info: 
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

建立一個name space：ns0，把p0埠接入到ns0裡，並且設定ip地址 192.168.1.100/24

ip netns add ns0
ip link set p0 netns ns0
ip netns exec ns0 ip addr add 192.168.1.100/24 dev p0
ip netns exec ns0 ifconfig p0 promisc up

檢視建立結果

# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
    Bridge ovs-switch
        Port "p0"
            Interface "p0"
                type: internal
        Port ovs-switch
            Interface ovs-switch
                type: internal
    ovs_version: "2.0.2"

重複步驟，建立p1和p2埠

ovs-vsctl add-port ovs-switch p1 -- set Interface p1 ofport_request=101
ovs-vsctl set Interface p1 type=internal
ip netns add ns1
ip link set p1 netns ns1
ip netns exec ns1 ip addr add 192.168.1.101/24 dev p1
ip netns exec ns1 ifconfig p1 promisc up

檢視建立結果

# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
    Bridge ovs-switch
        Port "p1"
            Interface "p1"
                type: internal
        Port "p0"
            Interface "p0"
                type: internal
        Port ovs-switch
            Interface ovs-switch
                type: internal
    ovs_version: "2.0.2"

建立p2

ovs-vsctl add-port ovs-switch p2 -- set Interface p2 ofport_request=102
ovs-vsctl set Interface p2 type=internal
ip netns add ns2
ip link set p2 netns ns2
ip netns exec ns2 ip addr add 192.168.1.102/24 dev p2
ip netns exec ns2 ifconfig p2 promisc up

檢視

# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
    Bridge ovs-switch
        Port "p1"
            Interface "p1"
                type: internal
        Port "p2"
            Interface "p2"
                type: internal
        Port "p0"
            Interface "p0"
                type: internal
        Port ovs-switch
            Interface ovs-switch
                type: internal
    ovs_version: "2.0.2"

檢視建立的交換機資訊，獲得dpid，埠openflow埠編號

# ovs-ofctl show ovs-switch
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000d23b94ce4146
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN 
SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_
DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 100(p0): addr:00:00:00:00:00:00
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
 101(p1): addr:00:00:00:00:00:00
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
 102(p2): addr:00:00:00:00:00:00
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
 LOCAL(ovs-switch): addr:2a:be:0c:72:40:45
     config:     PORT_DOWN
     state:      LINK_DOWN
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

獲取openflow埠編號

# ovs-vsctl get Interface p0 ofport
100
# ovs-vsctl get Interface p1 ofport
101
# ovs-vsctl get Interface p2 ofport
102

檢視 datapath 的資訊

# ovs-dpctl show
system@ovs-system:
        lookups: hit:34 missed:21 lost:0
        flows: 0
        port 0: ovs-system (internal)
        port 1: ovs-switch (internal)
        port 2: p0 (internal)
        port 3: p1 (internal)
        port 4: p2 (internal)

檢視mac地址

ip netns exec ns0 ping 192.168.1.100
ip netns exec ns0 ping 192.168.1.101
ip netns exec ns0 ping 192.168.1.102

然後執行

# ovs-appctl fdb/show ovs-switch      
 port  VLAN  MAC                Age
  102     0  22:8e:52:36:92:25   17
  100     0  d6:0f:7e:ed:11:e4    4
  101     0  f2:0d:06:ff:79:d7    4

檢視交換機所有table

ovs-ofctl dump-tables ovs-switch

檢視交換機中的所有流表項

ovs−ofctl dump−flows ovs-switch

刪除編號為 100 的埠上的所有流表項

ovs-ofctl del-flows ovs-switch "in_port=100"

檢視交換機埠資訊

ovs-ofctl show ovs-switch

修改封包

遮蔽所有進入 OVS 的乙太網廣播封包

$ ovs-ofctl add-flow ovs-switch "table=0, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=drop"

遮蔽 STP 協定的廣播封包

$ ovs-ofctl add-flow ovs-switch "table=0, dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0, actions=drop"

修改封包，新增新的 OpenFlow 條目，修改從埠 p0 收到的封包的源地址為 9.181.137.1

ovs-ofctl add-flow ovs-switch "priority=1 idle_timeout=0,
    in_port=100,actions=mod_nw_src:9.181.137.1,normal"

從埠 p0（192.168.1.100）傳送測試資料到埠 p1（192.168.1.101），就是沒啥響應

# ip netns exec ns0 ping 192.168.1.101
PING 192.168.1.101 (192.168.1.101) 56(84) bytes of data.

再開啟一個ssh終端，登入進去，執行tcpdump，需要等待幾分鐘，才能看到響應

~# ip netns exec ns1 tcpdump -i p1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on p1, link-type EN10MB (Ethernet), capture size 65535 bytes
06:21:23.802308 IP 9.181.137.1 > 192.168.1.101: ICMP echo request, id 4533, seq 19, length 64
06:21:24.802358 IP 9.181.137.1 > 192.168.1.101: ICMP echo request, id 4533, seq 20, length 64

重定向封包

新增新的 OpenFlow 條目，重定向所有的 ICMP 封包到埠 p2

ovs-ofctl add-flow ovs-switch idle_timeout=0,dl_type=0x0800,nw_proto=1,actions=output:102

從埠 p0 （192.168.1.100）傳送資料到埠 p1（192.168.1.101）

ip netns exec ns0 ping 192.168.1.101

這個時候你從p2裡，可以看到

# ip netns exec ns2 tcpdump -i p2 icmp 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on p2, link-type EN10MB (Ethernet), capture size 65535 bytes
06:25:38.252471 IP 192.168.1.100 > 192.168.1.101: ICMP echo request, id 4668, seq 35, length 64
06:25:39.260438 IP 192.168.1.100 > 192.168.1.101: ICMP echo request, id 4668, seq 36, length 64
06:25:40.268419 IP 192.168.1.100 > 192.168.1.101: ICMP echo request, id 4668, seq 37, length 64

修改vlan tag

修改埠 p1 的 VLAN tag 為 101，使埠 p1 成為一個隸屬於 VLAN 101 的埠

ovs-vsctl set Port p1 tag=101

現在由於埠 p0 和 p1 屬於不同的 VLAN，它們之間無法進行資料交換。我們使用 ovs-appctl ofproto/trace 生成一個從埠 p0 傳送到埠 p1 的封包，這個封包不包含任何 VLAN tag，並觀察 OVS 的處理過程

ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,
dl_dst=f2:0d:06:ff:79:d7 -generate

注意：上面??一個mac地址，是p0的，第二個mac地址是p1的，你需要替換，上面有獲取mac地址的方法。

# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,
> dl_dst=f2:0d:06:ff:79:d7 -generate
Flow: metadata=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000
Rule: table=0 cookie=0 priority=1,in_port=100
OpenFlow actions=mod_nw_src:9.181.137.1,NORMAL
no learned MAC for destination, flooding

Final flow: unchanged
Relevant fields: skb_priority=0,in_port=100,vlan_tci=0x0000/0x1fff,dl_src=d6:0f:7e:ed:11:e4,
dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000,nw_src=0.0.0.0,nw_proto=0,nw_frag=no
Datapath actions: 1,4

在第一行輸出中，“Flow:”之後的欄位描述了輸入的流的資訊。由於我們沒有指定太多資訊，所以多數位段（例如 dl_type 和 vlan_tci）被 OVS 設定為空值。

在第二行的輸出中，“Rule:” 之後的欄位描述了匹配成功的流表項。

在第三行的輸出中，“OpenFlow actions”之後的欄位描述了實際執行的操作。

最後一段以”Final flow”開始的欄位是整個處理過程的總結，“Datapath actions: 4,1”代表封包被傳送到 datapath 的 4 和 1 號埠。

建立一條新的 Flow

對於從埠 p0 進入交換機的封包，如果它不包含任何 VLAN tag，則自動為它新增 VLAN tag 101

ovs-ofctl add-flow ovs-switch "priority=3,in_port=100,dl_vlan=0xffff,
actions=mod_vlan_vid:101,normal"

再次嘗試從埠 p0 傳送一個不包含任何 VLAN tag 的封包，發現封包進入埠 p0 之後, 會被加上 VLAN tag101, 同時轉發到埠 p1 上

# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,
> dl_dst=f2:0d:06:ff:79:d7 -generate
Flow: metadata=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000
Rule: table=0 cookie=0 priority=1,in_port=100
OpenFlow actions=mod_nw_src:9.181.137.1,NORMAL
no learned MAC for destination, flooding

Final flow: unchanged
Relevant fields: skb_priority=0,in_port=100,vlan_tci=0x0000/0x1fff,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000,nw_src=0.0.0.0,nw_proto=0,nw_frag=no
Datapath actions: 1,4
root@ovs:~# ovs-ofctl add-flow ovs-switch "priority=3,in_port=100,dl_vlan=0xffff,
> actions=mod_vlan_vid:101,normal"
root@ovs:~# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=d6:0f:7e:ed:11:e4,
> dl_dst=f2:0d:06:ff:79:d7 -generate
Flow: metadata=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000
Rule: table=0 cookie=0 priority=3,in_port=100,vlan_tci=0x0000
OpenFlow actions=mod_vlan_vid:101,NORMAL
no learned MAC for destination, flooding

Final flow: metadata=0,in_port=100,dl_vlan=101,dl_vlan_pcp=0,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000
Relevant fields: skb_priority=0,in_port=100,vlan_tci=0x0000,dl_src=d6:0f:7e:ed:11:e4,dl_dst=f2:0d:06:ff:79:d7,dl_type=0x0000,nw_proto=0,nw_frag=no
Datapath actions: push_vlan(vid=101,pcp=0),1,pop_vlan,3,push_vlan(vid=101,pcp=0),4

反過來從埠 p1 傳送封包，由於 p1 現在是帶有 VLAN tag 101 的 Access 型別的埠，所以封包進入埠 p1 之後，會被 OVS 新增 VLAN tag 101 並行送到埠 p0

# ovs-appctl ofproto/trace ovs-switch in_port=101,dl_src=f2:0d:06:ff:79:d7,
> dl_dst=d6:0f:7e:ed:11:e4 -generate
Flow: metadata=0,in_port=101,vlan_tci=0x0000,dl_src=f2:0d:06:ff:79:d7,dl_dst=d6:0f:7e:ed:11:e4,dl_type=0x0000
Rule: table=0 cookie=0 priority=0
OpenFlow actions=NORMAL
no learned MAC for destination, flooding

Final flow: unchanged
Relevant fields: skb_priority=0,in_port=101,vlan_tci=0x0000,dl_src=f2:0d:06:ff:79:d7,dl_dst=d6:0f:7e:ed:11:e4,dl_type=0x0000,nw_proto=0,nw_frag=no
Datapath actions: push_vlan(vid=101,pcp=0),1,2,4

Floodlight

新建立一個ubuntu 14.04的虛擬機器。

apt-get update
apt-get install git
apt-get install ant
apt-get install openjdk-7-jdk

原始碼安裝

git clone git://github.com/floodlight/floodlight.git
cd floodlight/
ant
java -jar target/floodlight.jar

這個時候floodlight就啟動起來，最後一條命令，就是啟動floodlight。

登入OVS節點

設定ovs的控制器為floodlight，10.250.3.10，就是floodlight虛擬機器的IP。

ovs-vsctl set-controller ovs-switch tcp:10.250.3.10:6633

設定 OVS 的連線模式為 secure 模式

ovs-vsctl set Bridge ovs-switch fail-mode=secure

檢視

# ovs-vsctl show
6507c214-0c7a-4159-9813-977074f73aa1
    Bridge ovs-switch
        Controller "tcp:10.250.3.10:6633"
            is_connected: true
        fail_mode: secure
        Port "p1"
            tag: 101
            Interface "p1"
                type: internal
        Port "p2"
            Interface "p2"
                type: internal
        Port "p0"
            Interface "p0"
                type: internal
        Port ovs-switch
            Interface ovs-switch
                type: internal
    ovs_version: "2.0.2"

通過存取 Floodlight 提供的 Web 管理介面 http://<Host Address>:8080/ui/index.html，我們可以檢視 Floodlight 控制器的狀態以及所有連線到 Floodlight 的交換機列表

通過 Floodlight 的 RESTAPI，新增兩條新的規則讓埠 p0 和 p1 可以相互通訊。注意：替換命令列中的 switch 的 ID 為交換機的 datapath ID

注意curl命令，盡量別用 / 換行

curl -d '{"switch": "00:00:d2:3b:94:ce:41:46", "name":"my-flow1", "cookie":"0","priority":"32768","ingress-port":"100","active":"true", "actions":"output=flood"}' http://10.250.3.10:8080/wm/staticflowentrypusher/json

curl -d '{"switch": "00:00:d2:3b:94:ce:41:46", "name":"my-flow2", "cookie":"0","priority":"32768","ingress-port":"101","active":"true", "actions":"output=flood"}' http://10.250.3.10:8080/wm/staticflowentrypusher/json

驗證是否能從埠 p0 傳送封包到 p1

# ip netns exec ns0 ping -c4 192.168.1.101
PING 192.168.1.101 (192.168.1.101) 56(84) bytes of data.
64 bytes from 192.168.1.101: icmp_seq=1 ttl=64 time=0.625 ms
64 bytes from 192.168.1.101: icmp_seq=2 ttl=64 time=0.088 ms
64 bytes from 192.168.1.101: icmp_seq=3 ttl=64 time=0.082 ms
64 bytes from 192.168.1.101: icmp_seq=4 ttl=64 time=0.048 ms

在 OVS 端也可以看到，流表規則已經被 OVS 同步到本地。

# ovs-ofctl dump-flows ovs-switch
NXST_FLOW reply (xid=0x4):
 cookie=0xa00000626d6af5, duration=111.468s, table=0, n_packets=7, n_bytes=630, idle_age=2, in_port=100 actions=FLOOD
 cookie=0xa00000626d6af6, duration=83.717s, table=0, n_packets=7, n_bytes=630, idle_age=1, in_port=101 actions=FLOOD

通過 Floodlight 的 RestAPI，檢視交換機上的流表規則

curl http://10.250.3.10:8080/wm/staticflowentrypusher/list/00:00:d2:3b:94:ce:41:46/json | python -mjson.tool

採用python的輸出，好看很多的

# curl http://10.250.3.10:8080/wm/staticflowentrypusher/list/00:00:d2:3b:94:ce:41:46/json | python -mjson.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1435    0  1435    0     0   109k      0 --:--:-- --:--:-- --:--:--  116k
{
    "00:00:d2:3b:94:ce:41:46": {
        "my-flow1": {
            "actions": [
                {
                    "length": 8,
                    "lengthU": 8,
                    "maxLength": 32767,
                    "port": -5,
                    "type": "OUTPUT"
                }
            ],
            "bufferId": -1,
            "command": 0,
            "cookie": 45035997925042933,
            "flags": 0,
            "hardTimeout": 0,
            "idleTimeout": 0,
            "length": 80,
            "lengthU": 80,
            "match": {
                "dataLayerDestination": "00:00:00:00:00:00",
                "dataLayerSource": "00:00:00:00:00:00",
                "dataLayerType": "0x0000",
                "dataLayerVirtualLan": -1,
                "dataLayerVirtualLanPriorityCodePoint": 0,
                "inputPort": 100,
                "networkDestination": "0.0.0.0",
                "networkDestinationMaskLen": 0,
                "networkProtocol": 0,
                "networkSource": "0.0.0.0",
                "networkSourceMaskLen": 0,
                "networkTypeOfService": 0,
                "transportDestination": 0,
                "transportSource": 0,
                "wildcards": 4194302
            },
            "outPort": -1,
            "priority": -32768,
            "type": "FLOW_MOD",
            "version": 1,
            "xid": 0
        },
        "my-flow2": {
            "actions": [
                {
                    "length": 8,
                    "lengthU": 8,
                    "maxLength": 32767,
                    "port": -5,
                    "type": "OUTPUT"
                }
            ],
            "bufferId": -1,
            "command": 0,
            "cookie": 45035997925042934,
            "flags": 0,
            "hardTimeout": 0,
            "idleTimeout": 0,
            "length": 80,
            "lengthU": 80,
            "match": {
                "dataLayerDestination": "00:00:00:00:00:00",
                "dataLayerSource": "00:00:00:00:00:00",
                "dataLayerType": "0x0000",
                "dataLayerVirtualLan": -1,
                "dataLayerVirtualLanPriorityCodePoint": 0,
                "inputPort": 101,
                "networkDestination": "0.0.0.0",
                "networkDestinationMaskLen": 0,
                "networkProtocol": 0,
                "networkSource": "0.0.0.0",
                "networkSourceMaskLen": 0,
                "networkTypeOfService": 0,
                "transportDestination": 0,
                "transportSource": 0,
                "wildcards": 4194302
            },
            "outPort": -1,
            "priority": -32768,
            "type": "FLOW_MOD",
            "version": 1,
            "xid": 0
        }
    }
}

通過 Floodlight 的 RestAPI，刪除交換機上的流表規則

curl http://10.250.3.10:8080/wm/staticflowentrypusher/clear/00:00:d2:3b:94:ce:41:46/json

本文永久更新連結地址：http://www.linuxidc.com/Linux/2017-06/144772.htm

基於 Open vSwitch 的 OpenFlow 親測實踐

OVS

OpenFlow 命令

修改封包

修改vlan tag

Floodlight

熱門文章