本文共 12599 字,大约阅读时间需要 41 分钟。
Openstack的网络 Neutron,
Openstack 网络架构: 首先注意网络节点运行环境中所有的网络服务,但不包含网络API服务【这个服务在控制节点上运行】1、External Network/API Network【外部网络和API网络】
这个网络是连接外网的,无论是用户调用Openstack的API,还是创建出来的虚拟机要访问外网,或者外网要ssh到虚拟机,都需要通过这个网络 2、Instance Network, 就是上面说的虚拟机通信网络,虚拟机之间的数据传输通过这个网络来进行,比如一个虚拟机要连接另一个虚拟机,虚拟机要连接虚拟的路由都是通过这个网络来进行。 3、Management Network, 管理网络,Openstack各个模块之间的交互,连接数据库,连接Message Queue都是通过这个网络来就拿我部署的来说吧:
各个节点的ifconfig属性就是这样 控制节点: eth0 192.168.1.71 eth1 192.168.5.71 eth2 192.168.6.71 eth3 10.190.3.71计算节点:
eth0 192.168.1.72 eth1 192.168.5.72 eth2 192.168.6.72网络节点:
eth0 192.168.1.73 eth1 192.168.5.73 eth2 192.168.6.73存储节点:
eth0 192.168.1.74 eth1 192.168.5.74 eth2 192.168.6.74可以看出这几个网段就是对应的neutron所对应几个网络类型。
一般的,OpenStack中网络实现包括vlan、gre、vxlan 等模式
显示控制节点上有三个网桥 br-ex,br-int 和 br-tun。从命名上看我们大致能猜出他们的用途:
br-ex
连接外部(external)网络的网桥。br-int
集成(integration)网桥,所有 instance 的虚拟网卡和其他虚拟网络设备都将连接到该网桥。br-tun
隧道(tunnel)网桥,基于隧道技术的 VxLAN 和 GRE 网络将使用该网桥进行通信。这些网桥都是 Neutron 自动为我们创建的,但是通过 brctl show 命令却看不到它们。这是因为我们使用的是 Open vSwitch 【OpenvSwitch,简称OVS是一个虚拟交换软件,主要用于虚拟机VM环境】作为一个虚拟交换机
而非 Linux Bridge,需要用 Open vSwitch 的命令 ovs-vsctl show 查看, 网络节点: sles11sp3x64-hsm1-vm4:~ # ifconfig br-ex Link encap:Ethernet HWaddr F2:F4:51:1E:9F:43 inet6 addr: fe80::f4aa:d4ff:feaa:4838/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:13045738 errors:0 dropped:12249 overruns:0 frame:0 TX packets:118613 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1141500425 (1088.6 Mb) TX bytes:9808330 (9.3 Mb)br-int Link encap:Ethernet HWaddr 16:8E:5E:BF:11:4E
inet6 addr: fe80::ac6e:beff:fe15:cafd/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:50267 errors:0 dropped:35 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2605212 (2.4 Mb) TX bytes:180 (180.0 b)br-tun Link encap:Ethernet HWaddr D6:45:A8:89:4F:40
inet6 addr: fe80::2859:fbff:fe78:131a/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:180 (180.0 b)计算节点:
br-int Link encap:Ethernet HWaddr 5E:E9:D1:4E:3C:41 inet6 addr: fe80::fc23:68ff:fec5:f156/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:50412 errors:0 dropped:35 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2646940 (2.5 Mb) TX bytes:180 (180.0 b)br-tun Link encap:Ethernet HWaddr 2A:82:AD:9E:15:4F
inet6 addr: fe80::4c76:3dff:fe3a:a50f/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:180 (180.0 b)计算节点上也有 br-int 和 br-tun,但没有 br-ext。这是合理的,因为发送到外网的流量是通过网络节点上的虚拟路由器转发出去的,所以 br-ext 只会放在网络节点(devstack-controller)上。
在 Open vSwitch 环境中,一个数据包从 instance 发送到物理网卡大致会经过下面几个类型的网络设备:
qbr6a59d7 Link encap:Ethernet HWaddr 12:80:DF:71:5B:CC inet6 addr: fe80::70c1:21ff:fe2c:c5f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5715 errors:0 dropped:5 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:242607 (236.9 Kb) TX bytes:468 (468.0 b) 像这个qbr就是linux网桥 qbr9a0552 Link encap:Ethernet HWaddr C6:9A:A1:61:B6:0C inet6 addr: fe80::c0f7:acff:feeb:7ab2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1430 errors:0 dropped:5 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:71684 (70.0 Kb) TX bytes:468 (468.0 b)qbr94c4ce Link encap:Ethernet HWaddr 82:E8:DD:F6:75:F5
inet6 addr: fe80::cccc:56ff:fe93:24af/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:46472 errors:0 dropped:35 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1605306 (1.5 Mb) TX bytes:468 (468.0 b)qbr6906ea Link encap:Ethernet HWaddr 42:A5:F7:69:D9:A3
inet6 addr: fe80::2ced:58ff:fec1:5aa0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7969 errors:0 dropped:5 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:322725 (315.1 Kb) TX bytes:468 (468.0 b)qbr269378 Link encap:Ethernet HWaddr F2:DD:13:17:61:56
inet6 addr: fe80::a02e:22ff:feae:fc48/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:48208 errors:0 dropped:35 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1872712 (1.7 Mb) TX bytes:468 (468.0 b) 像这个qvb,qvo就是veth pair qvb6a59d7 Link encap:Ethernet HWaddr 12:80:DF:71:5B:CC inet6 addr: fe80::1080:dfff:fe71:5bcc/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:101094 errors:0 dropped:0 overruns:0 frame:0 TX packets:83407 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:23152149 (22.0 Mb) TX bytes:5712504 (5.4 Mb)qvb9a0552 Link encap:Ethernet HWaddr C6:9A:A1:61:B6:0C
inet6 addr: fe80::c49a:a1ff:fe61:b60c/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:828764 errors:0 dropped:0 overruns:0 frame:0 TX packets:561433 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1129595537 (1077.2 Mb) TX bytes:51140710 (48.7 Mb)qvb94c4ce Link encap:Ethernet HWaddr 82:E8:DD:F6:75:F5
inet6 addr: fe80::80e8:ddff:fef6:75f5/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:60427 errors:0 dropped:0 overruns:0 frame:0 TX packets:11476 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:6232114 (5.9 Mb) TX bytes:1013513 (989.7 Kb)qvb6906ea Link encap:Ethernet HWaddr 42:A5:F7:69:D9:A3
inet6 addr: fe80::40a5:f7ff:fe69:d9a3/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:156707 errors:0 dropped:0 overruns:0 frame:0 TX packets:148067 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:23789205 (22.6 Mb) TX bytes:24604552 (23.4 Mb)qvb269378 Link encap:Ethernet HWaddr F2:DD:13:17:61:56
inet6 addr: fe80::f0dd:13ff:fe17:6156/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:58492 errors:0 dropped:0 overruns:0 frame:0 TX packets:6436 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:10105534 (9.6 Mb) TX bytes:2488129 (2.3 Mb)qvo6a59d7 Link encap:Ethernet HWaddr 2A:9A:D4:3F:71:C9
inet6 addr: fe80::289a:d4ff:fe3f:71c9/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:83407 errors:0 dropped:0 overruns:0 frame:0 TX packets:101094 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:5712504 (5.4 Mb) TX bytes:23152149 (22.0 Mb)qvo9a0552 Link encap:Ethernet HWaddr C2:D1:F6:7C:36:EF
inet6 addr: fe80::c0d1:f6ff:fe7c:36ef/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:561433 errors:0 dropped:0 overruns:0 frame:0 TX packets:828764 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:51140710 (48.7 Mb) TX bytes:1129595537 (1077.2 Mb)qvo94c4ce Link encap:Ethernet HWaddr 52:72:A5:75:79:C8
inet6 addr: fe80::5072:a5ff:fe75:79c8/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:11476 errors:0 dropped:0 overruns:0 frame:0 TX packets:60427 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1013513 (989.7 Kb) TX bytes:6232114 (5.9 Mb)qvo6906ea Link encap:Ethernet HWaddr 4A:B1:D4:25:A8:7F
inet6 addr: fe80::48b1:d4ff:fe25:a87f/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:148067 errors:0 dropped:0 overruns:0 frame:0 TX packets:156707 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:24604552 (23.4 Mb) TX bytes:23789205 (22.6 Mb)qvo269378 Link encap:Ethernet HWaddr 6E:ED:CF:C1:6D:AD
inet6 addr: fe80::6ced:cfff:fec1:6dad/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6436 errors:0 dropped:0 overruns:0 frame:0 TX packets:58492 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2488129 (2.3 Mb) TX bytes:10105534 (9.6 Mb) Tap就是tap interface tap6a59d7 Link encap:Ethernet HWaddr FE:16:3E:F2:BE:D4 inet6 addr: fe80::fc16:3eff:fef2:bed4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:562 errors:0 dropped:0 overruns:0 frame:0 TX packets:841 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:67097 (65.5 Kb) TX bytes:58361 (56.9 Kb)tap9a0552 Link encap:Ethernet HWaddr FE:16:3E:34:4D:73
inet6 addr: fe80::fc16:3eff:fe34:4d73/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2591 errors:0 dropped:0 overruns:0 frame:0 TX packets:2857 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:230615 (225.2 Kb) TX bytes:201822 (197.0 Kb)tap94c4ce Link encap:Ethernet HWaddr FE:16:3E:CF:A5:1C
inet6 addr: fe80::fc16:3eff:fecf:a51c/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4595 errors:0 dropped:0 overruns:0 frame:0 TX packets:4818 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:445551 (435.1 Kb) TX bytes:2629730 (2.5 Mb)tap6906ea Link encap:Ethernet HWaddr FE:16:3E:E4:E3:61
inet6 addr: fe80::fc16:3eff:fee4:e361/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:147397 errors:0 dropped:0 overruns:0 frame:0 TX packets:154806 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:24575493 (23.4 Mb) TX bytes:23518091 (22.4 Mb)1网段:是flat网络,
Flat:所有虚拟机实例都连接在同一网络中,并且和宿主机可以运行在同一网络中,没有对网络数据包进行打标签(vlan tag)或者隔离。 flat 网络是无 vlan tagging 的网络。flat 网络中的 instance 能与位于同一网络的 instance 通信,并且可以跨多个节点 Neutron 有好几个模块或者说是服务,部署在控制节点和计算节点上。 Controller节点: neutron-server,用于接受API请求创建网络,子网,路由器等,然而创建的这些东西仅仅是一些数据结构在数据库里面。 neutron-l3-agent,用于创建和管理虚拟路由器,当neutron-server将路由器的数据结构创建好,它是做具体的事情的,真正的调用命令行将虚拟路由器,路由表,namespace,iptables规则全部创建好。 neutron-dhcp-agent,用于创建和管理虚拟DHCP Server,每个虚拟网络都会有一个DHCP Server,这个DHCP Server为这个虚拟网络里面的虚拟机提供IP。 neutron-openvswith-plugin-agent,这个是用于创建虚拟的L2的switch的,在网络节点上,Router和DHCP Server都会连接到二层的switch上。 Compute节点: neutron-openvswith-plugin-agent,这个是用于创建虚拟的L2的switch的,在Compute节点上,虚拟机的网卡也是连接到二层的switch上。6网段:是gre网络,是一种封装数据包的协议,
subnet 是一个 IPv4 或者 IPv6 地址段。instance 的 IP 从 subnet 中分配。每个 subnet 需要定义 IP 地址的范围和掩码。
外网连接到 Neutron 的虚拟路由器,这样 instance 才能访问外网。【Route1连接了6网段和1网段】