Network Architecture

Network Architecture

The network setup is of is not a simple one, this due to several reasons:

  • Multi-site setup, with servers on different locations, connected either directly or through vpn
  • Multi-tenancy setup, multiple layers of access on the network need to be implemented
  • Multiple vpn endpoints, this to connect the different tenants
  • Multiple servers within a single Kubernetes cluster.
  • Virtual network interfaces with virtual network policies in Kubernetes.
  • Multiple networking hardware devices such as routers, managed switches and IOT devices.

To break down these complexities we can split the network architecture into two parts:

  1. Technical Layer - (OSI layers 1 - 4)
  2. Data Layer - (OSI layers 5 - 7)

Network topology table

The following table summarizes all network cidrs and addresses

192.1681.0/24Empty, not used, will indicate wrongly configured devices-
192.1682.0/24Common devices, laptops, phones, etc.1
192.1683.0/24IOT Devices with dedicated connection to server1
192.1684.0/24Network infrastructure, switches, routers, etc1
192.1684.0/244.1Router and DHCP Server1
192.1684.0/244.2Central network switch1
192.1684.0/244.3Access Point living room1
192.1684.0/244.4Access Point office1
192.1685.0/205.100haproxy VIP entrypoint Kubernetes1
192.1686.0/24Kubernetes MetalLB services1
192.1686.0/25LAN Kubernetes MetalLB services1
192.1686.0/256.1LAN Traefik 2.x1
192.1686.0/256.60Plex server1
192.1686.0/256.71qBittorent listen1
192.1686.0/256.65Kubernetes Dashboard1
192.1686.0/256.77Log server1
192.1686.0/256.88Tekton server1
192.1686.0/256.90Consul LAN DNS (DNS UDP)1
192.1686.0/256.91Consul LAN DNS (Admin UI backup)1
192.1686.0/256.99DNS Blackhole (pihole)1
192.1686.128/25Online Kubernetes MetalLB services1
192.1686.128/256.128Online Traefik 2.x1
10.2440.0/16Kubernetes internal cidrkubernetes.local
10.82.0/24Shared VPN accessopenvpn shared
10.82.0/242.1Shared VPN serveropenvpn shared
10.84.0/24Private VPN accessopenvpn private
10.84.0/244.1Private VPN serveropenvpn private

Technical Layer

TODO: Update diagram with new hardware

The Technical Layer is structured as follows:

Network with WeaveWorks

Initially the decision was made to use flannel as network provider, since this is a pretty standard choice for many k8s implementations. Unfortunately this gave several networking, performance and upgrading issues over time, especially with our multi cpu architecture environment. After a tool selection process weave works came out best because:

  • Substantial performance and stability improvements
  • Capable of complex network segregation which flannel was not able to do
  • Easy to setup, even if several sources on the way point out the opposite
  • Bonus: Weave works supplies a fancy and comprehensive dashboard of your entire network

Data Layer

The Data Layer is structured as follows: