Home Network, a novice Design

Home Network, a novice Design

May 16, 2017
Networking
Depricated, Ubiquiti

I’ve written before about network hardware selection, where I surmise that Ubiquiti’s EdgeMax products are what I typically rely on when building out a network. Here I’ll lay out what I think is a good design for a home network using some of the inexpensive EdgeMax and Unifi products. It might seem silly that I would include Unifi in a discussion of implementation with EdgeMax, but really you don’t have a better choice for wireless access point than what Unifi offers. Furthermore, I chose EdgeMax for this implementation because it will help to solidify understanding of the standards better than if one would use the software defined networking approach of Unifi.

We make some assumptions before proceeding:

  • you can acquire the listed hardware, or alternatives
  • you have some networking basics
  • you can read peripheral direction from the manufacture (e.g. firmware update process)
  • your service providers modem uses DHCP (static isn’t a huge change but isn’t written in the document)
  • you’re using a single switch and don’t need to muck with things like STP

Design & Standards #

As I’ve dug into more projects that require knowledge of network design I’ve begun to truly appreciate the artful nature of network composition. With a small home network you’ll likely never appreciate all of the available design choices and the craft associated expressing a design that addresses needs and anticipates growth vectors. However, you can use a small home network to get some exposure into the standards that ground all of these things. With a project like this you’re going to get exposure to a bunch of standard from the Internet Protocol Suite which typically developed as a series of RFC that eventually become documented as standards by a standards body, in this case it is IEEE 802.

When you’re wanting to understand something via discussion, or seach, it’s important to be able to layer these standards in your mind so that you understand the context you should be searching within. There are also other standards than the Internet Protocol Suite such as the OSI Stack that are incredibly useful as conceptual bounds for your exploration.

At first all these things might seem overwhelming and tedious, but if you take the time to examine them you’ll see that smart people have gone before you and created wonderful foundations for you to build upon. One that I like to have new people read is the IEEE 802.1Q because it really highlights how much work has to go into something as simple as adding a frame header to a switched packet, and how it can be designed to operate elegantly.

Design #

You should always write out your plan before you begin.

our network goals:

  • a border router with a strong implementation of firewall
  • two sub-networks, one for the home users, another for guests
  • isolation of traffic between those sub-networks
  • wireless access for both home and guest users off the same access point
  • a bastion host that we can use as a hardened proxy to get back into our network
  • use of fiber modules (because fiber is cool)

our networking kit:

You can change out pretty much everything in this for other things that you wan based on your needs. I typically purchase the ES-24-Lite because its the least expensive managed switch that I can easily drop into environments. For the purposes of the write up I’ll assume the following kit so that I can use interface numbers to indicate wiring.

Fiber is also something you can choose to drop, Ethernet over the distances in a typical home is certainty adequate. I personally like to get people to play around with Fiber because I want them to realize its a technology that they can use today, which might make them more interested in municipal broadband and other things that would break up the monstrous trusts that control our countries internet today. For Fiber I typically purchase all of my accessories from FiberStore due to them being so inexpensive, but you need to know what you’re looking for. In a project like this you’d just need Multi Mode SFP modules, no need for SFP+ because you’re using gigabit interfaces. You’ll need LC-LC fiber, and I typically prefer to purchase the armored version from FiberStore because I’m less likely to damage it. Another alternative to fiber that allows you to use the SFP form factor is Direct Attached Copper (DAC) which allow you to use SFP over relatively short distances without the downsides of fiber (e.g. worrying about termination cleanliness).

our layer 1 topology:

Pardon the terrible drawing, but here is the topology:

  • modem -> EdgeRouter eth0
  • EdgeRouter eth5 (fiber) -> EdgeSwitch eth9
  • EdgeSwitch eth8 -> AP-AC-Lite
  • EdgeSwitch eth7 -> Odroid C2
   EdgeRouter X-SFP
+---------------------+
|  +               +--------> diagnostic
+--|------------------+
   |
   +-------------------------+
           EdgeSwitch 8      |
+----------------------------|--+
|                            v  |
|           +-+      +-+        |
+-------------|------|----------+
              |      |
     OdroidC2 |      | AP-AC-Lite
  +-----------v-+   +v------------+
  |             |   |             |
  |             |   |             |
  +-------------+   +-------------+

our layer 2 and 3 topology:

We will utilize Virtual LANs via 802.1Q to segment our home and guest networks. We will also create a sub-network on the interface of our router as a diagnostic backup for whenever something might go wrong, allowing you to plug right into the router to diagnose issues.

- 172.16.16.0/24   :   DIAG    : diagnostic network on eth4 of router
- 172.16.0.0/24    :   HOME    : where we want to hang out with all of our protected stuff
- 172.16.1.0/24    :   ILPR    : where we want guests to go when they authenticate to wireless

For 172.16.0.0/24 and 172.16.1.0/24 we will use VLAN ID 7 and VLAN ID 14 respectively. I’ve chosen these arbitrarily, but you’ll see them consistently throughout the write up, so if you want to change because you have numbers you like better… make sure you’re consistent.

our final prep:

  • wire everything up as it should be
  • hard reset the devices, ensure that you’ve not affected some configuration change on them before beginning
  • update the firmware of both the router and switch, reboot them

Routing #

We’re going to get away from the GUI as soon as we can because there are a lot of cases where the GUI presents information in a way that isn’t clear, or actually sets values on the system in a way we do not want. This is much more prevalent in the EdgeSwitch, but it is preferred to learn what’s under the hood on both EdgeSwith and EdgeRouter.

  • Get connected (eth0) and use the GUI to set up the settings tab:
hostname: router
timezone: Americas:United States:Eastern (or wherever you are)
nameserver: 127.0.0.1
domainname: your.domain.name
ntp: enabled
ssh: enabled, port 8888
ubntdiscovery: disabled
  • Use the GUI to make changes to the interfaces on the main screen:
eth0 rename to WAN, leave alone for now (must configure DIAG)
eth1 disable
eth2 disable
eth3 disable
eth4 rename to DIAG, set static address of 172.16.16.1/24
eth5 rename to TRNK
  • Now move yourself over to eth4, set a static address of 172.16.16.2, and get back into the router at 172.16.16.1. Set eth0 to DHCP, and if your modem is functioning you should now see it pull an address.
  • Now set up a username for yourself, log out, log back in, delete the ubnt user.
  • Now scp your public key into the router: scp -P 8888 key.pub 172.16.16.1:key.pub
  • Now ssh into the router and we will begin running through our configuration. First we load the key-file.
ssh -p 8888 <user>@172.16.16.1
configure
loadkey <user> key.pub
save

You should exit back out and ssh back in to verify the key load works.

Once back in it is always nice to set a cool login banner, generate them here:

set system login banner pre-login "\n\n           ___  __     __           ___  ___\n          |__  |__) | |__) |__| \ /  |  |__\n          |___ |    | |    |  |  |   |  |___\n\n                [email protected]\n\n"
commit;save;

Which results in:


           ___  __     __           ___  ___
          |__  |__) | |__) |__| \ /  |  |__
          |___ |    | |    |  |  |   |  |___

                [email protected]

We’ll now do several things at once, and I’ll explain them below:

configure
set service dhcp-server use-dnsmasq enable
set service dns forwarding name-server 1.1.1.1
set service dns forwarding name-server 1.0.0.1
set service dns forwarding cache-size 512
set service dhcp-server shared-network-name DIAG authoritative enable
set service dhcp-server shared-network-name DIAG subnet 172.16.16.0/24 default-router 172.16.16.1
set service dhcp-server shared-network-name DIAG subnet 172.16.16.0/24 dns-server 172.16.16.1
set service dhcp-server shared-network-name DIAG subnet 172.16.16.0/24 domain-name diag.domain.name
set service dhcp-server shared-network-name DIAG subnet 172.16.16.0/24 lease 360
set service dhcp-server shared-network-name DIAG subnet 172.16.16.0/24 start 172.16.16.10
set service dhcp-server shared-network-name DIAG subnet 172.16.16.0/24 start 172.16.16.10 stop 172.16.16.20
set service dns forwarding listen-on eth4
set interfaces ethernet eth5 vif 7 address 172.16.0.1/24
set interfaces ethernet eth5 vif 7 description HOME
set service dhcp-server shared-network-name HOME authoritative enable
set service dhcp-server shared-network-name HOME subnet 172.16.0.0/24
set service dhcp-server shared-network-name HOME subnet 172.16.0.0/24 default-router 172.16.0.1
set service dhcp-server shared-network-name HOME subnet 172.16.0.0/24 dns-server 172.16.0.1
set service dhcp-server shared-network-name HOME subnet 172.16.0.0/24 domain-name home.domain.name
set service dhcp-server shared-network-name HOME subnet 172.16.0.0/24 lease 86400
set service dhcp-server shared-network-name HOME subnet 172.16.0.0/24 start 172.16.0.100
set service dhcp-server shared-network-name HOME subnet 172.16.0.0/24 start 172.16.0.100 stop 172.16.0.200
set service dns forwarding listen-on eth5.7
set interfaces ethernet eth5 vif 14 address 172.16.1.1/24
set interfaces ethernet eth5 vif 14 description ILPR
set service dhcp-server shared-network-name ILPR authoritative enable
set service dhcp-server shared-network-name ILPR subnet 172.16.1.0/24
set service dhcp-server shared-network-name ILPR subnet 172.16.1.0/24 default-router 172.16.1.1
set service dhcp-server shared-network-name ILPR subnet 172.16.1.0/24 dns-server 172.16.1.1
set service dhcp-server shared-network-name ILPR subnet 172.16.1.0/24 domain-name guest.domain.name
set service dhcp-server shared-network-name ILPR subnet 172.16.1.0/24 lease 360
set service dhcp-server shared-network-name ILPR subnet 172.16.1.0/24 start 172.16.1.100
set service dhcp-server shared-network-name ILPR subnet 172.16.1.0/24 start 172.16.1.100 stop 172.16.1.110
set service dns forwarding listen-on eth5.14
commit;save;

In the above we:

  • set the DHCP server to use dnsmasq, which has more features than the default ISC implementation
  • set our DNS forwarders to Cloudflare DNS at 1.1.1.1 and 1.0.0.1 (alternatively you could set it to use opendns, quad9, or another resolver you prefer (maybe run your own)).
  • created a DHCP server called DIAG
  • set our dns forwarder to listen on the interface we want DIAG to be a part of
  • created a VLAN on eth5 with the VLAN ID 7, and assigned an address in our HOME sub-network
  • created a DHCP server called HOME
  • created a VLAN on eth5 with the VLAN ID 14, and assigned an address in our ILPR sub-network
  • created a DHCP server called ILPR

Now that our addresses are in place we are going to restrict the GUI and ssh to listen only to the DIAG and HOME sub-networks:

configure
set service ssh disable-password-authentication
set service ssh listen-address 172.16.16.1
set service ssh listen-address 172.16.0.1
set service gui listen-address 172.16.16.1
set service ssh listen-address 172.16.0.1
set service gui older-ciphers disable
commit;save;

We now set up our NAT masquerade

configure
set service nat rule 5000 outbound-interface eth0
set service nat rule 5000 type masquerade
commit;save;

With that we’re pretty much ready to connect devices to all of our LANs, but we have no firewall in place. So let’s build a firewall to protect us from WAN connections:

configure
set firewall name WAN_IN default-action drop
set firewall name WAN_IN description "packets from WAN through router to LAN"
set firewall name WAN_IN enable-default-log
set firewall name WAN_IN rule 1 action accept
set firewall name WAN_IN rule 1 description "allow established sessions"
set firewall name WAN_IN rule 1 log disable
set firewall name WAN_IN rule 1 protocol all
set firewall name WAN_IN rule 1 state established enable
set firewall name WAN_IN rule 1 state invalid disable
set firewall name WAN_IN rule 1 state new disable
set firewall name WAN_IN rule 1 state related enable
set firewall name WAN_IN rule 2 action drop
set firewall name WAN_IN rule 2 description "drop invalid state"
set firewall name WAN_IN rule 2 log enable
set firewall name WAN_IN rule 2 protocol all
set firewall name WAN_IN rule 2 state invalid enable
set firewall name WAN_LOCAL default-action drop
set firewall name WAN_LOCAL description "packets from WAN to router"
set firewall name WAN_LOCAL enable-default-log
set firewall name WAN_LOCAL rule 1 action accept
set firewall name WAN_LOCAL rule 1 description "allow established sessions"
set firewall name WAN_LOCAL rule 1 log disable
set firewall name WAN_LOCAL rule 1 protocol all
set firewall name WAN_LOCAL rule 1 state established enable
set firewall name WAN_LOCAL rule 1 state invalid disable
set firewall name WAN_LOCAL rule 1 state new disable
set firewall name WAN_LOCAL rule 1 state related enable
set firewall name WAN_LOCAL rule 2 action drop
set firewall name WAN_LOCAL rule 2 description "drop invalid state"
set firewall name WAN_LOCAL rule 2 log enable
set firewall name WAN_LOCAL rule 2 protocol all
set firewall name WAN_LOCAL rule 2 state invalid enable
set interfaces ethernet eth0 firewall in name WAN_IN
set interfaces ethernet eth0 firewall local name WAN_LOCAL
commit;save;

You’ll notice that at the end we applied these two new rules WAN_IN and WAN_LOCAL to our eth0 interfaces. To understand the IN and LOCAL directives you’d need to read up on the evaluation chain within iptables, or documentation straight from Ubiquiti, but I’ll try with an explanation here:

  • WAN_IN: traffic from the internet, through the router, and onward to the LAN.
  • WAN_OUT: traffic that has been forwarded through the router and is about to exit the interface
  • WAN_LOCAL: traffic destined for the router (e.g. ssh, or the gui)

I’ve spent a lot of time reading about these ideas, this was another good post about the evaluation order and firewall composition. Above is something fairly simple, but you can get quite complex with what you do in these firewalls, and the order in which you evaluate things can have tremendous performance impact.

Although the edgeswitch line claims to have layer3 switching the limitations can be hit pretty quickly. We’ll instead route the traffic between vlans through the router.

The most efficient way to construct rules is to always drop packets as close to their origination as possible. We will build our inter-vlan routing rules based on this principle. Our objective is to:

  • allow HOME to get out to the internet
  • allow HOME to get into the ILPR subnet
  • allow ILPR to get out to the ineternet
  • not-allow ILPR to get into the HOME subnet

We’ll start with a chain for HOME

configure
set firewall name HOME_IN default-action reject
set firewall name HOME_IN description 'IN from HOME, VLAN 7, 172.16.0.0/24'
set firewall name HOME_IN rule 10 action accept
set firewall name HOME_IN rule 10 description 'allow related and established'
set firewall name HOME_IN rule 10 protocol all
set firewall name HOME_IN rule 10 state established enable
set firewall name HOME_IN rule 10 state related enable
set firewall name HOME_IN rule 100 action accept
set firewall name HOME_IN rule 100 description 'allow initiation to any of the local networks'
set firewall name HOME_IN rule 100 destination address 172.16.0.0/12
set firewall name HOME_IN rule 100 source address 172.16.0.0/24
set firewall name HOME_IN rule 200 action accept
set firewall name HOME_IN rule 200 description 'allow all outbound activity'
set firewall name HOME_IN rule 200 destination address !172.16.0.0/12
set firewall name HOME_IN rule 200 source address 172.16.0.0/24
set interfaces ethernet eth5 vif 7 firewall in name HOME_IN

This chain:

  • rejects as a default action
  • accepts related and established (rule 10)
  • accepts 172.16.0.0/24 initiating to anything within 172.16.0.0/12 (rule 100)
  • accepts all traffic not bound to 172.16.0.0/12 (rule 200)

The last rule (rule 200) is a negation match !172.16.0.0/12, which is a hack to match all packets that are destined to not our local networks, which should, in essence, be the internet.

We’ll then build a chain for ILPR:

set firewall name ILPR_IN default-action reject
set firewall name ILPR_IN description 'IN from ILPR, VLAN 14, 172.16.1.0/24'
set firewall name ILPR_IN rule 10 action accept
set firewall name ILPR_IN rule 10 description 'allow related and established'
set firewall name ILPR_IN rule 10 protocol all
set firewall name ILPR_IN rule 10 state established enable
set firewall name ILPR_IN rule 10 state related enable
set firewall name ILPR_IN rule 200 action accept
set firewall name ILPR_IN rule 200 description 'allow all outbound activity'
set firewall name ILPR_IN rule 200 destination address !172.16.0.0/12
set firewall name ILPR_IN rule 200 source address 172.16.1.0/24
set interfaces ethernet eth5 vif 14 firewall in name ILPR_IN

This chain:

  • rejects as a default action
  • accepts related and established (rule 10)
  • accepts all traffic not bound to 172.16.0.0/12 (rule 200)

It might be considered an overlook to not create a DIAG_IN chain, but we’re relying on that interface in the event that things go truely wrong so convoluting it would be a mistake.

We will force both of our sub-networks to use our internal DNS via a NAT destination rewrite:

set service nat rule 7 description "captive DNS for HOME vlan"
set service nat rule 7 destination port 53
set service nat rule 7 inbound-interface eth5.7
set service nat rule 7 inside-address address 172.16.0.1
set service nat rule 7 protocol tcp_udp
set service nat rule 7 type destination
set service nat rule 14 description "captive DNS for ILPR vlan"
set service nat rule 14 destination port 53
set service nat rule 14 inbound-interface eth5.14
set service nat rule 14 inside-address address 172.16.1.1
set service nat rule 14 protocol tcp_udp
set service nat rule 14 type destination

We will also do some level of “protection” for the router on from ILPR via the LOCAL directive:

set firewall name ILPR_LOCAL description "rules for traffic LOCAL from VLANs"
set firewall name ILPR_LOCAL default-action drop
set firewall name ILPR_LOCAL rule 1 action accept
set firewall name ILPR_LOCAL rule 1 description "allow related and established"
set firewall name ILPR_LOCAL rule 1 log disable
set firewall name ILPR_LOCAL rule 1 protocol all
set firewall name ILPR_LOCAL rule 1 state established enable
set firewall name ILPR_LOCAL rule 1 state related enable
set firewall name ILPR_LOCAL rule 2 action accept
set firewall name ILPR_LOCAL rule 2 description "allow DNS"
set firewall name ILPR_LOCAL rule 2 log disable
set firewall name ILPR_LOCAL rule 2 protocol tcp_udp
set firewall name ILPR_LOCAL rule 2 destination port 53
set firewall name ILPR_LOCAL rule 3 action accept
set firewall name ILPR_LOCAL rule 3 description "allow DHCP"
set firewall name ILPR_LOCAL rule 3 log disable
set firewall name ILPR_LOCAL rule 3 protocol udp
set firewall name ILPR_LOCAL rule 3 destination port 67
set interfaces ethernet eth5 vif 14 firewall local name ILPR_LOCAL

Note if you’d apply this ILPR_LOCAL policy to your HOME you’d no longer be able to use ssh with the router. You’d need to create a separate HOME_LOCAL type chain for HOME if you wanted to get really restrictive, in this case I’m assuming you trust those who are allowed into the HOME sub-network. We’ve written ILPR_LOCAL so that you can apply it to multiple networks that can be treated similarly to ILPR.

Switching #

Remember before that we have two VLANs that we need switched, for HOME and ILPR we’ll need VLAN ID 7 and VLAN ID 14 respectively. The switches are more fickle devices initially than the routers. You will need to do a bootstrap sequence by plugging directly into the switch and using the GUI to coax it onto the management VLAN:

  • connect your machine to any port of the switch, I typically choose eth0
  • assign a static address of 192.168.1.5
  • navigate to 192.168.1.2 via your browser
  • add the VLAN ID to the VLAN database that you want to use for management, in this case 7.
  • above the VLAN database you’ll see trunking, select eth9 as a trunk
  • navigate to system->connectivity->ipv4 and set the management VLAN to 7

Once you click submit on that management VLAN the device will cease to respond to you, it is no longer listening for management control via the default VLAN, which on EdgeSwitch is 1. You will now need to plug into the diagnostic port that we set up earlier, eth4, from here you should get DHCP‘ed an address in the 172.16.16.0/24 sub-network, and then log into the router GUI (as GUI is easier for examining DHCP leases).

Once in the router, navigate to services, then select the HOME DHCP server and view leases. You should see a new lease from the EdgeSwitch. Now you should be able to navigate to that new address and save the configuration of the switch (as its been in unsaved mode since we made our changes). In the router you should static map the switch to an address that you’d like it to use. Some will say you should static your devices from their own configuration, but if you ever want to move devices it’s easier to do that from a central location so I personally favor using DHCP. You should then log into the gui and save the configuration so that the switch will continue to use 7 as the management vlan.

Once rebooted, it should go to the new address, then you should log in do the final preparation work to take it over via ssh:

  • create yourself a user account
  • navigate to system->management->ssh and generate both ssh keys, turn off v1, and set the port you’d like to use

Now we can ssh into the switch, we’ll assume that its at 172.16.0.2 with port 8888, we’ll first take back over the VLAN database:

ssh -p 8888 172.16.0.2
en
hostname 172.16.0.2
no network ipv6 enable
vlan database
vlan 7,14
vlan name 7 "HOME"
vlan name 14 "ILPR"
show vlan
configure
no username "ubnt"
spanning-tree mst priority 0 0
interface 0/1-0/10
vlan participation auto 7
interface 3/1-3/6
vlan participation auto 7
interface 0/9
description 'trunk to router'
switchport mode trunk
switchport trunk allowed vlan 7,14
exit
exit
write memory

This is doing several key things:

  • setting the hostname, and prompt, to 172.16.0.2 so that you know what device you’re in. This becomes important if you add several more switches.
  • setting ipv6 off, as we’re only using ipv4 in this design
  • setting up 7,14 in your vlan database and giving them descriptive names
  • disabling the default ubnt account
  • setting this switch as the stp root bridge. This becomes important if you add several more switches.
  • cleaning up the vlan notation that the gui set when we added vlan 7 in the initial bootstrapping (for both interfaces and link aggregate channels).
  • setting the one arm link to the router as a trunk and only allowing vlan 7,14 to pass on the trunk.

We’ll now set up the HOME ports as well as our wireless port:

exit
interface 0/10
shutdown
exit
interface 0/8
description 'wireless'
switchport mode trunk
switchport trunk native vlan 7
switchport trunk allowed vlan 7,14
exit
interface 0/1-0/7
description 'participating in HOME'
switchport mode access
switchport access vlan 7
spanning-tree edgeport
spanning-tree bpdufilter
port-security
port-security max-dynamic 1
exit
exit
write memory

This does the following:

  • shuts off port 0/9, which you’re not currently using
  • sets up the wireless interface as a trunk, sets the native vlan (which is important for your access point to be managed on your HOME network)
  • sets the switchport mode to access and the access vlan to HOME on interfaces 0/1-0/7
  • sets up spanning tree protections on those access ports, more discussion here and here
  • sets up port security for only a single MAC on each port

At this time you should be able to plug into port 0/1 through 0/7 and pull an address in the 172.16.0.0/24 sub-network.

Wireless #

With wireless I’ll assume you can run the software from your management machine, an alternative is to purchase a Unifi Cloud Key or run the Unifi software on your Odroid.

If you run the interface from a machine in the HOME sub-network you will see your wireless access point listed under devices. You can then select to provision the device.

All of the necessary settings, for now, are located in the Settings->Wireless Networks section of the interface. You’ll want to create a HOME and GUEST network, using whatever names you like. Just make sure that you set the VLAN ID properly in the advanced settings.

You should now connect to each wireless network and observe that you get DHCP‘ed into the proper sub-network. When you are in the GUEST sub-network you should try to see if you can navigate to something in the HOME sub-network to verify that your router is preventing inter-vlan routing as we designed it.

Jump Host #

We have an odroidc2 running archlinuxarm at 172.16.0.8. We need to modify the router to forward the port to this host:

ssh -p 8888 172.16.0.1
configure
set port-forward auto-firewall enable
set port-forward harpin-nat enable
set port-forward lan-interface eth5.7
set port forward rule 1 description 'arbiter'
set port-forward rule 1 forward-to address 172.16.0.8
set port-forward rule 1 forward-to port 8888
set port-forward rule 1 original-port 8888
set port-forward rule 1 protocol tcp
set port-forward wan-interface eth0
commit;save;

Ideally you would never punch a hole in your firewall without having a known source. If you had a server set up in the cloud with a static address you could chain Jump Hosts and set a limitation of source on your port forward.

Now you should be able to utilize this host as a Jump Host, which I write about here. You can do a simple VPN over ssh with sshuttle.