Migrate edge from Ryan's basement to the cloud
For the past 3 years, Ryan has had an AT&T enterprise, and the network edge has been announced in his basement (called site "RET"). The contract is up, and Ryan will not be renewing. At this point, Ryan has setup gigabit fiber broadband at his home that he will be moving over to. The enterprise connection is still up, but he intends to only leave it up until the end of April 2022. This means that announcing the network needs to take place someplace else. Ryan did reach out to the PSDR group about opting in to their open peering policy, however no response was received. As a result, Ryan is suggesting an alternative that results in the least financial impact to MemHamWAN while maintaining the group's ability to exist independently.
Current state
Currently, MemHamWAN has a dedicated uplink radio at the LEB site that connects to a "uplink" node on Ryan's roof. We call this the RET site. At the RET site, on AT&T enterprise, Ryan hosts an enterprise edge router that has memhamwan's network space announced. This is where traffic for 44net addresses goes out and comes in to the network. The speed for this announcement is 10/10 Mbit. Ryan also has a 1/1 Gbit consumer broadband service now, but it is not presently part of the hamwan network.
Proposed Alternative
As this enterprise edge router will be going away, Ryan is proposing moving the data path to the Broadband service at his home, and introducing an edge router hosted in a VPS from vultr.com. This means there will be a tunnel over broadband from Ryan's home to the new edge router, and ultimately traffic for the 44net addresses will go out and come in through that VPS site.
Details of the changes are as follows:
- Ryan will continue to donate the use of his AS397968; this is valued at $150/yr
- Ryan will continue to offer an uplink node at his house to the network edge in order to offer a physical path to the internet
- Ryan is asking that MemHamWAN purchase a small VPS from Vultr, who permit BGP
- The recurring cost to hamwan would be $6/month, which provides a VPS that has 1 core, 1 GB of memory, and 1 TB of network transfer; 1 TB may sound low, but their policy is to bill whichever is highest of inbound vs outbound; so that almost certainly means that we'd be billed for the 1 TB inbound before filters
- To make network maintenance easier, Ryan is asking that HamWAN purchase a license for Mikrotik's "Cloud Hosted Router"; a P-1 license would permit 1 Gb traffic, which is plenty fast; this would be a one-time fee of $45 (wiki). The idea here is that then at Vultr, we would be running a mikrotik routeros device so that all of our configurations and management can be done using a similar pattern (vs needing to use a *nix package for managing routes, tunnels, etc).
With this change, there may be some increased latency. Management of the network should be simplified as well, as there will no longer be a privately-managed (read: ryan's home router) device in the critical path to the internet.
Proposed Charges to MemHamWAN
Recurring
- $6/month to Vultr; bandwidth overages would be billed at $0.01/GB
Fixed
- $45 for a MikroTik CHR P1 license, which would be transferable to any future CHR should we outgrow this solution
Implementation Plan
If this proposal is approved by the board, Ryan would suggest the following implementation plan
-
Setup the Vultr VPS; detailed instructions for this are provided on this blog post; this can be done by any netop; end with just getting an IPv4 interface setup using Vultr's IP and user accounts setup for netops; this would start the 60 day trial for the MirkroTik CHR P1 license -
Get a letter-of-authorization from AMPRnet for announcing 44.34.128.0/21 from Vultr via Ryan's AS; this will take an e-mail, and as Ryan is the holder of this allocation, it will need to come from anyone -
Submit Vultr's BGP setup form here, wait for permission -
Setup BGP peering with Vultr on the VPS, but prepend our route 3x so that it doesnt interrupt existing traffic flow:
/routing bgp instance
set default as=397968 out-filter=bgp-out router-id=vultrs-assigned-public-ip-goes-here
/routing bgp network
add network=44.34.128.0/21
/routing bgp peer
add name=vultr-ipv4 remote-address=169.254.169.254 remote-as=64515 multihop=yes ttl=2
add address-families=ipv6 name=vultr-ipv6 remote-address=2001:19f0:ffff::1 remote-as=64515 multihop=yes ttl=2
/routing filter
add action=accept chain=bgp-out prefix=44.34.128.0/21 set-bgp-prepend=3
add action=accept chain=default-in prefix=0.0.0.0/0
/ip route
add distance=1 dst-address=44.34.128.0/21 type=unreachable
... plus some sort of local subnet for the router itself as an address on an interface
-
Setup a tunnel between Ryan's uplink node and the Vultr VPS; confirm that traffic can flow; a protocol like SSTP may be preferred -
Prepend BGP at Ryan's edge router, remove the prepend to the Vultr router; this will take some time to globally take effect, but it will result in traffic flowing via the VPS and tunnel -
After at least a day, move Ryan's uplink node from his enterprise network over to his broadband network, remove the route and BGP announcement from Ryan's enterprise edge router -
Purchase the MikroTik CHR P1 license
Start to finish, this process would likely take 2 weeks.
I would appreciate feedback from @All members on this proposal.