Network¶
The Pub Invest corporate metro network: a fibre-and-60GHz metro linking eight hub sites to a central colocation facility, delivering corporate LAN, POS, staff and guest WiFi to the venues, plus the server-room fabric at the colo.
Two businesses, shared metal
Two distinct businesses share physical infrastructure but never share L3:
- Corporate metro (this documentation) —
10.0.0.0/8internally, guest WiFi on172.16.0.0/12. - Retail ISP / WAN business (separate, AS 204258,
185.109.40.0/22) — documented only where it touches the metro as an L2 circuit carrier. See WAN & Internet Edge.
They share only: the colo OOB network, the metro as an L2 carrier, and a common MikroTik CCR2004 spares pool.
The network at a glance¶
| Layer | Role | Hardware |
|---|---|---|
| Edge | Internet, NAT, central captive portal, server-room firewall | OPNsense HA pair (CARP), 8× 10G SFP+ each |
| Core (colo) | BGP route reflectors, MPLS PE, WAN hand-off, ECMP hub | 2× CCR2004-1G-12S+2XS (CR-COLO-01/02) |
| Metro | Per-hub aggregation, VPLS PE, venue hand-offs | CCR2004 per rack site; RB5009 on roof sites |
| Venue | L3 gateway, VLANs, DHCP, local firewall, guest → VPLS | RB5009UG+S+IN (1G) / CCR2004-16G-2S+ (10G) |
| Server room | Storage + VM switching | 2× CRS326 (storage MLAG) + 2× CRS326 (VM MLAG) |
| Backup WAN | Starlink failover for payments | RB4011 at colo, eBGP |
Design principles¶
- One protocol per job — OSPF for infra reachability, iBGP (route-reflected) for services, LDP for labels, BGP-signalled VPLS for L2 overlays. Static routes are banned in the core (except each router's blackhole anchor).
- Never redistribute — every prefix is originated explicitly with a
networkstatement and an output filter. - Templates, not snowflakes — three device classes (transport, venue CE, switch fabric), each built from composable roles.
- Address-lists are the policy API — membership, not rule edits, drives access.
- Config is code — NetBox + git are the source of truth; Ansible converges the fleet; devices are scraped back for drift.
- Fail toward backup — the default route is emergent: if the WAN dies the
edge withdraws
0.0.0.0/0fleet-wide, and payment traffic survives via Starlink.
In this section¶
| Page | What's in it |
|---|---|
| Topology & sites | Layered model, the eight hubs, per-hub redundancy, ring closure |
| IP addressing & VLANs | 10/8 plan, per-venue VLAN standard, loopback registry |
| Routing | OSPF, iBGP/route reflectors, MPLS/LDP, VPLS, venue eBGP, traffic engineering |
| WAN & Internet edge | Corporate edge, Starlink payments failover, retail ISP separation |
| Firewall (OPNsense) | Edge HA pair, zones, NAT, IDS/IPS, tuning |
| VPN (WireGuard) | Remote access design and migration |
| 60 GHz wireless | Backup radio links, MTU constraints, recovery |
| Server-room network | VLAN/zone plan, Ceph & VM fabrics, OOB |
| Physical layer | Fibre & patching, transceivers, Intel X710 NICs |
| IPAM (NetBox) | Source of truth, the network.yaml registry |
| Network automation | Ansible playbooks, roles, CI/CD |
| Device reference | Per-role config facts: colo-rr, metro-pe, venue-ce, starlink, crs-fabric, radios |
| Config templates | Worked RouterOS configs + raw .rsc files |
| Design (project docs) | Brief, discovery, target design, move plan, IaC plan |
| OPNsense GUI build runbook | Step-by-step GUI build of the edge HA pair |
| WAN / ISP business | AS 204258 target design and as-built |
Secrets are not in this portal
Private keys, WireGuard keys, radio PSKs, RADIUS/vault secrets and the like are
excluded from these docs. They live in the Ansible vault, gitignored
discovery/credentials.md, and keys/ in the automation repo. Where a
secret is relevant, the page names it and points at where it lives — it never
reproduces it.