Network Automation¶

Registry-driven Ansible manages the MikroTik RouterOS 7.x fleet — colo core CCRs, metro/rooftop PEs, venue routers, the Starlink gateway and the CRS326 server-room switch fabrics. NetBox is the source of truth; the playbooks make the devices match it.

Repo: ansible/network/.

Workflow¶

Connection: community.routeros api_modify/api over api-ssl :8729 (TLS); a few backup/verify tasks use ssh :22.
Inventory: inventory/netbox.yml (netbox.netbox.nb_inventory) manages only devices with status active and platform routeros.
CI/CD: GitLab CI + ARA. Stages: lint → plan (--check --diff on MRs) → manual apply on main (protected-environment approval) → nightly drift (--check). A self-hosted runner sits inside the mgmt network.
Vault: device/service creds in ansible-vault (vault_ros_user/password, vault_pdns_api_key, optional vault_romon_secret, SNMP passwords). Never in NetBox, never in these docs. keys/ holds the CA and per-device certs (excluded).

The changed=3 convergence signature

Every play runs the backup role first, which always takes a backup (even under --check). So a healthy repeat run reports changed=3 per device — just the three backup tasks. The nightly drift job depends on this: anything above 3 is real drift.

Everyday commands¶

# validate the NetBox model before touching a device
python3 scripts/netbox_lint.py [DEVICE …]

# preview (read the REMOVED lines carefully)
ansible-playbook -i inventory/netbox.yml playbooks/colo-rr.yml --check --diff

# apply
ansible-playbook -i inventory/netbox.yml playbooks/colo-rr.yml
ansible-playbook -i inventory/netbox.yml playbooks/metro-pe.yml --limit cr-fenwick-1

# whole estate, dependency order
ansible-playbook -i inventory/netbox.yml site.yml

# inspect variable resolution without touching a device
ansible-inventory -i inventory/netbox.yml --host CR-COLO-1 --yaml

Playbooks¶

site.yml imports the five class playbooks in dependency order: colo-rr → metro-pe → venue-ce → starlink → crs-fabric.

Playbook	What it does
`colo-rr.yml`	Colo core/RR: backup, baseline, bgp_instance, transport, rr, static_routes, venue_handoff, vpls(hub); asserts BGP sessions up
`metro-pe.yml`	Metro/roof PE: + transport + vpls(spoke); asserts OSPF adjacency up
`venue-ce.yml`	Venue CE (`as=64512+venue_id`); asserts default route received
`starlink.yml`	Starlink gateway (`as=65502`)
`crs-fabric.yml`	CRS326 MLAG fabric (switch profile)
`bootstrap.yml`	Onboarding step 1 — create the vaulted `ansible` user over SSH
`certs.yml`	Onboarding step 2 — CA + per-device cert, enable api-ssl / www-ssl
`upgrade.yml`	RouterOS + RouterBOOT upgrade to an exact version; reboots only with `-e allow_reboot=true`; refuses downgrades
`netbox-setup.yml`	Creates the platform, roles, `ros_role` choice set and every custom field
`netbox-bgp-setup.yml`	Seeds `netbox_bgp` plugin defaults
`netbox-sync-facts.yml`	Reverse-sync live serial / ROS version / firmware / MACs into NetBox
`netbox-describe.yml`	Writes derived interface descriptions back into NetBox
`netbox-sync-l2vpn.yml`	Documents derived VPLS into NetBox L2VPN tables
`netbox-sync-venue-ipam.yml`	Documents each venue's IPAM (containers, prefixes, DHCP ranges)
`powerdns-sync.yml`	NetBox `dns_name` → PowerDNS forward A + reverse PTR
`oob-sync.yml`	NetBox `oob_ip` → PowerDNS `<device>.oob.pubinvest.co.uk`
`netbox-seed-demo.yml`	Seeds a small cabled demo topology into a test NetBox

Roles¶

Dependency order: backup · baseline · bgp_instance · transport · rr · static_routes · venue_handoff · vpls · venue_ce · starlink · crs_fabric. Each RouterOS config path has exactly one owning role, tagged ans:<role>, so authoritative cleanup is scoped and roles never clobber each other or hand-managed objects.

Role	Purpose
`backup`	Runs first; git-diffable `/export` + on-device binary restore point. Source of the `changed=3` signature
`baseline`	Identity, `ip service` hardening, NTP/DNS/SNMP(v3)/RoMON, address-lists, the canonical input firewall chain
`bgp_instance`	Creates/updates the BGP instance (as / router-id / cluster-id)
`transport`	Loopback, link addresses, OSPF area 0 + BFD, LDP, `mpls-mtu 1600` + `l2mtu 9000`, radio-mgmt VLANs, iBGP to the RRs
`rr`	The six community/policy filter chains, aggregate anchors, RR template + client connections, edge/Starlink eBGP
`static_routes`	Renders `config_context.static_routes` (+ blackhole anchors); scoped to `ans:static`
`venue_handoff`	PE side of a venue: `/31`, per-venue eBGP, from/to-venue filters, guest VLAN 20
`vpls`	BGP-signalled VPLS hub/spoke — the one untyped path, managed by teardown-by-tag then add
`venue_ce`	Venue L3 gateway, VLANs/DHCP, zone firewall, eBGP CE (no OSPF/LDP/VPLS)
`starlink`	DHCP WAN, core `/31`s, eBGP AS 65502, add-only touch of the app-mastered `starlink-announce`
`crs_fabric`	CRS326 MLAG pairs — jumbo 9092, peer-link bond, host LACP bonds, vlan-filtering bridge

House rules¶

No redistribute anywhere — everything is originated with network statements; the transport role asserts this live on every run.
Hands-off (never managed): /ip dhcp-server lease, the app-mastered starlink-announce and payment-prefixes lists, OPNsense, the OOB kit, and the 60 GHz dishes (modelled in NetBox for docs/monitoring only).
Authoritative-path pattern: handle_absent_entries: remove + ensure_order + handle_entries_content: remove_as_much_as_possible + restrict: scoping cleanup to owned values. The only real collection gap is routing bgp vpls (handled with the generic api + ans:vpls tag sweep).

VPLS numbering¶

Services are declared once in inventory/group_vars/all/vpls.yml; circuits derive from NetBox. Numbering is arithmetic and unique fleet-wide:

RT / VSI identity = 65500:(id_base + circuit_id) — 1000-block per service (2000 guest, 3000 pop, 4000 xc).
Route-distinguisher = <this PE loopback>:(id_base + circuit_id) — per-PE, so both colos advertise the same hub site-id with different RDs (multihoming).
Hub hand-off VLAN = (vlan_base or id_base) + circuit_id.
Spoke site-id = circuit_id (<1000); hub site-id = hub_site_id (≥1000, same on both colos → multihoming + DF election).

LibreNMS integration¶

The baseline role writes cable-graph descriptions as port comments (SNMP ifAlias: CORE::, UPLINK::, VENUE::<tri>, EBGP::), giving machine-parsable port categorisation with zero per-port effort. The same strings are written to NetBox by netbox-describe.yml, so NetBox, the device and LibreNMS stay identical.