3. Interfaces, VLANs & VIPs¶
Do this on both nodes (LAGGs and VLAN interfaces are per-node assignment, not synced). CARP VIPs (§3.4) are created on the primary and do sync — but you set each node's advskew locally.
3.1 LAGGs¶
Interfaces ▸ Other Types ▸ LAGG.
Server trunk — LACP to the VM MLAG pair¶
| Field | Value |
|---|---|
| Parent interfaces | the two ixl? server ports (one to each MLAG member) |
| Proto | LACP |
| LACP hash | L3 + L4 (matches MikroTik layer-3-and-4) |
| Fast timeout | on (matches RouterOS lacp-rate 1sec — both ends or neither) |
| MTU | 9000 |
This becomes lagg0 → assign as SRVTRUNK (page 2 §2.3). The MikroTik MLAG presents
one system-id across both switches, so OPNsense sees a single normal LACP partner.
40G upgrade path
§4 leaves 2 spare SFP+. If east-west/backup saturates 20G, add two more members (one to each MLAG switch) here — no other change.
pfSync bond — active-backup to the peer node¶
| Field | Value |
|---|---|
| Parent interfaces | the two igb? 1G RJ45 ports (direct Cat6 to the peer, no switch) |
| Proto | Failover |
| MTU | 1500 |
This becomes lagg1 → assign as PFSYNC. Failover, not LACP — pfSync is a single
UDP flow to a single peer IP, so LACP hashing would pin it to one member anyway and
buy nothing; failover gives clean redundancy without reordering state packets (§6).
The two RJ45s go straight into the peer node with no switch between them, so
PFSYNC is an isolated two-host segment. Give it a small dedicated /30 with no
gateway — e.g. FW-01 10.0.0.1/30, FW-02 10.0.0.2/30 — carrying only pfSync
state + XMLRPC config sync (page 4). It is never advertised, never routed,
and needs no firewall rules beyond the auto pfSync/sync pass.
3.2 Server-room VLANs¶
Interfaces ▸ Other Types ▸ VLAN. Create one VLAN per routed server zone on parent
SRVTRUNK, then Assignments to name each, then address the interface with the
gateway IP.
| VLAN tag | Assign as | Interface IP (this is the VIP target next) | MTU |
|---|---|---|---|
| 910 | VL_BACKUP |
node real IP in 10.128.10.0/24 (e.g. .2/.3) |
1500 |
| 930 | VL_TIER1 |
node real IP in 10.128.32.0/22 |
1500 |
| 931 | VL_TIER2 |
node real IP in 10.128.36.0/22 |
1500 |
| 932 | VL_DMZ |
node real IP in 10.128.40.0/24 |
1500 |
| 940 | VL_HYPMGMT |
node real IP in 10.128.44.0/24 |
1500 |
Give each VLAN interface a unique per-node real IP (.2 on FW-01, .3 on FW-02).
The .1 gateway is a CARP VIP (next section) — clients use .1, which floats.
Never create VLANs 920 / 921 / 922 here
Ceph public/cluster/iSCSI are L2-only, jumbo, no gateway anywhere (DESIGN §7.2). They live only on the storage fabric and hypervisor NICs. Putting them on OPNsense would break the storage design. They must not exist on this box.
3.3 Interface MTU summary¶
| Interface(s) | MTU | Why |
|---|---|---|
SRVTRUNK (lagg0, L2) |
9000 | jumbo headroom for the trunk |
| Routed VLAN interfaces | 1500 | routed traffic stays 1500 (§4) |
WAN, WAN2, COREA, COREB |
1500 | transit |
PFSYNC, OOB |
1500 | — |
3.4 CARP VIPs¶
Interfaces ▸ Virtual IPs ▸ Settings. Create on the primary; they sync to the
backup once HA is on (page 4). Set advskew per node: <CARP_ADVSKEW_PRIMARY> (0)
on FW-01, <CARP_ADVSKEW_BACKUP> (100) on FW-02 — same VHID + password both sides.
Gateway VIPs — one per routed VLAN¶
| VIP | Interface | Address | VHID | advskew |
|---|---|---|---|---|
| Backup GW | VL_BACKUP |
10.128.10.1/24 |
10 | per node |
| Tier-1 GW | VL_TIER1 |
10.128.32.1/22 |
30 | per node |
| Tier-2 GW | VL_TIER2 |
10.128.36.1/22 |
31 | per node |
| DMZ GW | VL_DMZ |
10.128.40.1/24 |
32 | per node |
| Hyp-mgmt GW | VL_HYPMGMT |
10.128.44.1/24 |
40 | per node |
Public VIPs — on a loopback/opt carrier¶
The <PUBLIC_BLOCK> addresses float as CARP VIPs, decoupled from the WAN /31 transit
(§5a). Create a loopback (Interfaces ▸ Other Types ▸ Loopback), assign it (e.g.
PUBVIP), and put the public CARP VIP(s) on it:
| VIP | Interface | Address | VHID | Purpose |
|---|---|---|---|---|
| Routing/anchor VIP | PUBVIP |
<PUBLIC_ROUTING_VIP>/32 |
50 | the VIP FRR failover tracks (§3); NAT + service anchor |
Add more public VIPs (inbound service / 1:1 targets) on PUBVIP as needed from
<PUBLIC_BLOCK> — see NAT.
The 'routing' VIP is the master election everything follows
FRR's CARP-failover tracks one VIP. Because every VIP shares advskew per node, tracking VHID 50 makes all of FRR (core + WAN) follow one election. Note which VIP you pick — page 5 §5.2 references it.