pdbmux backfill: consume the puppet lldp fact to populate NetBox switch/port cabling #7

New Issue

2026-08-08T18:33:52+10:00

unkinben commented

2026-08-08 18:33:52 +10:00

Problem

NetBox has no source for physical switch/port topology: which switch and which port each host is cabled to. It is the one reality dimension that exists nowhere else in the estate today (not in intent YAML, not in any other fact, not in DHCP/DNS). Without it, NetBox device interfaces have no cable/connection endpoint, so the "reality" side of the backfill (PR #6, issue #1) is incomplete for the physical fleet.

The upstream data now exists. puppet-prod PR (benvin/lldpd) runs lldpd on physical hosts and publishes a structured lldp custom fact. LLDP is the only topology source available, so the backfill should consume it.

`lldp` fact shape

PuppetDB serves the fact as a map keyed by local interface; only interfaces that have a neighbour appear. Real sample (prodnxsr0019, interface enp1s0):

{
  "enp1s0": {
    "neighbor_chassis_name":  "sg3429x-m2-02",
    "neighbor_chassis_mac":   "b8:fb:b3:a7:f9:5d",
    "neighbor_chassis_descr": "48-Port Gigabit Smart Managed Pro Switch",
    "neighbor_port_id":       "24",
    "neighbor_port_descr":    "two-gigabitEthernet 1/0/11",
    "vlan_id":                "201",
    "vlan_name":              "fabric_common_02"
  }
}

Proposal

Extend tools/backfill/ (pdbmux.go / reality.go / emit.go) to:

Read the lldp fact alongside the interface/inventory facts already consumed, decoding it into a map[localIface]Neighbor.
For each local interface with a neighbour, resolve the far end in NetBox:
- map neighbor_chassis_name (LLDP SysName, e.g. sg3429x-m2-02) to a NetBox device;
- map neighbor_port_descr (LLDP PortDescr, e.g. two-gigabitEthernet 1/0/11) to that device's interface, falling back to neighbor_port_id where PortDescr is not a usable interface name.
Create the connection on the reality side: a NetBox cable / interface-connection between the local device interface and the resolved switch interface (or, if switch-side interfaces are not yet modelled, record chassis name + port descr on a custom field / a LLDP: <sysname> <port> note so the data lands idempotently now and can be upgraded to a real cable later).
Keep it idempotent and non-destructive, matching the existing reality emit (skip when the switch device or its port is absent from NetBox rather than inventing it).

Prerequisite: the switches themselves must exist as NetBox devices with their ports for a true cable to be created; until then, the custom-field/note fallback is acceptable.

Acceptance

Backfill reads the lldp fact and, for a physical host with LLDP neighbours, creates the local-interface -> switch/port connection in NetBox (or records the neighbour on the fallback field when the switch/port is not yet modelled).
Re-running the backfill is idempotent (no duplicate cables, no churn).
Hosts with no lldp fact (VMs, hosts where lldpd is down) are skipped cleanly.
Unit test with a lldp fact fixture (using the prodnxsr0019 sample above) asserting the emitted connection/record.

Refs

Upstream fact + install: puppet-prod PR benvin/lldpd (profiles::lldpd on physicals + lldp fact).
Consumer: PR #6 (Backfill NetBox reality from pdbmux), issue #1 (Automate PuppetDB reality backfill into NetBox).
LLDP is the only switch/port topology source in the estate.

## Problem NetBox has no source for physical switch/port topology: which switch and which port each host is cabled to. It is the one reality dimension that exists nowhere else in the estate today (not in intent YAML, not in any other fact, not in DHCP/DNS). Without it, NetBox device interfaces have no cable/connection endpoint, so the "reality" side of the backfill (PR #6, issue #1) is incomplete for the physical fleet. The upstream data now exists. puppet-prod PR (`benvin/lldpd`) runs lldpd on physical hosts and publishes a structured `lldp` custom fact. LLDP is the only topology source available, so the backfill should consume it. ## `lldp` fact shape PuppetDB serves the fact as a map keyed by local interface; only interfaces that have a neighbour appear. Real sample (prodnxsr0019, interface `enp1s0`): ```json { "enp1s0": { "neighbor_chassis_name": "sg3429x-m2-02", "neighbor_chassis_mac": "b8:fb:b3:a7:f9:5d", "neighbor_chassis_descr": "48-Port Gigabit Smart Managed Pro Switch", "neighbor_port_id": "24", "neighbor_port_descr": "two-gigabitEthernet 1/0/11", "vlan_id": "201", "vlan_name": "fabric_common_02" } } ``` ## Proposal Extend `tools/backfill/` (pdbmux.go / reality.go / emit.go) to: 1. Read the `lldp` fact alongside the interface/inventory facts already consumed, decoding it into a `map[localIface]Neighbor`. 2. For each local interface with a neighbour, resolve the far end in NetBox: - map `neighbor_chassis_name` (LLDP SysName, e.g. `sg3429x-m2-02`) to a NetBox device; - map `neighbor_port_descr` (LLDP PortDescr, e.g. `two-gigabitEthernet 1/0/11`) to that device's interface, falling back to `neighbor_port_id` where PortDescr is not a usable interface name. 3. Create the connection on the reality side: a NetBox cable / interface-connection between the local device interface and the resolved switch interface (or, if switch-side interfaces are not yet modelled, record chassis name + port descr on a custom field / a `LLDP: <sysname> <port>` note so the data lands idempotently now and can be upgraded to a real cable later). 4. Keep it idempotent and non-destructive, matching the existing reality emit (skip when the switch device or its port is absent from NetBox rather than inventing it). Prerequisite: the switches themselves must exist as NetBox devices with their ports for a true cable to be created; until then, the custom-field/note fallback is acceptable. ## Acceptance - Backfill reads the `lldp` fact and, for a physical host with LLDP neighbours, creates the local-interface -> switch/port connection in NetBox (or records the neighbour on the fallback field when the switch/port is not yet modelled). - Re-running the backfill is idempotent (no duplicate cables, no churn). - Hosts with no `lldp` fact (VMs, hosts where lldpd is down) are skipped cleanly. - Unit test with a `lldp` fact fixture (using the prodnxsr0019 sample above) asserting the emitted connection/record. ## Refs - Upstream fact + install: puppet-prod PR `benvin/lldpd` (profiles::lldpd on physicals + `lldp` fact). - Consumer: PR #6 (Backfill NetBox reality from pdbmux), issue #1 (Automate PuppetDB reality backfill into NetBox). - LLDP is the only switch/port topology source in the estate.

unkinben referenced this issue from unkin/puppet-prod

2026-08-08 18:34:12 +10:00

lldpd: run on physicals and expose neighbour topology as the lldp fact #513

benvin referenced this issue from a commit

2026-08-08 18:41:51 +10:00

lldpd: run on physicals and expose neighbour topology as the lldp fact (#513)

Sign in to join this conversation.