In data center topologies, right cabling is a time-consuming endeavor and is error prone. Prescriptive Topology Manager (PTM) is a dynamic cabling verification tool to help detect and eliminate such errors. It takes a Graphviz-DOT specified network cabling plan (something many operators already generate), stored in a
topology.dot file, and couples it with runtime information derived from LLDP to verify that the cabling matches the specification. The check is performed on every link transition on each node in the network.
You can customize the
topology.dot file to control
ptmd at both the global/network level and the node/port level.
PTM runs as a daemon, named
For more information, see
- Topology verification using LLDP.
ptmdcreates a client connection to the LLDP daemon,
lldpd, and retrieves the neighbor relationship between the nodes/ports in the network and compares them against the prescribed topology specified in the
- Only physical interfaces, like swp1 or eth0, are currently supported. Cumulus Linux does not support specifying virtual interfaces like bonds or subinterfaces like eth0.200 in the topology file.
- Forwarding path failure detection using Bidirectional Forwarding Detection (BFD); however, demand mode is not supported. For more information on how BFD operates in Cumulus Linux, read the Bidirectional Forwarding Detection - BFD chapter and read
- Integration with FRRouting (PTM to FRRouting notification).
- Client management:
ptmdcreates an abstract named socket
/var/run/ptmd.socketon startup. Other applications can connect to this socket to receive notifications and send commands.
- Event notifications: see Scripts below.
- User configuration via a
topology.dotfile; see below.
ptmd verifies the physical network topology against a DOT-specified network graph file,
PTM supports undirected graphs.
ptmd connects to
lldpd, the LLDP daemon, over a Unix socket and retrieves the neighbor name and port information. It then compares the retrieved port information with the configuration information that it read from the topology file. If there is a match, then it is a PASS, else it is a FAIL.
Basic Topology Example
This is a basic example DOT file and its corresponding topology diagram. You should use the same
topology.dot file on all switches, and don't split the file per device; this allows for easy automation by pushing/pulling the same exact file on each device!
ptmd executes scripts at
/etc/ptm.d/if-topo-fail for each interface that goes through a change, running
if-topo-pass when an LLDP or BFD check passes and running
if-topo-fails when the check fails. The scripts receive an argument string that is the result of the
ptmctl command, described in the
ptmd commands section below.
You should modify these default scripts as needed.
You can configure
ptmd parameters in the topology file. The parameters are classified as host-only, global, per-port/node and templates.
Host-only parameters apply to the entire host on which PTM is running. You can include the
hostnametype host-only parameter, which specifies whether PTM should use only the host name (
hostname) or the fully-qualified domain name (
fqdn) while looking for the
self-node in the graph file. For example, in the graph file below, PTM will ignore the FQDN and only look for switch04, since that is the host name of the switch it's running on:
It’s a good idea to always wrap the hostname in double quotes, like "www.example.com". Otherwise,
ptmd can fail if you specify a fully-qualified domain name as the hostname and do not wrap it in double quotes.
Further, to avoid errors when starting the
ptmd process, make sure that
/etc/hostname both reflect the hostname you are using in the
However, in this next example, PTM will compare using the FQDN and look for switch05.cumulusnetworks.com, which is the FQDN of the switch it’s running on:
Global parameters apply to every port listed in the topology file. There are two global parameters: LLDP and BFD. LLDP is enabled by default; if no keyword is present, default values are used for all ports. However, BFD is disabled if no keyword is present, unless there is a per-port override configured. For example:
Per-port parameters provide finer-grained control at the port level. These parameters override any global or compiled defaults. For example:
Templates provide flexibility in choosing different parameter combinations and applying them to a given port. A template instructs
ptmd to reference a named parameter string instead of a default one. There are two parameter strings
bfdtmpl, which specifies a custom parameter tuple for BFD.
lldptmpl, which specifies a custom parameter tuple for LLDP.
In this template, LLDP1 and LLDP2 are templates for LLDP parameters while BFD1 and BFD2 are templates for BFD parameters.
Supported BFD and LLDP Parameters
ptmd supports the following BFD parameters:
upMinTx: the minimum transmit interval, which defaults to 300ms, specified in milliseconds.
requiredMinRx: the minimum interval between received BFD packets, which defaults to 300ms, specified in milliseconds.
detectMult: the detect multiplier, which defaults to 3, and can be any non-zero value.
afi: the address family to be supported for the edge. The address family must be one of the following:
- v4: BFD sessions will be built for only IPv4 connected peer. This is the default value.
- v6: BFD sessions will be built for only IPv6 connected peer.
- both: BFD sessions will be built for both IPv4 and IPv6 connected peers.
The following is an example of a topology with BFD applied at the port level:
ptmd supports the following LLDP parameters:
match_type, which defaults to the interface name (
ifname), but can accept a port description (
portdescr) instead if you want
lldpdto compare the topology against the port description instead of the interface name. You can set this parameter globally or at the per-port level.
match_hostname, which defaults to the host name (
hostname), but enables PTM to match the topology using the fully-qualified domain name (
fqdn) supplied by LLDP.
The following is an example of a topology with LLDP applied at the port level:
When you specify
ptmd will match the entire FQDN, like cumulus-2.domain.com in the example below. If you do not specify anything for
ptmd will match based on hostname only, like cumulus-3 below, and ignore the rest of the URL:
Bidirectional Forwarding Detection (BFD)
BFD provides low overhead and rapid detection of failures in the paths between two network devices. It provides a unified mechanism for link detection over all media and protocol layers. Use BFD to detect failures for IPv4 and IPv6 single or multihop paths between any two network devices, including unidirectional path failure detection. For information about configuring BFD using PTM, see the BFD topic.
Check Link State with FRRouting
The FRRouting routing suite enables additional checks to ensure that routing adjacencies are formed only on links that have connectivity conformant to the specification, as determined by
You only need to do this to check link state; you don't need to enable PTM to determine BFD status.
The check is enabled by default. Every interface has an implied
ptm-enable line in the configuration stanza in the interfaces file.
To disable the checks, delete the
ptm-enable parameter from the interface. For example:
If you need to re-enable PTM for that interface, run:
With PTM enabled on an interface, the
zebra daemon connects to
ptmd over a Unix socket. Any time there is a change of status for an interface,
ptmd sends notifications to
zebra. Zebra maintains a
ptm-status flag per interface and evaluates routing adjacency based on this flag. To check the per-interface
ptmd Service Commands
PTM sends client notifications in CSV format.
cumulus@switch:~$ sudo systemctl start|restart|force-reload ptmd.service: Starts or restarts the
ptmd service. The
topology.dot file must be present in order for the service to start.
cumulus@switch:~$ sudo systemctl reload ptmd.service: Instructs
ptmd to read the
topology.dot file again without restarting, applying the new configuration to the running state.
cumulus@switch:~$ sudo systemctl stop ptmd.service: Stops the
cumulus@switch:~$ sudo systemctl status ptmd.service: Retrieves the current running state of
ptmctl is a client of
ptmd; it retrieves the operational state of the ports configured on the switch and information about BFD sessions from
ptmctl parses the CSV notifications sent by
man ptmctl for more information.
The examples below contain the following keywords in the output of the cbl status column, which are described here:
|cbl status Keyword||Definition|
|pass||The interface is defined in the topology file, LLDP information is received on the interface, and the LLDP information for the interface matches the information in the topology file.|
|fail||The interface is defined in the topology file, LLDP information is received on the interface, and the LLDP information for the interface does not match the information in the topology file.|
The interface is defined in the topology file, but no LLDP information is received on the interface. The interface may be down or disconnected, or the neighbor is not sending LLDP packets.
The "N/A" and "fail" statuses may indicate a wiring problem to investigate.
The "N/A" status is not shown when using the
For basic output, use
ptmctl without any options:
For more detailed output, use the
To return information on active BFD sessions
ptmd is tracking, use the
To return LLDP information, use the
-l option. It returns only the active neighbors currently being tracked by
To return detailed information on active BFD sessions
ptmd is tracking, use the
-d options (results are for an IPv6-connected peer):
ptmctl Error Outputs
If there are errors in the topology file or there isn’t a session, PTM will return appropriate outputs. Typical error strings are:
If you encounter errors with the
topology.dot file, you can use
dot (included in the Graphviz package) to validate the syntax of the topology file.
By simply opening the topology file with Graphviz, you can ensure that it is readable and that the file format is correct.
If you edit
topology.dot file from a Windows system, be sure to double check the file formatting; there may be extra characters that keep the graph from working correctly.
Caveats and Errata
Prior to version 2.1, Cumulus Linux stored the
ptmdconfiguration files in
/etc/cumulus/ptm.d. When you upgrade to version 2.1 or later, all the existing
ptmdfiles are copied from their original location to
dpkg-oldextension, except for
topology.dot, which gets copied to
If you customized the
if-topo-failscripts, they are also copied to
dpkg-old, and you must modify them so they can parse the CSV output correctly.
if-topo-failscripts are available in
/etc/ptm.d. A sample
topology.dotfile is available in
When PTMD is incorrectly in a failure state and the Zebra interface is enabled, PIF BGP sessions are not establishing the route, but the subinterface on top of it does establish routes.
If the subinterface is configured on the physical interface and the physical interface is incorrectly marked as being in a PTM FAIL state, routes on the physical interface are not processed in Quagga, but the subinterface is working.