Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: add BFD limitation on Spectrum

...

  • Without using cl-acltool, rules are not installed into hardware.

  • Running cl-acltool -i (the installation command) resets all rules and deletes anything that is not stored in /etc/cumulus/acl/policy.conf.

    For example, running the following command works:

    Code Block
    languagetext
    cumulus@switch:~$ sudo iptables -A INPUT -p icmp --icmp-type echo-request -j DROP

    And the rules appear when you run cl-acltool -L:

    Code Block
    languagetext
    cumulus@switch:~$ sudo cl-acltool -L ip
    -------------------------------
    Listing rules of type iptables:
    -------------------------------
    TABLE filter :
    Chain INPUT (policy ACCEPT 72 packets, 5236 bytes)
     pkts bytes target  prot opt in   out   source    destination
    
        0     0 DROP    icmp --  any  any   anywhere  anywhere      icmp echo-request

    However, running cl-acltool -i or reboot removes them. To ensure all rules that can be in hardware are hardware accelerated, place them in the /etc/cumulus/acl/policy.conf file, then run cl-acltool -i.

Mellanox Spectrum Hardware Limitations

Due to hardware limitations in the Spectrum ASIC, BFD policers are shared between all BFD-related control plane rules. Specifically the following default rules share the same policer in the 00control_plan.rules file:

Code Block
languagetext
[iptables]
-A $INGRESS_CHAIN -p udp --dport $BFD_ECHO_PORT -j POLICE --set-mode pkt --set-rate 2000 --set-burst 2000
-A $INGRESS_CHAIN -p udp --dport $BFD_PORT -j POLICE --set-mode pkt --set-rate 2000 --set-burst 2000
-A $INGRESS_CHAIN -p udp --dport $BFD_MH_PORT -j POLICE --set-mode pkt --set-rate 2000 --set-burst 2000

[ip6tables]
-A $INGRESS_CHAIN --in-interface $INGRESS_INTF -p udp --dport $BFD_ECHO_PORT -j POLICE --set-mode pkt --set-rate 2000 --set-burst 2000 --set-class 7
-A $INGRESS_CHAIN --in-interface $INGRESS_INTF -p udp --dport $BFD_PORT -j POLICE --set-mode pkt --set-rate 2000 --set-burst 2000 --set-class 7
-A $INGRESS_CHAIN --in-interface $INGRESS_INTF -p udp --dport $BFD_MH_PORT -j POLICE --set-mode pkt --set-rate 2000 --set-burst 2000 --set-class 7

To work around this limitation, set the rate and burst of all 6 of these rules to the same values, using the --set-rate and --set-burst options.

Where to Assign Rules

  • If a switch port is assigned to a bond, any egress rules must be assigned to the bond.
  • When using the OUTPUT chain, rules must be assigned to the source. For example, if a rule is assigned to the switch port in the direction of traffic but the source is a bridge (VLAN), the traffic is not affected by the rule and must be applied to the bridge.
  • If all transit traffic needs to have a rule applied, use the FORWARD chain, not the OUTPUT chain.

...