Juniper Host Path Congestion

Recently ran into an issue on an ACX evo platform that caused random routing protocols to intermittently drop.

We found that the DDoS (quick side tangent on why there are a number of other possible names for this feature, that DDoS does not feel like the right fit) thresholds were increased well past default. After some time with JTAC debugging, we found that the below commands were helpful in identifying host-path congestion which was the ultimate culprit for our issue.

Some level of dropped packets may be normal, especially with reconvergence.

root@redacted> start shell pfe network fpc0

root@redacted:pfe> show host-path ddos all-policers
Host Path Ddos Policers:

Group           Policer         Id     BWidth  Burst  P  Q  Ena Rcvd Packets  Drops         Rate(pps) States Scfd  Flows
ppp             aggregate       0x0400 4000    1024   1  -  on  0             0             0         ok     off   0
ppp             unclassified    0x0401 500     512    0  2  on  0             0             0         ok     off   0
ppp             lcp             0x0402 3000    1024   1  3  on  0             0             0         ok     off   0
ppp             authentication  0x0403 1000    512    1  3  on  0             0             0         ok     off   0
ppp             ipcp            0x0404 1000    512    2  3  on  0             0             0         ok     off   0


root@redacted:pfe> show evo-pfemand host pkt-stats

------ ------ ------------- ---------  --------------- --------------- ----------- --------------- ---------------------------
UNIT   CMICQ     Pkt        Pkt_Rate        Bytes       Drp Pkts       DrpPktsRate    DrpBytes               Proto(s)
------ ------ ------------- ---------  --------------- --------------- ----------- --------------- ---------------------------
0      200    237033        28         31445861        0               0           0               recv-nh
0      201    882           0          56448           0               0           0               arp
0      202    8610          0          2853590         0               0           0               l2hi-pri/lacp, lldp, lfm, pvstp, stp, esmc
0      203    1290357       86         685037558       0               0           0               l3hi-pri/bgp,isis,ldp,ospf,rip,rsvp
0      204    32373         0          6359115         0               0           0               Mcast/igmp,pim
0      205    0             0          0               0               0           0               l2pt
0      206    28663         0          38877487        2977340         0           4055081029      unknown-mc
0      207    1054369       112        1415898840      306061012       30586       416845750015    ttl-excep
0      208    16            0          1150            0               0           0               resolve
0      209    0             0          0               0               0           0               bcast
0      210    0             0          0               0               0           0               l2pt-rw-uni
0      211    14310         0          1246476         0               0           0               l3low-pri/nd
0      212    0             0          0               0               0           0               local-nh
0      213    606228        66         55122496        0               0           0               rxq-bfd,bfdv6
0      214    0             0          0               0               0           0               rxq-cfm
0      215    0             0          0               0               0           0               rxq-vrrp
0      216    7223          0          1438000         19158           0           3979939         nh-reject
0      217    0             0          0               0               0           0               CpuMtuExpCore0
0      218    0             0          0               0               0           0               CpuMtuExpCore1
0      219    0             0          0               0               0           0               sample-source
0      220    0             0          0               0               0           0               sample-jflow
0      221    5             0          792             0               0           0               CpuDebug
0      222    2116          0          169461          0               0           0               CpuIpOption
0      223    0             0          0               0               0           0               netrounds
0      224    291           0          29274           0               0           0               rxq-dhcp
0      225    0             0          0               0               0           0               mpls-alert
0      226    168           0          221436          0               0           0               discard
0      227    0             0          0               0               0           0               overlay
0      228    0             0          0               0               0           0               ptp
0      229    108           0          14256           0               0           0               inline-bfd-punt
0      231    0             0          0               0               0           0               inline-ccm-rxq


root@redacted:pfe> show host-path ports

Name               State      Tx-Packets    Tx-Rate(pps)  Rx-Packets    Rx-Rate(pps)  Drop-Packets  Drop-Rate(pps)
===================================================================================================================
fp0                READY      1451694       249           2569855       399           3152          0
pktin0             READY      0             0             0             0             0             0
ptp0               READY      0             0             0             0             0             0
sflow0             READY      0             0             0             0             0             0
cp0                READY      1723430       286           1432375       246           0             0
jflow0             READY      0             0             0             0             0             0
am0                READY      132           0             1             0             0             0
ppm0               READY      464660        64            464719        64            0             0

root

Network engineer by night, deep sleeper by day.

Leave a Reply

Your email address will not be published. Required fields are marked *