Dear all,
We are testing SRX5600 performance with UDP packets which are used mostly in gaming application. The testing scenario is as the below:
We have simply two subnets and two group of three servers which are packet generators and receivers. We tried to send UDP packet at several rate from right side to left side and monitored SPU load, flow session and cp-session to examinate what is performance limits.
Startup configuration of the SRX is quite simple:
SRX5600> show chassis hardware models
Hardware inventory:
Item Version Part number Serial number FRU model number
Midplane REV 02 760-063936 ACRF5922 SRX5600X-CHAS
FPM Board REV 01 760-058098 CAFZ0086
PEM 0 Rev 04 740-034701 QCS15260904D SRX5600-PWR-2520-AC-S
PEM 1 Rev 04 740-034701 QCS15290901B SRX5600-PWR-2520-AC-S
PEM 2 Rev 04 740-034701 QCS1541090JW SRX5600-PWR-2520-AC-S
PEM 3 Rev 04 740-034701 QCS1541090LF SRX5600-PWR-2520-AC-S
Routing Engine 0 REV 02 740-056658 9013104720 SRX5K-RE-1800X4
CB 0 REV 03 750-062257 CAEW9777 SRX5K-SCB3
FPC 4 REV 24 750-061489 CAHV6708 SRX5K-SPC-4-15-320
CPU BUILTIN BUILTIN
FPC 5 REV 08 750-061262 CAFE1321 SRX5K-MPC
MIC 0 REV 07 750-049488 CAFF0743 SRX-MIC-10XG-SFPP
MIC 1 REV 10 750-049488 CAHD8073 SRX-MIC-10XG-SFPP
Fan Tray SRX5600-HC-FAN
SRX5600> show interfaces terse xe-5/0/0
Interface Admin Link Proto Local Remote
xe-5/0/0 up up
xe-5/0/0.2025 up up inet 61.28.240.1/24
multiservice
xe-5/0/0.2026 up up inet 61.28.241.1/24
multiservice
xe-5/0/0.32767 up up multiservice
SRX5600> show route
inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
61.28.240.0/24 *[Direct/0] 5d 02:08:09
> via xe-5/0/0.2025
61.28.240.1/32 *[Local/0] 5d 02:08:12
Local via xe-5/0/0.2025
61.28.241.0/24 *[Direct/0] 5d 02:08:09> via xe-5/0/0.2026
61.28.241.1/32 *[Local/0] 5d 02:08:12
Local via xe-5/0/0.2026
SRX5600> show security zones
Security zone: VLAN2025
Send reset for non-SYN session TCP packets: Off
Policy configurable: Yes
Interfaces bound: 1
Interfaces:
xe-5/0/0.2025
Security zone: VLAN2026
Send reset for non-SYN session TCP packets: Off
Policy configurable: Yes
Interfaces bound: 1
Interfaces:
xe-5/0/0.2026
Security zone: junos-host
Send reset for non-SYN session TCP packets: Off
Policy configurable: Yes
Interfaces bound: 0
Interfaces:
SRX5600> show security policies
Default policy: deny-all
From zone: VLAN2026, To zone: VLAN2025
Policy: T1, State: enabled, Index: 4, Scope Policy: 0, Sequence number: 1
Source addresses: 61.28.241.0/24
Destination addresses: 61.28.240.0/24
Applications: any
Action: permit
SRX5600> show security screen status
Screen status:
Screen trap interval : 2 second(s)
SRX5600> show security log
Security logging is disabled
TEST 1:
We tried to send a single UDP stream from 61.28.241.14:1210 to 61.28.240.10:2121, destination a real server existing in the left side of our scenario. We sent 10.000.000 packet in total at rate of 500K pps. Since there was a single session, that stream was proceeded by a single SPU, the SPU utilization was 30-40%. It seem to be fine!
SRX5600> show security monitoring
Flow session Flow session CP session CP session
FPC PIC CPU Mem current maximum current maximum
4 0 0 11 0 0 0 0
4 1 0 5 0 6291456 0 7549747
4 2 38 5 8 6291456 10 7549747
4 3 0 5 1 6291456 2 7549747
Total Sessions: 9 18874368 12 22649241
SRX5600> show security flow statistics
Flow Statistics of FPC4 PIC1:
Current sessions: 0
Packets forwarded: 8
Packets dropped: 4
Fragment packets: 0
Flow Statistics of FPC4 PIC2:
Current sessions: 2
Packets forwarded: 10793528
Packets dropped: 16
Fragment packets: 0
Flow Statistics of FPC4 PIC3:
Current sessions: 0
Packets forwarded: 34
Packets dropped: 17
Fragment packets: 0
Flow Statistics Summary:
System total valid sessions: 2
Packets forwarded: 10793570
Packets dropped: 37
Fragment packets: 0
SRX5600> show security flow cp-session
DCP Flow Sessions on FPC4 PIC0:
Total sessions: 0
DCP Flow Sessions on FPC4 PIC1:
Total sessions: 0
DCP Flow Sessions on FPC4 PIC2:
Session ID: 180862127, SPU: 18, Valid
In: 61.28.241.14/1210 --> 61.28.240.10/2121;udp,
Out: 61.28.240.10/2121 --> 61.28.241.14/1210;udp,
Session ID: 180908976, SPU: 18, Invalidated
In: 61.28.241.14/1210 --> 61.28.240.10/2121;udp,
Out: 61.28.240.10/2121 --> 61.28.241.14/1210;udp,
Session ID: 181263604, SPU: 18, Invalidated
In: 61.28.240.10/44812 --> 172.16.97.10/10051;tcp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Session ID: 181343296, SPU: 18, Invalidated
In: 61.28.241.14/1210 --> 61.28.240.10/2121;udp,
Out: 61.28.240.10/2121 --> 61.28.241.14/1210;udp,
Session ID: 181686285, SPU: 18, Invalidated
In: 61.28.241.14/1210 --> 61.28.240.10/2121;udp,
Out: 61.28.240.10/2121 --> 61.28.241.14/1210;udp,
Session ID: 181989157, SPU: 18, Invalidated
In: 61.28.241.14/1210 --> 61.28.240.10/2121;udp,
Out: 61.28.240.10/2121 --> 61.28.241.14/1210;udp,
Session ID: 182846046, SPU: 18, Valid
In: 61.28.240.7/46408 --> 61.28.240.1/22;tcp,
Out: 61.28.240.1/22 --> 61.28.240.7/46408;tcp,
Total sessions: 7
DCP Flow Sessions on FPC4 PIC3:
Session ID: 191449139, SPU: 19, Invalidated
In: 61.28.241.5/50684 --> 172.16.97.10/10051;tcp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Total sessions: 1
SRX5600> show security flow session
Flow Sessions on FPC4 PIC1:
Total sessions: 0
Flow Sessions on FPC4 PIC2:
Session ID: 181555326, Policy name: self-traffic-policy/1, Timeout: 1800, Valid
In: 61.28.240.7/46408 --> 61.28.240.1/22;tcp, If: xe-5/0/0.2025, Pkts: 2902, Bytes: 211617, CP Session ID: 182846046
Out: 61.28.240.1/22 --> 61.28.240.7/46408;tcp, If: .local..0, Pkts: 1866, Bytes: 382195, CP Session ID: 182846046
Session ID: 181556134, Policy name: T1/4, Timeout: 60, Valid
In: 61.28.241.14/1210 --> 61.28.240.10/2121;udp, If: xe-5/0/0.2026, Pkts: 316414, Bytes: 8859592, CP Session ID: 181975112
Out: 61.28.240.10/2121 --> 61.28.241.14/1210;udp, If: xe-5/0/0.2025, Pkts: 0, Bytes: 0, CP Session ID: 181975112
Total sessions: 2
Flow Sessions on FPC4 PIC3:
Total sessions: 0
SRX5600> show interfaces xe-5/0/0.2026 extensive
Logical interface xe-5/0/0.2026 (Index 71) (SNMP ifIndex 621) (Generation 136)
Flags: Up SNMP-Traps 0x4000 VLAN-Tag [ 0x8100.2026 ] Encapsulation: ENET2
Traffic statistics:
Input bytes : 280000612
Output bytes : 2098
Input packets: 10000010
Output packets: 31
Local statistics:
Input bytes : 192
Output bytes : 138
Input packets: 3
Output packets: 3
Transit statistics:
Input bytes : 0 0 bps
Output bytes : 0 0 bps
Input packets: 0 0 pps
Output packets: 0 0 pps
Security: Zone: VLAN2026
Flow Statistics :
Flow Input statistics :
Self packets : 0
ICMP packets : 0
VPN packets : 0
Multicast packets : 0TEST 2:
We sent similar stream in TEST 1 but to different destination. In this case, we sent to no existing IP/subnet: 61.28.242.10:2121. We expected the packets is dropped silently because of no route to that destination and didn't deplete SPU resources. But the result was different, SPU utilization got 99%!
SRX5600> show security monitoring
Flow session Flow session CP session CP session
FPC PIC CPU Mem current maximum current maximum
4 0 0 11 0 0 0 0
4 1 0 5 1 6291456 1 7549747
4 2 0 5 1 6291456 1 7549747
4 3 99 5 132060 6291456 1 7549747
Total Sessions: 132062 18874368 3 22649241
SRX5600> show security flow statistics
Flow Statistics of FPC4 PIC1:
Current sessions: 1
Packets forwarded: 84
Packets dropped: 2
Fragment packets: 0
Flow Statistics of FPC4 PIC2:
Current sessions: 1
Packets forwarded: 505
Packets dropped: 7
Fragment packets: 0
Flow Statistics of FPC4 PIC3:
Current sessions: 0
Packets forwarded: 9919326
Packets dropped: 459624
Fragment packets: 0
Flow Statistics Summary:
System total valid sessions: 2
Packets forwarded: 9919915
Packets dropped: 459633
Fragment packets: 0
SRX5600> show security flow cp-session
DCP Flow Sessions on FPC4 PIC0:
Total sessions: 0
DCP Flow Sessions on FPC4 PIC1:
Session ID: 170756285, SPU: 17, Invalidated
In: 61.28.241.14/33986 --> 172.16.97.6/10051;tcp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Session ID: 173520009, SPU: 17, Pending
In: 61.28.240.11/43024 --> 172.16.97.6/10051;tcp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Session ID: 174609306, SPU: 17, Invalidated
In: 61.28.240.11/42828 --> 172.16.97.10/10051;tcp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Session ID: 174973268, SPU: 17, Valid
In: 61.28.240.7/58102 --> 61.28.240.1/161;udp,
Out: 61.28.240.1/161 --> 61.28.240.7/58102;udp,
Total sessions: 4
DCP Flow Sessions on FPC4 PIC2:
Session ID: 181973328, SPU: 18, Invalidated
In: 61.28.240.7/41286 --> 122.201.9.245/10051;tcp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Session ID: 182189245, SPU: 18, Invalidated
In: 61.28.240.10/33380 --> 122.201.9.245/10051;tcp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Session ID: 182846046, SPU: 18, Valid
In: 61.28.240.7/46408 --> 61.28.240.1/22;tcp,
Out: 61.28.240.1/22 --> 61.28.240.7/46408;tcp,
Total sessions: 3
DCP Flow Sessions on FPC4 PIC3:
Session ID: 196756651, SPU: 19, Pending
In: 61.28.241.14/1210 --> 61.28.242.10/2121;udp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Session ID: 197093279, SPU: 19, Invalidated
In: 61.28.241.14/1210 --> 61.28.242.10/2121;udp,
Out: 0.0.0.0/0 --> 0.0.0.0/0;0,
Total sessions: 2
SRX5600> show security flow session
Flow Sessions on FPC4 PIC1:
Session ID: 171768053, Policy name: self-traffic-policy/1, Timeout: 60, Valid
In: 61.28.240.7/58102 --> 61.28.240.1/161;udp, If: xe-5/0/0.2025, Pkts: 652, Bytes: 49934, CP Session ID: 174973268
Out: 61.28.240.1/161 --> 61.28.240.7/58102;udp, If: .local..0, Pkts: 652, Bytes: 50687, CP Session ID: 174973268
Total sessions: 1
Flow Sessions on FPC4 PIC2:
Session ID: 181555326, Policy name: self-traffic-policy/1, Timeout: 1800, Valid
In: 61.28.240.7/46408 --> 61.28.240.1/22;tcp, If: xe-5/0/0.2025, Pkts: 8195, Bytes: 570557, CP Session ID: 182846046
Out: 61.28.240.1/22 --> 61.28.240.7/46408;tcp, If: .local..0, Pkts: 6388, Bytes: 1465119, CP Session ID: 182846046
Total sessions: 1
Flow Sessions on FPC4 PIC3:
Total sessions: 0
SRX5600> show interfaces xe-5/0/0.2026 extensive
Logical interface xe-5/0/0.2026 (Index 71) (SNMP ifIndex 621) (Generation 136)
Flags: Up SNMP-Traps 0x4000 VLAN-Tag [ 0x8100.2026 ] Encapsulation: ENET2
Traffic statistics:
Input bytes : 280000732
Output bytes : 5322
Input packets: 10000012
Output packets: 75
Local statistics:
Input bytes : 192
Output bytes : 138
Input packets: 3
Output packets: 3
Transit statistics:
Input bytes : 0 480 bps
Output bytes : 0 464 bps
Input packets: 0 1 pps
Output packets: 0 1 pps
Security: Zone: VLAN2026
Flow Statistics :
Flow Input statistics :
Self packets : 0
ICMP packets : 0
VPN packets : 0
Multicast packets : 0
Bytes permitted by policy : 0
Connections established : 0
Flow Output statistics:
Multicast packets : 0
Bytes permitted by policy : 0
Flow error statistics (Packets dropped due to):
Address spoofing: 0
Authentication failed: 0
Incoming NAT errors: 0
Invalid zone received packet: 0
Multiple user authentications: 0
Multiple incoming NAT: 0
No parent for a gate: 0
No one interested in self packets: 0
No minor session: 0
No more sessions: 0
No NAT gate: 0
No route present: 463406
No SA for incoming SPI: 0
No tunnel found: 0
No session for a gate: 0
No zone or NULL zone binding 0
Policy denied: 0
Security association not active: 0
TCP sequence number out of window: 0
Syn-attack protection: 0
User authentication errors: 0
We affraid there was something wrong in the routing config and force a discard default route but nothing better.
SRX5600> show route
inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 00:00:11
Discard
61.28.240.0/24 *[Direct/0] 5d 03:35:49
> via xe-5/0/0.2025
61.28.240.1/32 *[Local/0] 5d 03:35:52
Local via xe-5/0/0.2025
61.28.241.0/24 *[Direct/0] 5d 03:35:49> via xe-5/0/0.2026
61.28.241.1/32 *[Local/0] 5d 03:35:52
Local via xe-5/0/0.2026
SRX5600> show security monitoring
Flow session Flow session CP session CP session
FPC PIC CPU Mem current maximum current maximum
4 0 0 11 0 0 0 0
4 1 0 5 4 6291456 3 7549747
4 2 0 5 10 6291456 5 7549747
4 3 99 5 133320 6291456 5 7549747
Total Sessions: 133334 18874368 13 22649241
Could you please explain for me why SPU got 99% in TEST 2? Because 500K pps stream is far more under SPC II limit which spectify support 5Mpps/SPC ~ 1.25Mpps/SPU?
Thank you in advance,
Trung