Regex not working for event splitting

sansri7680 — Mon, 28 Sep 2020 13:55:48 GMT

Sorry I am a newbie to Splunk and the question may sound silly but the splunk regex that I used to split events in the file doesn't work

props.conf
[3GPP]
BREAK_ONLY_BEFORE = ^Session-ID:\s
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true

The file looks like below. The Event split has to happen before the Session-ID field. Can someone help me please. I tried various combinations but all gives the same result and the split happens in a place where it shouldn't
Session-ID: 8:885 Username: 240010000099923@3glab.com
Callid: 02166055 IMSI/MSID: 240010000099923
ACSMgr Instance: 8 ACSMgr Card/Cpu: 16/0
SessMgr Instance: 8
Client-IP: 10.127.1.63
NAS-IP: 10.10.10.253
Access-NAS-IP(FA):
NAS-PORT: 0 NSAPI: 5
Acct-Session-ID: 0A0A0BFC02555693
NAS-ID: n/a
Access-NAS-ID(FA): n/a
3GPP2-BSID: n/a
Access-Correlation-ID(FA): n/a
3GPP2-Correlation-ID: n/a
MEID: n/a
Carrier-ID: n/a ESN: n/a
Uplink Bytes: 19570539 Downlink Bytes: 405939823
Uplink Packets: 360119 Downlink Packets: 709687
Injected Uplink Bytes: 0 Injected Downlink Bytes: 0
Injected Uplink Packets: 0 Injected Downlink Packets: 0
Buffered Uplink Packets: 0 Buffered Downlink Packets: 0
Buffered Uplink Bytes: 0 Buffered Downlink Bytes: 0
Uplink Packets in Buffer: 0 Uplink Bytes in Buffer: 0
Downlink Packets in Buffer: 0 Downlink Bytes in Buffer: 0
Buff Over-limit Uplink Pkts: 0 Buff Over-limit Uplink Bytes: 0
Buff Over-limit Downlink Pkts: 0 Buff Over-limit Downlink Bytes: 0
Processed Uplink Packets: 0 Processed Downlink Packets: 0
Dropped Uplink Packets: 0 Dropped Downlink Packets: 0
Uplink Out of Order Packets: 3 Downlink Out of Order Packets: 3865
Dyn FUI Redirected Flows: 0 Dyn FUI Discarded Pkts: 0
ITC Terminated Flows: 0 ITC Redirected Flows: 0
ITC Dropped Packets: 0 ITC ToS Remarked Packets: 0
Flow action Terminated Flows: 0
PP Flow action Terminated Flows: 0
CC Dropped Uplink Packets: 0 CC Dropped Uplink Bytes: 0
CC Dropped Downlink Packets: 0 CC Dropped Downlink Bytes: 0
NRUPC Req Made: 1 NRUPC Req Success: 1
NRUPC Req Failed: 0 NRUPC Req Time Out: 0
Current Readdressed Sessions: 0
Total Readdressed Uplink Pkts: 0
Total Readdressed Uplink Bytes: 0
Total Readdressed Downlink Pkts: 0
Total Readdressed Downlink Bytes: 0
Total Readdressing Failure: 0
Creation Time: Tuesday May 14 04:14:44 GMT 2013
Last Pkt Time: Tuesday May 14 05:40:04 GMT 2013
Duration: 01h:25m:20s
Active Charging Service name: ecs
Rule Base name: Rapids_Silver_Rule
Bandwidth Policy: n/a
FW-and-NAT Policy: n/a
NAT Policy: Not-required
TPO Policy: n/a
CF Policy ID: n/a
Old CF Policy ID: n/a
Dynamic Charging: Enabled
Dynamic Chrg Msg Received: 1 Rule Definitions Received: 5
Installs Received: 5 Removes Received: 0
Installs Succeeded: 5 Installs Failed: 0
Removes Succeeded: 0 Removes Failed: 0
Uplink Dynamic Rule Packets: 0 Uplink Dynamic Rule Bytes: 0
Downlink Dynamic Rule Packets: 0 Downlink Dynamic Rule Bytes: 0
Dynamic Charging Packet Drop statistics:
Bearer BW Limit Upl Pkts: 0 Bearer BW Limit Dnl Pkts: 0
Bearer BW Limit Upl Bytes: 0 Bearer BW Limit Dnl Bytes: 0
PCC Rule BW Limit Upl Pkts: 0 PCC Rule BW Limit Dnl Pkts: 0
PCC Rule BW Limit Upl Bytes: 0 PCC Rule BW Limit Dnl Bytes: 0
PCC Rule Gating Upl Pkts: 0 PCC Rule Gating Dnl Pkts: 0
PCC Rule Gating Upl Bytes: 0 PCC Rule Gating Dnl Bytes: 0
RuleMatch Fail Upl Pkts: 0 RuleMatch Fail Dnl Pkts: 0
RuleMatch Fail Upl Bytes: 0 RuleMatch Fail Dnl Bytes: 0
Credit-Control: Off
QoS Renegotiate Up: 0 QoS Renegotiate Dn: 0
Current TCP Proxy Flows: 0 Total TCP Proxy Flows: 0
TCP-proxy reset for non-SYN flows: 0
Current IP Flows: 128 Current ICMP Flows: 1
Current IPv6 Flows: 0 Current ICMPv6 Flows: 0
Current TCP Flows: 44 Current UDP Flows: 83
Current HTTP Flows: 0 Current HTTPS Flows: 0
Current FTP Flows: 0 Current POP3 Flows: 0
Current SMTP Flows: 0 Current SIP Flows: 0
Current RTSP Flows: 0 Current RTP Flows: 0
Current RTCP Flows: 0 Current IMAP Flows: 0
Current WSP-CO Flows: 0 Current WSP-CL Flows: 0
Current MMS Flows: 0 Current DNS Flows: 0
Current PPTP-GRE Flows: 0 Current PPTP Flows: 0
Current P2P Flows: 0 Current H323 Flows: 0
Current TFTP Flows: 0
Current UNKNOWN Flows: 127

CAE-Readdressing:
GET Requests redirected: 0
POST Requests redirected: 0
Other Requests redirected: 0
HTTP Responses redirected: 0
Requests having xheader inserted: 0
Total connect failed to video server: 0
Total Uplink Bytes: 0
Total Uplink Packets: 0
Total Downlink Bytes: 0
Total Downlink Bytes: 0

Transrating:
Total Transrated Video Connections: 0
Transrated Sorenson H263 Connections: 0
Transrated H264 Connections: 0
Failed Sorenson H263 Connections: 0
Failed H264 Connections: 0
Total Input Video Data Bytes: 0
SH263 Input Video Data Bytes: 0
H264 Input Video Data Bytes: 0
Total Output Video Data Bytes: 0
SH263 Output Video Data Bytes: 0
H264 Output Video Data Bytes: 0
Average Input Video Bit Rate: 0
SH263 Input Video Bit Rate: 0
H264 Input Video Bit Rate: 0
Average Output Video Bit Rate: 0
SH263 Output Video Bit Rate: 0
H264 Output Video Bit Rate: 0
Average Bit Rate Reduction: 0
SH263 Bit Rate Reduction: 0
H264 Bit Rate Reduction: 0
TCP-Proxy Session Stats: n/a

WiMAX Hotlining Status: n/a
Link Monitoring Average Throughput: 0 kbps
Link Monitoring Average RTT: 0 ms

Ruledef Name Pkts-Down Bytes-Down Pkts-Up Bytes-Up Hits

ip_any 709687 405939823 360119 19570539 1069806
Dynamic Charging Rule Name Statistics: n/a
Total Dynamic Rules: 5
Total Predefined Rules: 0
Total Firewall Predefined Rules: 0

Dynamic Charging Rule Definition(s) Configured:
Name Prior Content-Id Chrg-Type Rule Parameters

Gold_time1_3577508100 1 0 None Gate Status: Discard All
QoS Class Identifier: 5
ARP Priority Level: 1
Reporting Level: Rating Grp
Metering Method: Volume
Uplink MBR: 100000000
Downlink MBR: 200000000
Uplink GBR: 10000001
Downlink GBR: 20000001
Rule Activation Time:
Tuesday May 14 08:15:00 GMT 2013
Rule De-activation Time:
Tuesday May 14 09:15:00 GMT 2013
Filter 1:
Direction: Uplink
Dst Addr 0.0.0.0/0
Filter 2:
Direction: Downlink
Src Addr 0.0.0.0/0
Gold_time1_3577515300 1 0 None Gate Status: Discard All
QoS Class Identifier: 5
ARP Priority Level: 1
Reporting Level: Rating Grp
Metering Method: Volume
Uplink MBR: 100000000
Downlink MBR: 200000000
Uplink GBR: 10000001
Downlink GBR: 20000001
Rule Activation Time:
Tuesday May 14 10:15:00 GMT 2013
Rule De-activation Time:
Tuesday May 14 11:15:00 GMT 2013
Filter 1:
Direction: Uplink
Dst Addr 0.0.0.0/0
Filter 2:
Direction: Downlink
Src Addr 0.0.0.0/0
Gold_time1_3577522500 1 0 None Gate Status: Discard All
QoS Class Identifier: 5
ARP Priority Level: 1
Reporting Level: Rating Grp
Metering Method: Volume
Uplink MBR: 100000000
Downlink MBR: 200000000
Uplink GBR: 10000001
Downlink GBR: 20000001
Rule Activation Time:
Tuesday May 14 12:15:00 GMT 2013
Rule De-activation Time:
Tuesday May 14 13:15:00 GMT 2013
Filter 1:
Direction: Uplink
Dst Addr 0.0.0.0/0
Filter 2:
Direction: Downlink
Src Addr 0.0.0.0/0
Gold_time1_3577533300 1 0 None Gate Status: Discard All
QoS Class Identifier: 5
ARP Priority Level: 1
Reporting Level: Rating Grp
Metering Method: Volume
Uplink MBR: 100000000
Downlink MBR: 200000000
Uplink GBR: 10000001
Downlink GBR: 20000001
Rule Activation Time:
Tuesday May 14 15:15:00 GMT 2013
Rule De-activation Time:
Tuesday May 14 16:15:00 GMT 2013
Filter 1:
Direction: Uplink
Dst Addr 0.0.0.0/0
Filter 2:
Direction: Downlink
Src Addr 0.0.0.0/0
Gold_time1_3577540500 1 0 None Gate Status: Discard All
QoS Class Identifier: 5
ARP Priority Level: 1
Reporting Level: Rating Grp
Metering Method: Volume
Uplink MBR: 100000000
Downlink MBR: 200000000
Uplink GBR: 10000001
Downlink GBR: 20000001
Rule Activation Time:
Tuesday May 14 17:15:00 GMT 2013
Rule De-activation Time:
Tuesday May 14 18:15:00 GMT 2013
Filter 1:
Direction: Uplink
Dst Addr 0.0.0.0/0
Filter 2:
Direction: Downlink
Src Addr 0.0.0.0/0
Predefined Rules Enabled List: n/a
Predefined Firewall Rules Enabled List: n/a

Total acs sessions matching specified criteria: 1

Session-ID: 8:886 Username: 240010000099924@3glab.com
Callid: 02166055 IMSI/MSID: 240010000099924
ACSMgr Instance: 8 ACSMgr Card/Cpu: 16/0
SessMgr Instance: 8
Client-IP: 10.127.1.64
NAS-IP: 10.10.10.254
Access-NAS-IP(FA):
NAS-PORT: 0 NSAPI: 5
Acct-Session-ID: 0A0A0BFC02555694
NAS-ID: n/a
Access-NAS-ID(FA): n/a
3GPP2-BSID: n/a
Access-Correlation-ID(FA): n/a
3GPP2-Correlation-ID: n/a
MEID: n/a
Carrier-ID: n/a ESN: n/a
Uplink Bytes: 19570540 Downlink Bytes: 405939824
Uplink Packets: 360120 Downlink Packets: 709688
Injected Uplink Bytes: 0 Injected Downlink Bytes: 0
Injected Uplink Packets: 0 Injected Downlink Packets: 0
Buffered Uplink Packets: 0 Buffered Downlink Packets: 0
Buffered Uplink Bytes: 0 Buffered Downlink Bytes: 0
Uplink Packets in Buffer: 0 Uplink Bytes in Buffer: 0
Downlink Packets in Buffer: 0 Downlink Bytes in Buffer: 0
Buff Over-limit Uplink Pkts: 0 Buff Over-limit Uplink Bytes: 0
Buff Over-limit Downlink Pkts: 0 Buff Over-limit Downlink Bytes: 0
Processed Uplink Packets: 0 Processed Downlink Packets: 0
Dropped Uplink Packets: 0 Dropped Downlink Packets: 0
Uplink Out of Order Packets: 3 Downlink Out of Order Packets: 3861
Dyn FUI Redirected Flows: 0 Dyn FUI Discarded Pkts: 0
ITC Terminated Flows: 0 ITC Redirected Flows: 0
ITC Dropped Packets: 0 ITC ToS Remarked Packets: 0
Flow action Terminated Flows: 0
PP Flow action Terminated Flows: 0
CC Dropped Uplink Packets: 0 CC Dropped Uplink Bytes: 0
CC Dropped Downlink Packets: 0 CC Dropped Downlink Bytes: 0
NRUPC Req Made: 1 NRUPC Req Success: 1
NRUPC Req Failed: 0 NRUPC Req Time Out: 0
Current Readdressed Sessions: 0
Total Readdressed Uplink Pkts: 0
Total Readdressed Uplink Bytes: 0
Total Readdressed Downlink Pkts: 0
Total Readdressed Downlink Bytes: 0
Total Readdressing Failure: 0
Creation Time: Tuesday May 14 05:15:44 GMT 2013
Last Pkt Time: Tuesday May 14 05:46:04 GMT 2013
Duration: 00h:41m:20s
Active Charging Service name: ecs
Rule Base name: Rapids_Silver_Rule
Bandwidth Policy: n/a
FW-and-NAT Policy: n/a
NAT Policy: Not-required
TPO Policy: n/a
CF Policy ID: n/a
Old CF Policy ID: n/a
Dynamic Charging: Enabled
Dynamic Chrg Msg Received: 1 Rule Definitions Received: 5
Installs Received: 5 Removes Received: 0
Installs Succeeded: 5 Installs Failed: 0
Removes Succeeded: 0 Removes Failed: 0
Uplink Dynamic Rule Packets: 0 Uplink Dynamic Rule Bytes: 0
Downlink Dynamic Rule Packets: 0 Downlink Dynamic Rule Bytes: 0
Dynamic Charging Packet Drop statistics:
Bearer BW Limit Upl Pkts: 0 Bearer BW Limit Dnl Pkts: 0
Bearer BW Limit Upl Bytes: 0 Bearer BW Limit Dnl Bytes: 0
PCC Rule BW Limit Upl Pkts: 0 PCC Rule BW Limit Dnl Pkts: 0
PCC Rule BW Limit Upl Bytes: 0 PCC Rule BW Limit Dnl Bytes: 0
PCC Rule Gating Upl Pkts: 0 PCC Rule Gating Dnl Pkts: 0
PCC Rule Gating Upl Bytes: 0 PCC Rule Gating Dnl Bytes: 0
RuleMatch Fail Upl Pkts: 0 RuleMatch Fail Dnl Pkts: 0
RuleMatch Fail Upl Bytes: 0 RuleMatch Fail Dnl Bytes: 0
Credit-Control: Off
QoS Renegotiate Up: 0 QoS Renegotiate Dn: 0
Current TCP Proxy Flows: 0 Total TCP Proxy Flows: 0
TCP-proxy reset for non-SYN flows: 0
Current IP Flows: 128 Current ICMP Flows: 1
Current IPv6 Flows: 0 Current ICMPv6 Flows: 0
Current TCP Flows: 44 Current UDP Flows: 83
Current HTTP Flows: 0 Current HTTPS Flows: 0
Current FTP Flows: 0 Current POP3 Flows: 0
Current SMTP Flows: 0 Current SIP Flows: 0
Current RTSP Flows: 0 Current RTP Flows: 0
Current RTCP Flows: 0 Current IMAP Flows: 0
Current WSP-CO Flows: 0 Current WSP-CL Flows: 0
Current MMS Flows: 0 Current DNS Flows: 0
Current PPTP-GRE Flows: 0 Current PPTP Flows: 0
Current P2P Flows: 0 Current H323 Flows: 0
Current TFTP Flows: 0
Current UNKNOWN Flows: 127

WiMAX Hotlining Status: n/a
Link Monitoring Average Throughput: 0 kbps
Link Monitoring Average RTT: 0 ms

Ruledef Name Pkts-Down Bytes-Down Pkts-Up Bytes-Up Hits

ip_any 709687 405939823 360119 19570539 1069806
Dynamic Charging Rule Name Statistics: n/a
Total Dynamic Rules: 5
Total Predefined Rules: 0
Total Firewall Predefined Rules: 0

Dynamic Charging Rule Definition(s) Configured:
Name Prior Content-Id Chrg-Type Rule Parameters

Total acs sessions matching specified criteria: 1

Re: Regex not working for event splitting

kristian_kolb — Fri, 17 May 2013 09:52:09 GMT

They were some rather large events. There are several timestamps in your events, and you must define which one you'd like to use. In the example below, the timestamp at 'Creation Time: ' is used. Change it to suit your preference.

[3GPP]
SHOULD_LINEMERGE = false
LINEBREAKER = ([\r\n]+)Session-ID:
TRUNCATE = 0
TIME_PREFIX = (?m)Creation\sTime:\s+
TIME_FORMAT = %A %b %d %H:%M:%S %Z %Y
MAX_TIMESTAMP_LOOKAHEAD = 30

Depending on how the day-part of your timestamp is shown for the first nine days of the month, you might have to change the %d to a %e (01-31, vs 1-31).

See http://www.strftime.net for more info on this.

Hope this helps,

Re: Regex not working for event splitting

sansri7680 — Tue, 21 May 2013 15:18:44 GMT

hi,

i tried this but again it is doing the same thing. is there any work around for this

Re: Regex not working for event splitting

sansri7680 — Mon, 28 Sep 2020 13:57:22 GMT

I found out the solution. We need to specify the BREAK_ONLY_BEFORE=^(\s+)Session-ID and SHOULD_LINEMERGE=true and the events split correctly

topic Re: Regex not working for event splitting in Splunk Search

Regex not working for event splitting

Re: Regex not working for event splitting

Re: Regex not working for event splitting

Re: Regex not working for event splitting