Getting Data In

How to configure Splunk to break events that start with a certain pattern followed by a timestamp?

ngeorgieff
New Member

I have the logs like below pattern. I want to break the events that starts with <94>1and then timestamp

<94>1 2016-08-31T17:31:25.633-07:00 hostname-1-p02.domain.com GAMFT - FTP Audit Log [gamft-ftp@46583 event_type="Connection Successful" remote_ip="10.0.203.75" severity="I" start_time="8/31/16 5:31:25 PM" end_time="8/31/16 5:31:25 PM" local_ip="10.11.215.194" local_port="8021" command="Connect" time_taken="14" remarks="Connection established" system_name="hostname-1-p02"]<94>1 2016-08-31T17:31:29.166-07:00 hostname-1-p02.domain.com GAMFT - FTP Audit Log [gamft-ftp@46583 event_type="Login Successful" remote_ip="10.0.203.75" user_name="splunk_test" severity="I" start_time="8/31/16 5:31:29 PM" end_time="8/31/16 5:31:29 PM" local_ip="10.11.215.194" local_port="8021" command="Login" time_taken="68" remarks="230 User logged in, proceed." system_name="hostname-1-p02" domain="Infrastructure Services"]<94>1 2016-08-31T17:31:31.402-07:00 hostname-1-p02.domain.com GAMFT - FTP Audit Log [gamft-ftp@46583 event_type="Logout" remote_ip="10.0.203.75" user_name="splunk_test" severity="I" start_time="8/31/16 5:31:31 PM" end_time="8/31/16 5:31:31 PM" local_ip="0.0.0.0" local_port="8021" command="Logout" time_taken="10" remarks="221 Goodbye." system_name="hostname-1-p02" domain="Infrastructure Services"]<94>1 2016-08-31T17:31:31.414-07:00 hostname-1-p02.domain.com GAMFT - FTP Audit Log [gamft-ftp@46583 event_type="Disconnect" remote_ip="10.0.203.75" user_name="splunk_test" severity="I" start_time="8/31/16 5:31:31 PM" end_time="8/31/16 5:31:31 PM" local_ip="0.0.0.0" local_port="8021" command="Disconnect" time_taken="8" remarks="Disconnected" system_name="hostname-1-p02" domain="Infrastructure Services"]
0 Karma
1 Solution

vprisiajni_splu
Splunk Employee
Splunk Employee

In my test with these sample events, the following seemed to work:

LINE_BREAKER = (<94>1)(\s)
SHOULD_LINEMERGE = false

Explanation from our props.conf docs:

  • The regex must contain a capturing group -- a pair of parentheses which   defines an identified subcomponent of the match.
  • Wherever the regex matches, Splunk considers the start of the first   capturing group to be the end of the previous event, and considers the end   of the first capturing group to be the start of the next event.
  • The contents of the first capturing group are discarded, and will not be   present in any event.  You are telling Splunk that this text comes between   lines.

View solution in original post

0 Karma

vprisiajni_splu
Splunk Employee
Splunk Employee

In my test with these sample events, the following seemed to work:

LINE_BREAKER = (<94>1)(\s)
SHOULD_LINEMERGE = false

Explanation from our props.conf docs:

  • The regex must contain a capturing group -- a pair of parentheses which   defines an identified subcomponent of the match.
  • Wherever the regex matches, Splunk considers the start of the first   capturing group to be the end of the previous event, and considers the end   of the first capturing group to be the start of the next event.
  • The contents of the first capturing group are discarded, and will not be   present in any event.  You are telling Splunk that this text comes between   lines.
0 Karma

ngeorgieff
New Member

It works. Thank you very much for your help and explanation.

0 Karma

ddrillic
Ultra Champion

A place to investigate it at Configure event line breaking

0 Karma

ngeorgieff
New Member

I've tried with LB regex, data prefix but none worked for me.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...