Getting Data In

How to split multiple lines of data into a single individual line in splunk?

Hemnaath
Motivator

Hi All, We are monitoring the wtmpx data from the Unix machines via splunk using the Splunk add-on for Unix, based on this add-on we could see the data from the wtmpx file in splunk, but currently we could see the data ingesting from the wtmpx file into splunk with multiple line like shown below.

  6/20/18
6:26:03.000 AM  
USERNAME        LINE        HOSTNAME                                  TIME
HXXX019        pts/1       w442xty1.XXXX.com                     Jun 19 18:40
Hxxx007        pts/5       yb33gnn1.XXXX.com                     Jun 19 08:53


6/20/18
6:23:33.000 AM  
USERNAME        LINE        HOSTNAME                                  TIME
HXXX019        pts/1       w442xty1.XXXX.com                     Jun 19 18:40
HXXX007        pts/5       yb33gnn1.XXXX.com                     Jun 19 08:53

But instead of multiple line we want to have it as single event like this

USERNAME        LINE        HOSTNAME                                  TIME
HXXX019        pts/1       w442xty1.XXXX.com                     Jun 19 18:40

USERNAME        LINE        HOSTNAME                                  TIME
HXXX007        pts/5       yb33gnn1.XXXX.com                    Jun 19 08:53

And also if there is no content in the log then it should be removed from splunk.
example:

    USERNAME        LINE        HOSTNAME                                  TIME

host=r3crp00   source=who        sourcetype=who

Props.conf details for the sourcetype=who

[who]
SHOULD_LINEMERGE=false
LINE_BREAKER=^()$
TRUNCATE=1000000
DATETIME_CONFIG = CURRENT

sourcetype=lastlog

[lastlog]
## Override system/default lastlog sourcetype invalidation
invalid_cause =
SHOULD_LINEMERGE=false
LINE_BREAKER=^()$
TRUNCATE=1000000
DATETIME_CONFIG = CURRENT

Kindly guide me how to configure this in the props.conf file.

thanks in advance.

Tags (2)
0 Karma
1 Solution

Hemnaath
Motivator

Hi All,

Got how to split the multiple event in to single individual event using the Line_Breaker stanza in props.conf

 [who]
 SHOULD_LINEMERGE=false
 LINE_BREAKER=([\r\n]+)
 TRUNCATE=1000000
 DATETIME_CONFIG = CURRENT

[lastlog]
 ## Override system/default lastlog sourcetype invalidation
 invalid_cause =
 SHOULD_LINEMERGE=false
 LINE_BREAKER=([\r\n]+)
 TRUNCATE=1000000
 DATETIME_CONFIG = CURRENT

View solution in original post

Hemnaath
Motivator

Hi All,

Got how to split the multiple event in to single individual event using the Line_Breaker stanza in props.conf

 [who]
 SHOULD_LINEMERGE=false
 LINE_BREAKER=([\r\n]+)
 TRUNCATE=1000000
 DATETIME_CONFIG = CURRENT

[lastlog]
 ## Override system/default lastlog sourcetype invalidation
 invalid_cause =
 SHOULD_LINEMERGE=false
 LINE_BREAKER=([\r\n]+)
 TRUNCATE=1000000
 DATETIME_CONFIG = CURRENT
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

From Data to Insight: Announcing the Winners of the Splunk Dashboard Contest

Hi Splunkers, First off, thank you to everyone who participated in our very first From Data to Insight: The ...

Splunk Developers: Construct Your Future at the .conf26 Builder Bar

Calling all Splunk architects, platform admins, and app developers: the site is open, and the blueprints are ...

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...