Getting Data In
Highlighted

How do I configure proper line breaking for my sample multiline event in Splunk 6.4?

Path Finder

Hi...

I am using a Mainframe log which has different type of events. I am only trying to split the lines of events which look like below and no other events. How can I configure this using Line_breaker.

MR0000000 DCXA 15217 01:00:01.96 INTERNAL 00000090 IEE949I 01.00.01 SMF DATA SETS 929
DR 929 00000090 NAME VOLSER SIZE(BLKS) %FULL STATUS
DR 929 00000090 P-SYS1.MAN1 C7SP09 99630 0 ALTERNATE
DR 929 00000090 S-SYS1.MAN2 C7SP14 99630 33 ACTIVE
DR 929 00000090 S-SYS1.MAN3 C7SP20 99630 0 ALTERNATE
DR 929 00000090 S-SYS1.MAN4 C7SP21 99630 0 ALTERNATE
DR 929 00000090 S-SYS1.MAN5 C7SP78 100080 0 ALTERNATE
DR 929 00000090 S-SYS1.MAN6 C7SP88 100080 0 ALTERNATE
ER 929 00000090 S-SYS1.MAN7 C7SP89 100080 0 ALTERNATE

0 Karma
Highlighted

Re: How do I configure proper line breaking for my sample multiline event in Splunk 6.4?

SplunkTrust
SplunkTrust

So all these lines should be part of single events? What are the other types of lines that you've?

0 Karma
Highlighted

Re: How do I configure proper line breaking for my sample multiline event in Splunk 6.4?

Path Finder

The events vary some are single line events and some are multi line events. The multi line events are of various types.

In the above event the reason I want to split is because I wan to extract two fields on of the data set name and the other is percentage utilized. Since this is a single event I am not able to successfully extract this data. highlighted is the field I want to extract.

ER 929 00000090 S-SYS1.MAN7 C7SP89 100080 0 ALTERNATE

Example of single line event

N 6000000 DCXA 15217 00:51:34.62 STC15574 00000090 GLO2106I INCREASING SECONDARY ALLOCATION TO 109 CYLS -AOPT

Example of multiline event

M 0020000 DCXA 15217 00:51:36.91 STC02536 00000281 SVTM052I STEP1 COPY ARIBAGL ( 59,953) SNODE=CD.VIPCD01 044
D 044 00000281 SVTM052I FROM GLP0.OUT.DFS.TRECS.DLYTRN.G1086V00
D 044 00000281 SVTM052I TO /opt/cdunix/data/INT...ly_20150805005136.dat
E 044 00000281 SVTM052I #### COMPLETED 00000008/SDE0210I

Example of multiline event

MR0000000 DCXA 15217 00:51:37.28 AUTSYSXA 00000090 IEE112I 00.51.37 PENDING REQUESTS 067
LR 067 00000090 RM=3 IM=0 CEM=0 EM=0 RU=0 IR=0 NOAMRF
LR 067 00000090 ID:R/K T JOB ID MESSAGE TEXT
DR 067 00000090 41 R C653STAT *41 IEF235D C653STAT SMFDUMP WAITING FOR
DR 067 00000090 VOLUMES. TO CANCEL WAIT REPLY 'NO'
DR 067 00000090 67 R CNMS *67 DSI802A H160O REPLY WITH VALID NCCF
DR 067 00000090 SYSTEM OPERATOR COMMAND
DR 067 00000090 24 R NETV160 *24 DSI802A H160N REPLY WITH VALID NCCF
ER 067 00000090 SYSTEM OPERATOR COMMAND

0 Karma
Highlighted

Re: How do I configure proper line breaking for my sample multiline event in Splunk 6.4?

Influencer

Firstly you can still extract all the values as a multivalued field if you want by setting max_match=0 if you are using the rex command, or modify your fields.conf if you're doing this via a REPORT transform (see http://docs.splunk.com/Documentation/Splunk/6.4.0/Knowledge/ConfigureSplunktoparsemulti-valuefields) for more info.

However you are rightly making the effort to break the events!

You need to be very clear on what constitutes a new event. From the samples you've provided I can hazard a guess that a new event starts with either:

  • A letter, a space, and a seven digit number followed by a space and the string DCXA
  • Two letters and a seven digit number followed by a space and the string DCXA

If that's the case then you could use something like:

LINE_BREAKER = ([\n\r]+)(?=\w{2}\d{7}\sDCXA|\w\s\d{7}\sDCXA)

There are loads of good regex websites - this is an example: https://regex101.com/

0 Karma
Highlighted

Re: How do I configure proper line breaking for my sample multiline event in Splunk 6.4?

Path Finder

So 1st I will need to stop splunk.
Then update the props.conf file and then start splunk. and the data will come up in different lines.

0 Karma