topic Combinate SHOULD_LINEMERGE with Filtering in Getting Data In

Combinate SHOULD_LINEMERGE with Filtering

danillopavan — Sat, 09 Dec 2017 20:22:32 GMT

Hi all,

I am trying to have a combination of SHOULD_LINEMERGE=true with filtering just to index some lines of the log file and diregards the others lines.

Trying to use the below but not working

[sourcetype]
TRANSFORMS-set= setnull,setparsing
SHOULD_LINEMERGE=true

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = lvsapsd
DEST_KEY = queue
FORMAT = indexQueue

Part of the text of the log file:
S Doing: print 1591111lllllll
S lvsapsd -> Print Job @>SPOREQ:1597246@ S print job @>SPOREQ:1597246@</1 has no list attributes
S replace user SAPSYS by 99718165

It is creating one event but not filtering just the second line. It is bringing all the lines.

How I can combinate the usage of SHOULD_LINEMERGE with Filtering?

Thanks and regards,
Danillo Pavan

Re: Combinate SHOULD_LINEMERGE with Filtering

sandyIscream — Tue, 12 Dec 2017 13:15:47 GMT

Hi @danillopavan,

Do you have any timestamps in your logs ? If not then Splunk considers both the line as one event.

Try to break the lines in the props itself.

[sourcetype]
TRANSFORMS-set= setnull,setparsing
SHOULD_LINEMERGE= false

This will separate each line then write your transforms.conf as it is.

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = lvsapsd
DEST_KEY = queue
FORMAT = indexQueue

Re: Combinate SHOULD_LINEMERGE with Filtering

danillopavan — Tue, 12 Dec 2017 13:24:32 GMT

Hi Sandy, thanks for your reply.

I have a timestamps in the logs, however it is registering the time minute by minute and not event by event, so I am not using timestamps as delimiter. My idea is to consider multiple lines as one event, because of that i am using the command SHOULD_LINEMERGE = true, but my expectation is to have just some lines filtered in the unique event and not all lines. So i would like to know if it is possible to filter merged lines. I tried everything on my side and it is not working. Or all lines are indexed in only one event, or the lines are filtered however having one event for each filtered line.

Still need help here.

Thanks and regards,
Danillo Pavan

Re: Combinate SHOULD_LINEMERGE with Filtering

danillopavan — Sat, 16 Dec 2017 15:05:09 GMT

Any answer?

Re: Combinate SHOULD_LINEMERGE with Filtering

niketn — Sat, 16 Dec 2017 18:48:06 GMT

@danillopavan this seems similar to other question your have posted: https://answers.splunk.com/answers/597389/filtering-data-using-should-linemerge.html

I would request you to consolidate required details against single question and keep only one of them open.

Re: Combinate SHOULD_LINEMERGE with Filtering

danillopavan — Tue, 19 Dec 2017 20:59:53 GMT

Closing this topic, keeping just the other one that I have created as it is similar:

https://answers.splunk.com/answers/597389/filtering-data-using-should-linemerge.html