I have a log file. I want to ignore lines from that log file before indexing it. can I do this? Please help me on this matter as soon as possible

[saibal6]

In the log file I have below mentioned a line :

EVENT_SESH;0;04/01/2018 06:30:23:5000;1;;1;0;;;END OF IMPORT PROCESS FOR THE MASTER STORE - PENDING_TXT(0).

I want to ignore all the lines after above mentioned line before indexing.

[gcusello · Accepted solution]

Hi saibal6,
let me understand:

• you have files with many logs to take,
• when there's the above event, do you want to discard alla events until the end of the file,

is it correct?

In this case, you could use in props.conf

SEDCMD-delete = s/(?ms)EVENT_SESH.*//g

Or a pre-parsing script to run before indexing

If instead you want to discard only the event with EVENT_SESH, you can follow http://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Routeandfilterdatad

Edit props.conf and add the following:

[your_sourcetype]
TRANSFORMS-set= setparsing,setnull

Edit transforms.conf and add the following:

[setnull]
REGEX = EVENT_SESH
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

Bye.
Giuseppe

[saibal6]

Thanks Giuseppe for the answer.

In our case your first suggestion is useful.
we added the line in " SEDCMD-delete = s/(?ms)EVENT_SESH.*//g" in props.conf but it didn't work. It's picking up all the lines from my Log file.

Could you please suggest the next step that we have to do.

[gcusello]

one additional information:
let me understand: do you have one line for each event or one event in the file?
do you want to maintain "EVENT_SESH" event and discard the others?

this regex discards all the events where there's "EVENT_SESH", it doesn't take more events, so if you want to discard all the events from the one with "EVENT_SESH" to the end of file, you have to find a different solution: for this reason I suggested also the second solution.

If instead you want to maintain "EVENT_SESH" event and discard the others, you have to try with a different regex, to test because I don't know if the event format is always the same:

SEDCMD-delete = s/(?ms)EVENT_SESH.*/EVENT_SESH;(.*;){8}.*\./g

Bye.
Giuseppe