topic Extracting from log file in Splunk Search

Extracting from log file

nravichandran — Tue, 29 Sep 2020 10:15:55 GMT

I have the following custom log file

2016-07-15_05:58:57.5857-est label="adbcf" lastmodifiedtime="2016-07-15_05:58:57.5857-est" filename="13948.xml" directory="d:\temp" operation="deleted" size_in_bytes=434493
2016-07-15_17:57:18.5718-est monitor_label="abcd" lastmodifiedtime="2016-07-15_17:57:18.5718-est" filename="late123" directory="d:\temp" operation="created" size_in_bytes=673639

I am able to ingest into Splunk, however when i search for operation="deleted" i did not get the result.
when i search with "deleted" i am able to get the result. operation="created" returns results.
In the interesting field it only shows "created" value for operation even though both created and deleted are present in the results.

Is there anything that could be done in the custom log differently to make Splunk include the "deleted"
| timechart span=1h count by operation gives only created and ignores deleted.

Thanks in advance.

Re: Extracting from log file

inventsekar — Sat, 16 Jul 2016 01:30:32 GMT

I am not sure, but, pls search using the index name...

index=indexname "created"

Re: Extracting from log file

sundareshr — Sat, 16 Jul 2016 01:33:05 GMT

You could extract it in your search

    ... |  rex "operation=\"(?<operation>\w+)" | timechart span=1h count by operation

You could also add this rex in the Field Extraction UI to make this field available to every search.'

Re: Extracting from log file

nravichandran — Sat, 16 Jul 2016 01:47:13 GMT

index=... | rex "operation=\"(?\w+)" | table operation returns - created only eventhough deleted is present.

Re: Extracting from log file

nravichandran — Tue, 29 Sep 2020 10:15:58 GMT

One more information i want to share. Since the logs are generating at the same time will it have any effect?

7/15/16
9:44:52.445 PM

2016-07-15_21:44:52.4452-est monitor_label="aaai" lastmodifiedtime="2016-07-15_21:44:52.4452-est" filename="late3.new" directory="d:\tesb11" operation="created" size_in_bytes=9457
2016-07-15_09:44:52.4452-est monitor_label="sssi" lastmodifiedtime="2016-07-15_09:44:52.4452-est" filename="113626.xml" directory="d:\testemp" operation="deleted" size_in_bytes=316005

Re: Extracting from log file

sundareshr — Sat, 16 Jul 2016 02:24:08 GMT

Try this regex then

 ... |  rex "(?<operation>deleted|created)" | timechart span=1h count by operation

Re: Extracting from log file

Raschko — Sat, 16 Jul 2016 02:26:15 GMT

Please try the following rex command:

| rex max_match=0 "operation=\"(?<operation>[^\"]*?)\""

Do you still have one result with a multivalue "operation" field?

Re: Extracting from log file

nravichandran — Mon, 18 Jul 2016 17:20:25 GMT

I was able to figure out the rex. The following works! Thanks everyone.

rex field=_raw "(?ms)^(?:[^\"\n]*\"){9}(?P\w+)"