topic Re: Manipulate logs in upload in Getting Data In

Manipulate logs in upload

alisaf — Wed, 07 Aug 2019 07:29:22 GMT

Hi,
I have logs that have in the top some data that doesn't relevant for me and I would like that it won't appear.

This is the data that I would like to remove:

Device version: D02.20.33
MCFG Version: Unknown

UP TIME: 00:05:24.292

emory pool at 0x000000008f000000, size 8 MiB

also, I have some rows in the log that not include a timestamp and I want to add the same timestamp as the one in the next line. for example, this is the logs:
--------- beginning of events
01-15 04:17:19.370 453 453 I auditd : type=2000 audit(0.0:1): initialized

and I would like that it will be:
01-15 04:17:19.370 453 453 --------- beginning of events
01-15 04:17:19.370 453 453 I auditd : type=2000 audit(0.0:1): initialized

can I do that with Splunk when I'm uploading the logs?
Thank you!

Re: Manipulate logs in upload

richgalloway — Wed, 07 Aug 2019 12:36:16 GMT

The first part probably can be done using SEDCMD in props.conf if you can come up with a regex that matches the lines to remove.
The second part, however, is not possible in Splunk. You'll need to create a scripted input or use a pre-processor such as Cribl (not sure Cribl can do that, though).

Re: Manipulate logs in upload

alisaf — Wed, 07 Aug 2019 14:48:29 GMT

Thank you!
Splunk default associates this line to the previous event, maybe there is some option to associate this line with the next event?

Re: Manipulate logs in upload

richgalloway — Wed, 07 Aug 2019 16:45:15 GMT

There is no such option. That's why I said "not possible".

Re: Manipulate logs in upload

alisaf — Thu, 08 Aug 2019 05:49:30 GMT

thanks 🙂