topic Can Splunk transform a log event before to be indexed? in Getting Data In

Can Splunk transform a log event before to be indexed?

lpolo — Fri, 01 Feb 2013 17:46:10 GMT

I have this log event:

2013-02-01 17:23:46,877 query id=a0e22777-2aaf-4486-9a56-fd1dae24bb82{
  "start" : 1,
  "returned" : 0,
  "count" : 0
}query

I need to index the same log event in splunk as follow:

{
  "query_time" : "2013-02-01 17:23:46,877",
  "id" : a0e22777-2aaf-4486-9a56-fd1dae24bb82,
  "start" : 1,
  "returned" : 0,
  "count" : 0
}

Is there a way to do it using transforms.conf and props.conf:?

Thanks,
Lp

Re: Can Splunk transform a log event before to be indexed?

martin_mueller — Fri, 01 Feb 2013 23:03:49 GMT

Yes, sed to the rescue: http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Anonymizedatausingconfigurationfiles#Through_a_sed_script

Something along these lines, untested:

SEDCMD-foo = s/([0-9- :,]+)\s+query\s+id=([0-9a-f]+){(.*)}query/{"query_time":"\1","id":"\2",\3}/

Re: Can Splunk transform a log event before to be indexed?

yannK — Sat, 02 Feb 2013 00:35:28 GMT

Beware, the sed may occur after the timestamp extraction, verify your timestamp in splunk.

Re: Can Splunk transform a log event before to be indexed?

lpolo — Mon, 04 Feb 2013 19:44:05 GMT

Thanks Martin. I am going to tested this week.
Lp

Re: Can Splunk transform a log event before to be indexed?

lpolo — Wed, 13 Mar 2013 13:59:35 GMT

I took a different approach to solve the problem. I extracted the json portion of the log event using a regex. Then, I was able to use spath without any problem.

Thanks,
Lp