Getting Data In

Can Splunk transform a log event before to be indexed?

lpolo
Motivator

I have this log event:

2013-02-01 17:23:46,877 query id=a0e22777-2aaf-4486-9a56-fd1dae24bb82{
  "start" : 1,
  "returned" : 0,
  "count" : 0
}query

I need to index the same log event in splunk as follow:

{
  "query_time" : "2013-02-01 17:23:46,877",
  "id" : a0e22777-2aaf-4486-9a56-fd1dae24bb82,
  "start" : 1,
  "returned" : 0,
  "count" : 0
}

Is there a way to do it using transforms.conf and props.conf:?

Thanks,
Lp

0 Karma

lpolo
Motivator

I took a different approach to solve the problem. I extracted the json portion of the log event using a regex. Then, I was able to use spath without any problem.

Thanks,
Lp

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Yes, sed to the rescue: http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Anonymizedatausingconfigurationfiles#Through_...

Something along these lines, untested:

SEDCMD-foo = s/([0-9- :,]+)\s+query\s+id=([0-9a-f]+){(.*)}query/{"query_time":"\1","id":"\2",\3}/

lpolo
Motivator

Thanks Martin. I am going to tested this week.
Lp

0 Karma

yannK
Splunk Employee
Splunk Employee

Beware, the sed may occur after the timestamp extraction, verify your timestamp in splunk.

0 Karma
Get Updates on the Splunk Community!

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...