Getting Data In

How to transform a numeric field at index-time (from microseconds to milliseconds)?

Path Finder

This is my situation:

I am currently using an older version of Apache which does not allow request times to be logged in milliseconds. I can't update the version of Apache until our next official release of our application that is in over a year. Eventually, the requests will be logged in milliseconds. In order to prevent conflicts at the point where the field becomes milliseconds, I want to transform the current value of the field (it's in microseconds) to milliseconds at index-time. This would mean that once the new version of Apache logs the requests in milliseconds, it will not affect the older data that isn't in the same format.

Is there a way to implement a temporary transformation of the field that gets indexed that could be removed once the modification has taken place?

For context: I am using a single indexer with multiple forwarders that send the logs to be indexed.

Any help would be greatly appreciated.


0 Karma

Esteemed Legend

The answer from @diogofgm creates a new field but this solution updates the _raw event by putting this in props.conf:

SEDCMD-1digitTo2 =s/ \(.\)$/ 0\1/
SEDCMD-2digitsTo3 = s/ \(..\)$/ 0\1/
SEDCMD-micro2milli = s/ \(.*\)\(...\)$/ \1\.\2/
0 Karma


In the props.conf in that sourcetype stanza you can do:

EVAL-request_seconds = request_time /1000000

Just replace the request_time with the field you already have being extracted for that number

Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

Revered Legend

What is the format of logs you have? Can you post some sample events?

0 Karma

Path Finder

They are standard Apache logs: - - [26/Aug/2015:14:26:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 9565 - - [26/Aug/2015:14:31:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 22270 - - [26/Aug/2015:14:36:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 17775 - - [26/Aug/2015:14:41:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 19384 - - [26/Aug/2015:14:46:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15199 - - [26/Aug/2015:14:51:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 13081 - - [26/Aug/2015:14:56:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14866 - - [26/Aug/2015:15:01:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14962 - - [26/Aug/2015:15:06:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14313 - - [26/Aug/2015:15:11:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18321 - - [26/Aug/2015:15:16:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18693 - - [26/Aug/2015:15:21:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15301 - - [26/Aug/2015:15:26:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15142 - - [26/Aug/2015:15:31:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18524

The last field being the request time in microseconds.

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...