Getting Data In

How to transform a numeric field at index-time (from microseconds to milliseconds)?

splunkmasterfle
Path Finder

This is my situation:

I am currently using an older version of Apache which does not allow request times to be logged in milliseconds. I can't update the version of Apache until our next official release of our application that is in over a year. Eventually, the requests will be logged in milliseconds. In order to prevent conflicts at the point where the field becomes milliseconds, I want to transform the current value of the field (it's in microseconds) to milliseconds at index-time. This would mean that once the new version of Apache logs the requests in milliseconds, it will not affect the older data that isn't in the same format.

Is there a way to implement a temporary transformation of the field that gets indexed that could be removed once the modification has taken place?

For context: I am using a single indexer with multiple forwarders that send the logs to be indexed.

Any help would be greatly appreciated.

Thanks

0 Karma

woodcock
Esteemed Legend

The answer from @diogofgm creates a new field but this solution updates the _raw event by putting this in props.conf:

[YourSourceTypeHere]
SEDCMD-1digitTo2 =s/ \(.\)$/ 0\1/
SEDCMD-2digitsTo3 = s/ \(..\)$/ 0\1/
SEDCMD-micro2milli = s/ \(.*\)\(...\)$/ \1\.\2/
0 Karma

diogofgm
SplunkTrust
SplunkTrust

In the props.conf in that sourcetype stanza you can do:

EVAL-request_seconds = request_time /1000000

Just replace the request_time with the field you already have being extracted for that number

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

somesoni2
Revered Legend

What is the format of logs you have? Can you post some sample events?

0 Karma

splunkmasterfle
Path Finder

They are standard Apache logs:

179.31.12.34 - - [26/Aug/2015:14:26:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 9565
179.31.12.34 - - [26/Aug/2015:14:31:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 22270
179.31.12.34 - - [26/Aug/2015:14:36:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 17775
179.31.12.34 - - [26/Aug/2015:14:41:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 19384
179.31.12.34 - - [26/Aug/2015:14:46:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15199
179.31.12.34 - - [26/Aug/2015:14:51:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 13081
179.31.12.34 - - [26/Aug/2015:14:56:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14866
179.31.12.34 - - [26/Aug/2015:15:01:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14962
179.31.12.34 - - [26/Aug/2015:15:06:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14313
179.31.12.34 - - [26/Aug/2015:15:11:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18321
179.31.12.34 - - [26/Aug/2015:15:16:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18693
179.31.12.34 - - [26/Aug/2015:15:21:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15301
179.31.12.34 - - [26/Aug/2015:15:26:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15142
179.31.12.34 - - [26/Aug/2015:15:31:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18524

The last field being the request time in microseconds.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...