topic Re: How to transform a numeric field at index-time (from microseconds to milliseconds)? in Getting Data In

How to transform a numeric field at index-time (from microseconds to milliseconds)?

splunkmasterfle — Thu, 27 Aug 2015 17:38:56 GMT

This is my situation:

I am currently using an older version of Apache which does not allow request times to be logged in milliseconds. I can't update the version of Apache until our next official release of our application that is in over a year. Eventually, the requests will be logged in milliseconds. In order to prevent conflicts at the point where the field becomes milliseconds, I want to transform the current value of the field (it's in microseconds) to milliseconds at index-time. This would mean that once the new version of Apache logs the requests in milliseconds, it will not affect the older data that isn't in the same format.

Is there a way to implement a temporary transformation of the field that gets indexed that could be removed once the modification has taken place?

For context: I am using a single indexer with multiple forwarders that send the logs to be indexed.

Any help would be greatly appreciated.

Thanks

Re: How to transform a numeric field at index-time (from microseconds to milliseconds)?

somesoni2 — Thu, 27 Aug 2015 18:54:06 GMT

What is the format of logs you have? Can you post some sample events?

Re: How to transform a numeric field at index-time (from microseconds to milliseconds)?

splunkmasterfle — Thu, 27 Aug 2015 20:44:23 GMT

They are standard Apache logs:

179.31.12.34 - - [26/Aug/2015:14:26:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 9565
179.31.12.34 - - [26/Aug/2015:14:31:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 22270
179.31.12.34 - - [26/Aug/2015:14:36:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 17775
179.31.12.34 - - [26/Aug/2015:14:41:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 19384
179.31.12.34 - - [26/Aug/2015:14:46:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15199
179.31.12.34 - - [26/Aug/2015:14:51:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 13081
179.31.12.34 - - [26/Aug/2015:14:56:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14866
179.31.12.34 - - [26/Aug/2015:15:01:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14962
179.31.12.34 - - [26/Aug/2015:15:06:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 14313
179.31.12.34 - - [26/Aug/2015:15:11:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18321
179.31.12.34 - - [26/Aug/2015:15:16:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18693
179.31.12.34 - - [26/Aug/2015:15:21:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15301
179.31.12.34 - - [26/Aug/2015:15:26:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 15142
179.31.12.34 - - [26/Aug/2015:15:31:04 -0400] "GET /AnApplication/ HTTP/1.1" 200 84 18524

The last field being the request time in microseconds.

Re: How to transform a numeric field at index-time (from microseconds to milliseconds)?

diogofgm — Thu, 27 Aug 2015 21:25:10 GMT

In the props.conf in that sourcetype stanza you can do:

EVAL-request_seconds = request_time /1000000

Just replace the request_time with the field you already have being extracted for that number

Re: How to transform a numeric field at index-time (from microseconds to milliseconds)?

woodcock — Fri, 28 Aug 2015 00:21:08 GMT

The answer from @diogofgm creates a new field but this solution updates the _raw event by putting this in props.conf:

[YourSourceTypeHere]
SEDCMD-1digitTo2 =s/ \(.\)$/ 0\1/
SEDCMD-2digitsTo3 = s/ \(..\)$/ 0\1/
SEDCMD-micro2milli = s/ \(.*\)\(...\)$/ \1\.\2/