topic Re: Combine date field with millisecond offset column in Getting Data In

Combine date field with millisecond offset column

mike_cmxx — Thu, 05 Sep 2013 21:29:46 GMT

Hi, I'm currently performing an evaluation on Splunk, so I am very new at this. I have a few questions concerning time stamps and combining fields.

Here is an example from the top of my data file:

Start Time: (September 11; 2009 11:19:0 am)

DataValue1,,DataValue2

601 ,45.416000 501 ,2.989220

1080 ,1000.03 980 ,1.124074

1200 ,45.483101 1080 ,2.946390

1741 ,992.955017 1671 ,1.124074

My file contains a single timestamp for the beginning of the log and then each data value is paired with a millisecond offset from that initial time. The first value is the offset and immediately after that is the parameter value. The offset and the value are always separated by a comma and individual "offset,value" groups are separated by a tab.

I would like to create the following data format within Splunk:

timestamp DataValue1 DataValue2

09/11/2009 11:19:00.501 null 2.989220

09/11/2009 11:19:00.601 45.416000 null

09/11/2009 11:19:00.980 null 1.124074

09/11/2009 11:19:01.080 1000.03 2.946390

09/11/2009 11:19:01.200 45.483101 null

09/11/2009 11:19:01.671 null 1.124074

09/11/2009 11:19:01.741 992.955017 null

I've been able to modify my props and transform to include basic header/field info but so far I am at a loss for how to do this type of field manipulation.

Re: Combine date field with millisecond offset column

mike_cmxx — Mon, 09 Sep 2013 17:41:32 GMT

Anyone have a suggestion here?

Re: Combine date field with millisecond offset column

sowings — Mon, 09 Sep 2013 19:11:35 GMT

Unfortunately, I don't think Splunk's time parser has the ability to do deltas in this way. Other folks have asked about startup logs which record the time since the system booted. The answer there was just as bleak.

What you might consider, however, is treating the whole thing as one "event", and then splitting the various parts out as needed when you search against them. This would work if the whole file is "only" a couple hundred lines.

Do you have any control of the log format as it's being written? We could offer suggestions on how to log efficiently....

Re: Combine date field with millisecond offset column

mike_cmxx — Mon, 09 Sep 2013 20:21:13 GMT

Unfortunately we do not have control over the format of the log file. And the real log file actually has hundreds of fields and thousands of rows.

Is it possible to add the time field to each row? And then grab the time and the offset/value pair as a search output? Giving me something like:

09/11/2009 11:19:00 501 2.98922
09/11/2009 11:19:00 601 45.416