Hi, I'm currently performing an evaluation on Splunk, so I am very new at this. I have a few questions concerning time stamps and combining fields.
Here is an example from the top of my data file:
Start Time: (September 11; 2009 11:19:0 am)
DataValue1,,DataValue2
601 ,45.416000 501 ,2.989220
1080 ,1000.03 980 ,1.124074
1200 ,45.483101 1080 ,2.946390
1741 ,992.955017 1671 ,1.124074
My file contains a single timestamp for the beginning of the log and then each data value is paired with a millisecond offset from that initial time. The first value is the offset and immediately after that is the parameter value. The offset and the value are always separated by a comma and individual "offset,value" groups are separated by a tab.
I would like to create the following data format within Splunk:
timestamp DataValue1 DataValue2
09/11/2009 11:19:00.501 null 2.989220
09/11/2009 11:19:00.601 45.416000 null
09/11/2009 11:19:00.980 null 1.124074
09/11/2009 11:19:01.080 1000.03 2.946390
09/11/2009 11:19:01.200 45.483101 null
09/11/2009 11:19:01.671 null 1.124074
09/11/2009 11:19:01.741 992.955017 null
I've been able to modify my props and transform to include basic header/field info but so far I am at a loss for how to do this type of field manipulation.
Unfortunately, I don't think Splunk's time parser has the ability to do deltas in this way. Other folks have asked about startup logs which record the time since the system booted. The answer there was just as bleak.
What you might consider, however, is treating the whole thing as one "event", and then splitting the various parts out as needed when you search against them. This would work if the whole file is "only" a couple hundred lines.
Do you have any control of the log format as it's being written? We could offer suggestions on how to log efficiently....
Unfortunately we do not have control over the format of the log file. And the real log file actually has hundreds of fields and thousands of rows.
Is it possible to add the time field to each row? And then grab the time and the offset/value pair as a search output? Giving me something like:
09/11/2009 11:19:00 501 2.98922
09/11/2009 11:19:00 601 45.416
Anyone have a suggestion here?