Pull Timestamp from Field in Raw Data

[rdschmidt]

Can anyone tell me how to configure my Props.conf to use a defined field "Event_Time" (Which is in Epoch Time) for the timestamp (_time) instead of pulling the time when the file was saved?

Currently i have this configured:

TIME_FORMAT = %b %d %H:%M:%S ctime(Event_Time)
MAX_TIMESTAMP_LOOKAHEAD = 32
TZ = US/Central

Thanks,

Riley

[ryainad]

Hi, Riley.

I have a question regarding Huawei CSOFTX3000 CDRs. What do you use to decode CDR file? I search for solution, and I found only this splunk application from dmillis. But still I dont understand how to use it to decode CDR files.

Thank you in advance.

[ryainad]

I got this Huawei CDR sample file. But I don't know know what type of the file is it. (250 byte or 350 byte or 450 byte) Could you please tell me how to know it?

[rdschmidt]

The above time_prefix did fix our TimeStamp issues on our AAA records. We are still working on the CSOFTX3000. The issue is we are using 450 byte CDRs and the app is built for 350 byte.

[linu1988]

TIME_FORMAT=%s
TIME_PREFIX= \d{7}\|\d{6}\|\d\|
MAX_TIMESTAMP_LOOKAHEAD = 10

could you try setting props.conf and index new data?

[Ayn]

http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

You don't extract timestamps from fields, because field extractions happen at a much later stage (and for most fields doesn't happen at index-time at all).

I took your sample event and threw it into RegExr (http://gskinner.com/RegExr/ ) and came up with a TIME_PREFIX regex that should work for you:

TIME_PREFIX = ^(?:[^|]*\|){34}

After that you can just use "TIME_FORMAT = %s" because it's an ordinary epoch timestamp.

[rdschmidt]

Here is my search : [email protected] | convert ctime(Event_Time) as TIME

I just want to make the TIME field automatically show up as the timestamp.

Thanks,

Riley

[rdschmidt]

Raw Data:
213|2|0|1|0|x||x.x.x.x|[email protected]|0782A8FC|07784722|0|0|x.x.x.x|x.x.x.x|x.x.x.x|0083|0|0|0|59|0|0|0|0|0|2|1|3|0|0|9747835|197309|0|1373498910|2019|0|0|0|0|0|211465|0|0|10|0|0|0|0|0||10026|2|0|1|541|3539|6668|1|07784722|0|0|0|24|x|0104000102040001|875560960||0|0|0|0||1373498539||0|311650|0|0|0|0|0|0|0|0|0||0|0|0|-1||||

timestamp: 7/10/13 6:54:51.000 PM

Derived Fields:
| Event_Time=1373498910 | TIME=07/10/2013 18:28:30

[linu1988]

Please use

TIME_FORMAT=%s
TIME_PREFIX= (regex)
MAX_TIMESTAMP_LOOKAHEAD = 32