Getting Data In

Pull Timestamp from Field in Raw Data

rdschmidt
Explorer

Can anyone tell me how to configure my Props.conf to use a defined field "Event_Time" (Which is in Epoch Time) for the timestamp (_time) instead of pulling the time when the file was saved?

Currently i have this configured:

TIME_FORMAT = %b %d %H:%M:%S ctime(Event_Time)
MAX_TIMESTAMP_LOOKAHEAD = 32
TZ = US/Central

Thanks,

Riley

Tags (3)
0 Karma

ryainad
Explorer

Hi, Riley.

I have a question regarding Huawei CSOFTX3000 CDRs. What do you use to decode CDR file? I search for solution, and I found only this splunk application from dmillis. But still I dont understand how to use it to decode CDR files.

Thank you in advance.

0 Karma

ryainad
Explorer

I got this Huawei CDR sample file. But I don't know know what type of the file is it. (250 byte or 350 byte or 450 byte) Could you please tell me how to know it?

0 Karma

rdschmidt
Explorer

The above time_prefix did fix our TimeStamp issues on our AAA records. We are still working on the CSOFTX3000. The issue is we are using 450 byte CDRs and the app is built for 350 byte.

linu1988
Champion

TIME_FORMAT=%s
TIME_PREFIX= \d{7}\|\d{6}\|\d\|
MAX_TIMESTAMP_LOOKAHEAD = 10

could you try setting props.conf and index new data?

0 Karma

Ayn
Legend

http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

You don't extract timestamps from fields, because field extractions happen at a much later stage (and for most fields doesn't happen at index-time at all).

I took your sample event and threw it into RegExr (http://gskinner.com/RegExr/ ) and came up with a TIME_PREFIX regex that should work for you:

TIME_PREFIX = ^(?:[^|]*\|){34}

After that you can just use "TIME_FORMAT = %s" because it's an ordinary epoch timestamp.

rdschmidt
Explorer

Here is my search : x@wireless.com | convert ctime(Event_Time) as TIME

I just want to make the TIME field automatically show up as the timestamp.

Thanks,

Riley

0 Karma

rdschmidt
Explorer

Raw Data:
213|2|0|1|0|x||x.x.x.x|x@wireless.com|0782A8FC|07784722|0|0|x.x.x.x|x.x.x.x|x.x.x.x|0083|0|0|0|59|0|0|0|0|0|2|1|3|0|0|9747835|197309|0|1373498910|2019|0|0|0|0|0|211465|0|0|10|0|0|0|0|0||10026|2|0|1|541|3539|6668|1|07784722|0|0|0|24|x|0104000102040001|875560960||0|0|0|0||1373498539||0|311650|0|0|0|0|0|0|0|0|0||0|0|0|-1||||

timestamp: 7/10/13 6:54:51.000 PM

Derived Fields:
| Event_Time=1373498910 | TIME=07/10/2013 18:28:30

0 Karma

linu1988
Champion

Please use

TIME_FORMAT=%s
TIME_PREFIX= (regex)
MAX_TIMESTAMP_LOOKAHEAD = 32

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...