topic Re: Pull Timestamp from Field in Raw Data in Getting Data In

Pull Timestamp from Field in Raw Data

rdschmidt — Mon, 28 Sep 2020 14:19:24 GMT

Can anyone tell me how to configure my Props.conf to use a defined field "Event_Time" (Which is in Epoch Time) for the timestamp (_time) instead of pulling the time when the file was saved?

Currently i have this configured:

TIME_FORMAT = %b %d %H:%M:%S ctime(Event_Time)
MAX_TIMESTAMP_LOOKAHEAD = 32
TZ = US/Central

Thanks,

Riley

Re: Pull Timestamp from Field in Raw Data

linu1988 — Mon, 28 Sep 2020 14:19:36 GMT

Please use

TIME_FORMAT=%s
TIME_PREFIX= (regex)
MAX_TIMESTAMP_LOOKAHEAD = 32

Re: Pull Timestamp from Field in Raw Data

rdschmidt — Thu, 11 Jul 2013 17:59:26 GMT

Raw Data:
213|2|0|1|0|x||x.x.x.x|x@wireless.com|0782A8FC|07784722|0|0|x.x.x.x|x.x.x.x|x.x.x.x|0083|0|0|0|59|0|0|0|0|0|2|1|3|0|0|9747835|197309|0|1373498910|2019|0|0|0|0|0|211465|0|0|10|0|0|0|0|0||10026|2|0|1|541|3539|6668|1|07784722|0|0|0|24|x|0104000102040001|875560960||0|0|0|0||1373498539||0|311650|0|0|0|0|0|0|0|0|0||0|0|0|-1||||

timestamp: 7/10/13 6:54:51.000 PM

Derived Fields:
| Event_Time=1373498910 | TIME=07/10/2013 18:28:30

Re: Pull Timestamp from Field in Raw Data

rdschmidt — Thu, 11 Jul 2013 17:59:36 GMT

Here is my search : x@wireless.com | convert ctime(Event_Time) as TIME

I just want to make the TIME field automatically show up as the timestamp.

Thanks,

Riley

Re: Pull Timestamp from Field in Raw Data

Ayn — Thu, 11 Jul 2013 18:21:09 GMT

http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

You don't extract timestamps from fields, because field extractions happen at a much later stage (and for most fields doesn't happen at index-time at all).

I took your sample event and threw it into RegExr (http://gskinner.com/RegExr/ ) and came up with a TIME_PREFIX regex that should work for you:

TIME_PREFIX = ^(?:[^|]*\|){34}

After that you can just use "TIME_FORMAT = %s" because it's an ordinary epoch timestamp.

Re: Pull Timestamp from Field in Raw Data

linu1988 — Mon, 28 Sep 2020 14:19:38 GMT

TIME_FORMAT=%s
TIME_PREFIX= \d{7}\|\d{6}\|\d\|
MAX_TIMESTAMP_LOOKAHEAD = 10

could you try setting props.conf and index new data?

Re: Pull Timestamp from Field in Raw Data

ryainad — Wed, 31 Jul 2013 10:24:32 GMT

Hi, Riley.

I have a question regarding Huawei CSOFTX3000 CDRs. What do you use to decode CDR file? I search for solution, and I found only this splunk application from dmillis. But still I dont understand how to use it to decode CDR files.

Thank you in advance.

Re: Pull Timestamp from Field in Raw Data

rdschmidt — Wed, 31 Jul 2013 16:07:46 GMT

The above time_prefix did fix our TimeStamp issues on our AAA records. We are still working on the CSOFTX3000. The issue is we are using 450 byte CDRs and the app is built for 350 byte.

Re: Pull Timestamp from Field in Raw Data

ryainad — Mon, 05 Aug 2013 03:31:04 GMT

I got this Huawei CDR sample file. But I don't know know what type of the file is it. (250 byte or 350 byte or 450 byte) Could you please tell me how to know it?