topic Re: Timestamp splitted in log files in Splunk Search

Timestamp splitted in log files

bizza — Wed, 30 Apr 2014 13:55:50 GMT

Hi all,
I'm trying to extract the timestamp from a crappy unstructured logs.
Every event is one line with 300 character/digits, and field are in fixed position (example: from 1 to 10 means hostname, from 11 to 12 means status, and so on).
I need to compose my timestamp merging 2 different fields: from position 15 to 20 and from position 60 to 66.

Any hints?

Regards

Re: Timestamp splitted in log files

lukejadamec — Wed, 30 Apr 2014 14:13:39 GMT

Can you post an example of the first 75 characters?

Re: Timestamp splitted in log files

bizza — Wed, 30 Apr 2014 14:26:25 GMT

Just an example, with 1 instead of orig digit and A instead of char, white space are actually the same.
Extracted timestamp shoud be 201404300833 in %Y%m&d%H%M format

1111111111 1.20140430AAA111 11AAAA AAAAA AA11111111111110833111A AAAAAAAAA 111111111 AAAAAAAA1111111

Re: Timestamp splitted in log files

richgalloway — Wed, 30 Apr 2014 14:57:51 GMT

Based on your example, the following regex will extract the date and time fields.

[\s\S]{13}(?<date>\d{8})[\s\S]{36}(?<time>\d{4})

You could then combine them at search time using eval ts=date.time.

Re: Timestamp splitted in log files

bizza — Mon, 28 Sep 2020 16:30:20 GMT

I put in my props.conf, in the right sourcetype:
EXTRACT-extract_time = your_regex

And I searched
sourcetype = mysourcetype | eval ts=date.time

If it is supposed to work it don't.
What I'm missing?

Re: Timestamp splitted in log files

somesoni2 — Wed, 30 Apr 2014 16:06:32 GMT

sourcetype = mysourcetype | table date, time

Do you get values for fields date, time?
And hope you restarted/refreshed splunk after props.conf change.

Re: Timestamp splitted in log files

bizza — Wed, 30 Apr 2014 16:12:09 GMT

Yep, I restarted Splunk after that.
Using table command I see correctly date and time fields, and in the left column too, but I'm not able to use it as timestamp in my searches

Re: Timestamp splitted in log files

richgalloway — Wed, 30 Apr 2014 16:25:50 GMT

Just how, exactly, are you trying to use the date and time fields?

Re: Timestamp splitted in log files

bizza — Wed, 30 Apr 2014 16:36:43 GMT

I just need to graph data using ts (from eval) as timestamp

Re: Timestamp splitted in log files

richgalloway — Wed, 30 Apr 2014 16:46:48 GMT

Assuming sourcetype=mysourcetype | eval ts=date.time | table date, time, ts produces results, you'll want to convert ts from a string into an integer using strptime() and use that value in your graph.

Re: Timestamp splitted in log files

bizza — Mon, 28 Sep 2020 16:30:29 GMT

sourcetype="mysourcetype" | eval ts=date.time | eval _time = strptime(ts, "%Y%m%d%H%M") | timechart count by my_field

works great!

thanks