topic Extract date from a varying source name in Getting Data In

Extract date from a varying source name

luv — Fri, 06 Dec 2013 18:13:06 GMT

Hi Guys,

My log files has events with the time stamp on it, just the time not the date but luckily the source name has the date in it and splunk automatically identifies date from the source name and displays it with the events accordingly.

My logs:-
10:32:21,453 INFO [2212] abcdxyz
10:32:21,112 INFO [2212] abcdxyz
10:32:22,409 INFO [1121] abcdxyz

source names :- server-nameA.2013-10-01
server-nameB.2013-10-01

splunk is showing the events after indexing like:-

2013/10/01 10:32:21,453 INFO [2212] abcdxyz
2013/10/01 10:32:21,112 INFO [2212] abcdxyz
2013/10/01 10:32:22,409 INFO [1121] abcdxyz

But sometimes my log files also has version number attached to them at the last.

source name with version number : server-nameA.2013-10-01.1
server-nameB.2013-10-01.1

Now splunk is also taking version number for the date and after indexing my events look like:

2010/10/01 10:33:23,343 INFO [2232] abcdxyz
2010/10/01 10:33:19,144 INFO [2394] abcdxyz
2010/10/01 10:34:23,239 INFO [1943] abcdxyz

i want the date to be 2013/10/01 not 2010/10/01 when the source name is something like server-nameA.2013-10-01.1

I have searched through the internet for an answer but none of them assured me a valid result.
Please, Can anyone help me fix this issue?

Many Regards...

Re: Extract date from a varying source name

somesoni2 — Fri, 06 Dec 2013 19:07:55 GMT

I don't see the event example you listed 2nd time is different from 1st one. Did you miss pasting the new data.?

Re: Extract date from a varying source name

luv — Fri, 06 Dec 2013 19:49:40 GMT

time stamp without version in source name:-
2013/10/01 10:32:21,453 INFO [2212] abcdxyz

time stamp with version in source name:-
2010/10/01 10:33:23,343 INFO [2232] abcdxyz

In the second example splunk is taking the version number of the source name hence the date is shifted from 2013/10/01 to 2010/10/01

Re: Extract date from a varying source name

dart — Tue, 17 Dec 2013 10:31:22 GMT

I'd strongly suggest that you get the application to log complete timestamps (ideally in ISO format with timezone).

If you are unable to do so, are you able to remove the date from the filename?

If you are unable to do so, you can try modifying your props.conf like so:

[my_application_source_type]
TIME_FORMAT = %H:%M:%S,%3N
MAX_DAYS_AGO=1

If none of those options are viable, you can just use the current time:

[my_application_source_type]
DATETIME_CONFIG = CURRENT

Re: Extract date from a varying source name

luv — Mon, 28 Sep 2020 15:30:51 GMT

[my_application_source_type]
TIME_FORMAT = %H:%M:%S,%3N
MAX_DAYS_AGO=1

I did the above changes in my props.conf and now splunk is taking the current date for my events. It's still not taking the date from the source name 😞