Splunk Search

TIME_PREFIX

surfjose
New Member

Hi
I have a log-file with diffrent time formats.
Is it possible to extract this diffrent timestamps with TIME_PREFIX in propts.conf?
Loglines:

<Feb 17, 2014 8:32:42 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING> 
[EL Finest]: 2014-02-17 08:32:45.961--ServerSession(998916174)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--property=eclipselink.logging.level.sql; value=FINEST; translated value=FINEST
[EL Finer]: 2014-02-17 08:32:45.973--ServerSession(998916174)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--Searching for default mapping file in file:/opt/user_projects/domains/tdapp12/servers/tsapp12maportal01/tmp/_WL_user/clientportal/a7daku/APP-INF/lib/sympany.clientportal.jpa.jar
[EL Config]: 2014-02-17 08:32:45.982--ServerSession(998916174)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--The access type for the persistent class [class ch.sympany.clientportal.model.UserPreferences] is set to [FIELD].
2014-02-17 08:32:54,248 [[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] DEBUG ch.sympany.clientportal.servlet.StartupServletContextListener  - StartupServletContextListener initialised.
Tags (1)
0 Karma

surfjose
New Member

It does not work.
It taks findes only the timestamp from this log entry:
2014-02-17 08:32:54,248 [[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] DEBUG ch.sympany.clientportal.servlet.StartupServletContextListener - StartupServletContextListener initialised.

The best filter i have create is this one:

[my_sourcetype]
MAX_TIMESTAMP_LOOKAHEAD = 50
TIME_PREFIX = \[EL Finest]:|\[EL Finer]:|\[EL Config]: |\<

But the result is not 100%.

Any ideas in this way?

0 Karma

kristian_kolb
Ultra Champion

Well TIME_PREFIX is used to ensure that you find the start of the timestamp in the events, whereas TIME_FORMAT will allow you to specify how the timstamp is formatted.

In your case, you seem to be having three different time formats, but the first seems to be of less interest (service started?). The second two differ very little, and if you can make do without the millisecond precision, you could probably have something like this config (in props.conf);

[your_sourcetype]
MAX_TIMESTAMP_LOOKAHEAD = 50
TIME_FORMAT = %Y-%m-%d %H:%M:%S

Unfortunately, TIME_FORMAT does not take a regex as parameter value, otherwise it would have been easy to handle the dot/comma difference.


UPDATE:

Perhaps you could also add the TIME_PREFIX as such;

TIME_PREFIX = (:\s|<|^)

The TIME_PREFIX will match the first instance it finds of the pattern, and will start looking for a timestamp immediately after it (which must be formatted according to the TIME_FORMAT)

Not sure it will have any success on the first message.

/K

0 Karma
Get Updates on the Splunk Community!

Splunk Cloud | Empowering Splunk Administrators with Admin Config Service (ACS)

Greetings, Splunk Cloud Admins and Splunk enthusiasts! The Admin Configuration Service (ACS) team is excited ...

Tech Talk | One Log to Rule Them All

One log to rule them all: how you can centralize your troubleshooting with Splunk logs We know how important ...

Splunk Security Content for Threat Detection & Response, Q1 Roundup

Join Principal Threat Researcher, Michael Haag, as he walks through:An introduction to the Splunk Threat ...