Splunk Search

TIME_PREFIX

surfjose
New Member

Hi
I have a log-file with diffrent time formats.
Is it possible to extract this diffrent timestamps with TIME_PREFIX in propts.conf?
Loglines:

<Feb 17, 2014 8:32:42 AM CET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING> 
[EL Finest]: 2014-02-17 08:32:45.961--ServerSession(998916174)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--property=eclipselink.logging.level.sql; value=FINEST; translated value=FINEST
[EL Finer]: 2014-02-17 08:32:45.973--ServerSession(998916174)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--Searching for default mapping file in file:/opt/user_projects/domains/tdapp12/servers/tsapp12maportal01/tmp/_WL_user/clientportal/a7daku/APP-INF/lib/sympany.clientportal.jpa.jar
[EL Config]: 2014-02-17 08:32:45.982--ServerSession(998916174)--Thread(Thread[[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads])--The access type for the persistent class [class ch.sympany.clientportal.model.UserPreferences] is set to [FIELD].
2014-02-17 08:32:54,248 [[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] DEBUG ch.sympany.clientportal.servlet.StartupServletContextListener  - StartupServletContextListener initialised.
Tags (1)
0 Karma

surfjose
New Member

It does not work.
It taks findes only the timestamp from this log entry:
2014-02-17 08:32:54,248 [[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] DEBUG ch.sympany.clientportal.servlet.StartupServletContextListener - StartupServletContextListener initialised.

The best filter i have create is this one:

[my_sourcetype]
MAX_TIMESTAMP_LOOKAHEAD = 50
TIME_PREFIX = \[EL Finest]:|\[EL Finer]:|\[EL Config]: |\<

But the result is not 100%.

Any ideas in this way?

0 Karma

kristian_kolb
Ultra Champion

Well TIME_PREFIX is used to ensure that you find the start of the timestamp in the events, whereas TIME_FORMAT will allow you to specify how the timstamp is formatted.

In your case, you seem to be having three different time formats, but the first seems to be of less interest (service started?). The second two differ very little, and if you can make do without the millisecond precision, you could probably have something like this config (in props.conf);

[your_sourcetype]
MAX_TIMESTAMP_LOOKAHEAD = 50
TIME_FORMAT = %Y-%m-%d %H:%M:%S

Unfortunately, TIME_FORMAT does not take a regex as parameter value, otherwise it would have been easy to handle the dot/comma difference.


UPDATE:

Perhaps you could also add the TIME_PREFIX as such;

TIME_PREFIX = (:\s|<|^)

The TIME_PREFIX will match the first instance it finds of the pattern, and will start looking for a timestamp immediately after it (which must be formatted according to the TIME_FORMAT)

Not sure it will have any success on the first message.

/K

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...