Getting Data In

Two dates in one log event is being split into two events

jlamble1
New Member

Hi,

I've ran in to a problem with an event that has a timestamp but also has another timestamp in the log itself.

For example;

17 Jun 2011 13:59:13,691 DEBUG com.message.MessageProcessor:96 - <EventPublishMessage>
  <oId>123146341</oId>
  <messageDate>2011-06-17 13:59:13.685 EST</messageDate>
  <userId>console</userId>
  <notes>None</notes>
</EventPublishMessage>

There are two events created,
the first being

17 Jun 2011 13:59:13,691 DEBUG com.message.MessageProcessor:96 - <EventPublishMessage>
  <oId>123146341</oId>
  <messageDate>

and the second

2011-06-17 13:59:13.685 EST</messageDate>
  <userId>console</userId>
  <notes>None</notes>
</EventPublishMessage>

Does anyone know how I can work around this problem?

Cheers,
Jason

Tags (5)
0 Karma

Ayn
Legend

You can solve this by specifying the time format Splunk should be looking for in props.conf. Right now in your installation Splunk breaks out a new event whenever it sees something that can be interpreted as a timestamp (which is the default behaviour), so by explicitly telling Splunk what the timestamp looks like it will only look for timestamps in that format.

Something like this should go into your props.conf (like $SPLUNK_HOME/etc/system/local/props.conf):

[yoursourcetype]
TIME_FORMAT = %d %b %Y %T

...where you should exchange yoursourcetype for the actual name of the sourcetype you're using for these logs. This will match "17 Jun 2011 13:59:13" but not "2011-06-17 13:59:13.685 EST", so Splunk will not break the event.

0 Karma
Get Updates on the Splunk Community!

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...