topic Re: Two dates in one log event is being split into two events in Getting Data In

Two dates in one log event is being split into two events

jlamble1 — Fri, 17 Jun 2011 05:22:24 GMT

Hi,

I've ran in to a problem with an event that has a timestamp but also has another timestamp in the log itself.

For example;

17 Jun 2011 13:59:13,691 DEBUG com.message.MessageProcessor:96 - <EventPublishMessage>
  <oId>123146341</oId>
  <messageDate>2011-06-17 13:59:13.685 EST</messageDate>
  <userId>console</userId>
  <notes>None</notes>
</EventPublishMessage>

There are two events created,
the first being

17 Jun 2011 13:59:13,691 DEBUG com.message.MessageProcessor:96 - <EventPublishMessage>
  <oId>123146341</oId>
  <messageDate>

and the second

2011-06-17 13:59:13.685 EST</messageDate>
  <userId>console</userId>
  <notes>None</notes>
</EventPublishMessage>

Does anyone know how I can work around this problem?

Cheers,
Jason

Re: Two dates in one log event is being split into two events

Ayn — Fri, 17 Jun 2011 07:14:54 GMT

You can solve this by specifying the time format Splunk should be looking for in props.conf. Right now in your installation Splunk breaks out a new event whenever it sees something that can be interpreted as a timestamp (which is the default behaviour), so by explicitly telling Splunk what the timestamp looks like it will only look for timestamps in that format.

Something like this should go into your props.conf (like $SPLUNK_HOME/etc/system/local/props.conf):

[yoursourcetype]
TIME_FORMAT = %d %b %Y %T

...where you should exchange yoursourcetype for the actual name of the sourcetype you're using for these logs. This will match "17 Jun 2011 13:59:13" but not "2011-06-17 13:59:13.685 EST", so Splunk will not break the event.