Splunk Search

Time of event off by three days

torustad
Path Finder

In the logfile (server.log from GlassFish):

[#|2011-05-16T17:13:37.622+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=61;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction
javax.transaction.xa.XAException: java.sql.SQLException: Failed to enlist:Connection reset by peer: socket write error

Searchresult:

5/13/11 12:57:01.000 PM

[#|2011-05-16T17:13:37.622+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=61;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction
javax.transaction.xa.XAException: java.sql.SQLException: Failed to enlist:Connection reset by peer: socket write error

IE Actual time i loggfile is 2011-05-16T17:13:37.622+0200
Time in searchresult is 5/13/11 12:57:01.000 PM


I think it is usually correct:

5/13/11 11:54:46.995 AM

[#|2011-05-13T11:54:46.995+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=62;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction

Why does this happen?

Thanks and regards,
Bård Tørustad
Research Council of Norway

Tags (3)

torustad
Path Finder

Hi and thank you for the response!

I (think) I have done as you wrote and as described by Splunk:

-- In ...\Splunk\etc\apps\launcher\local\inputs.conf

[monitor://\NFR-GF-EP02.nfr.prod\logs\glassfish\server.log]
disabled = false
followTail = 0
host = NFR-GF-EP02.nfr.prod
sourcetype = glassfish

-- In ...\Splunk\etc\system\local\props.conf
ONLY the following 4 lines in this file:

[glassfish]
TIME_PREFIX=^[#|
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=27

-- Is that the definition of the sourcetype "glassfish"? If so, shouldn't it appear in the list of sourcetypes when a new input is defined?

-- However

5/17/11 12:14:27.000 PM

[#|2011-05-19T16:05:29.491+0200|INFO|glassfish3.0.1|javax.enterprise.system.tools.admin.org.glassfish.server|_ThreadID=4119;_ThreadName=Thread-1;|BootAMXListener: connection made for service:jmx:rmi://....:9686/jndi/rmi://.....:9686/jmxrmi, booting AMX MBeans|#]
host=...... Options|

sourcetype=glassfish Options|

source=....\logs\glassfish\server.log Options|

...

The timestamp Splunk thinks is constantly at "5/17/11 12:14:27.000 PM" whereas the the actual timestamp is correct; in this case "2011-05-19T16:05:29.491".

Am I modifying the correct files?

Thanks and regards,
Bård

0 Karma

torustad
Path Finder

Yes, I have restarted Splunk.

Have I done the changes in the correct files (their names became illegible above - it is a Windows-server):

.../Splunk/etc/system/local/props.conf
.../Splunk/etc/apps/launcher/local/inputs.conf

Thanks and regards,
Bård

0 Karma

dwaddle
SplunkTrust
SplunkTrust

That is not the definition of the sourcetype glassfish as much as a definition of what to do with data OF the sourcetype glassfish. It appears like you have done the right thing by setting the sourcetype of your input in inputs.conf to glassfish. Did you restart splunk after making these changes?

0 Karma

dwaddle
SplunkTrust
SplunkTrust

I would look into TIME_FORMAT for this source/sourcetype. Setting TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD, and TIME_PREFIX are all useful tools for being sure that Splunk is properly parsing your time. Based on your example events, I would probably configure something like this in props.conf.

[glassfish]
TIME_PREFIX=^\[#\|
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=27

The docs cover this well at http://www.splunk.com/base/Documentation/4.2.1/Data/Configuretimestamprecognition

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...