Splunk Search

Time of event off by three days

Path Finder

In the logfile (server.log from GlassFish):

[#|2011-05-16T17:13:37.622+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=61;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction
javax.transaction.xa.XAException: java.sql.SQLException: Failed to enlist:Connection reset by peer: socket write error

Searchresult:

5/13/11 12:57:01.000 PM

[#|2011-05-16T17:13:37.622+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=61;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction
javax.transaction.xa.XAException: java.sql.SQLException: Failed to enlist:Connection reset by peer: socket write error

IE Actual time i loggfile is 2011-05-16T17:13:37.622+0200
Time in searchresult is 5/13/11 12:57:01.000 PM


I think it is usually correct:

5/13/11 11:54:46.995 AM

[#|2011-05-13T11:54:46.995+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=62;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction

Why does this happen?

Thanks and regards,
Bård Tørustad
Research Council of Norway

Tags (3)

Path Finder

Hi and thank you for the response!

I (think) I have done as you wrote and as described by Splunk:

-- In ...\Splunk\etc\apps\launcher\local\inputs.conf

[monitor://\NFR-GF-EP02.nfr.prod\logs\glassfish\server.log]
disabled = false
followTail = 0
host = NFR-GF-EP02.nfr.prod
sourcetype = glassfish

-- In ...\Splunk\etc\system\local\props.conf
ONLY the following 4 lines in this file:

[glassfish]
TIME_PREFIX=^[#|
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=27

-- Is that the definition of the sourcetype "glassfish"? If so, shouldn't it appear in the list of sourcetypes when a new input is defined?

-- However

5/17/11 12:14:27.000 PM

[#|2011-05-19T16:05:29.491+0200|INFO|glassfish3.0.1|javax.enterprise.system.tools.admin.org.glassfish.server|_ThreadID=4119;_ThreadName=Thread-1;|BootAMXListener: connection made for service:jmx:rmi://....:9686/jndi/rmi://.....:9686/jmxrmi, booting AMX MBeans|#]
host=...... Options|

sourcetype=glassfish Options|

source=....\logs\glassfish\server.log Options|

...

The timestamp Splunk thinks is constantly at "5/17/11 12:14:27.000 PM" whereas the the actual timestamp is correct; in this case "2011-05-19T16:05:29.491".

Am I modifying the correct files?

Thanks and regards,
Bård

0 Karma

Path Finder

Yes, I have restarted Splunk.

Have I done the changes in the correct files (their names became illegible above - it is a Windows-server):

.../Splunk/etc/system/local/props.conf
.../Splunk/etc/apps/launcher/local/inputs.conf

Thanks and regards,
Bård

0 Karma

SplunkTrust
SplunkTrust

That is not the definition of the sourcetype glassfish as much as a definition of what to do with data OF the sourcetype glassfish. It appears like you have done the right thing by setting the sourcetype of your input in inputs.conf to glassfish. Did you restart splunk after making these changes?

0 Karma

SplunkTrust
SplunkTrust

I would look into TIME_FORMAT for this source/sourcetype. Setting TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD, and TIME_PREFIX are all useful tools for being sure that Splunk is properly parsing your time. Based on your example events, I would probably configure something like this in props.conf.

[glassfish]
TIME_PREFIX=^\[#\|
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=27

The docs cover this well at http://www.splunk.com/base/Documentation/4.2.1/Data/Configuretimestamprecognition

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!