Time of event off by three days

[torustad]

In the logfile (server.log from GlassFish):

[#|2011-05-16T17:13:37.622+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=61;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction
javax.transaction.xa.XAException: java.sql.SQLException: Failed to enlist:Connection reset by peer: socket write error

Searchresult:

5/13/11 12:57:01.000 PM

[#|2011-05-16T17:13:37.622+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=61;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction
javax.transaction.xa.XAException: java.sql.SQLException: Failed to enlist:Connection reset by peer: socket write error

IE Actual time i loggfile is 2011-05-16T17:13:37.622+0200
Time in searchresult is 5/13/11 12:57:01.000 PM

---

I think it is usually correct:

5/13/11 11:54:46.995 AM

[#|2011-05-13T11:54:46.995+0200|WARNING|glassfish3.0.1|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=62;_ThreadName=Thread-1;|JTS5041: The resource manager is doing work outside a global transaction

Why does this happen?

Thanks and regards,
Bård Tørustad
Research Council of Norway

[torustad]

Hi and thank you for the response!

I (think) I have done as you wrote and as described by Splunk:

-- In ...\Splunk\etc\apps\launcher\local\inputs.conf

[monitor://\NFR-GF-EP02.nfr.prod\logs\glassfish\server.log]
disabled = false
followTail = 0
host = NFR-GF-EP02.nfr.prod
sourcetype = glassfish

-- In ...\Splunk\etc\system\local\props.conf
ONLY the following 4 lines in this file:

[glassfish]
TIME_PREFIX=^[#|
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=27

-- Is that the definition of the sourcetype "glassfish"? If so, shouldn't it appear in the list of sourcetypes when a new input is defined?

-- However

5/17/11 12:14:27.000 PM

[#|2011-05-19T16:05:29.491+0200|INFO|glassfish3.0.1|javax.enterprise.system.tools.admin.org.glassfish.server|_ThreadID=4119;_ThreadName=Thread-1;|BootAMXListener: connection made for service:jmx:rmi://....:9686/jndi/rmi://.....:9686/jmxrmi, booting AMX MBeans|#]
host=...... Options|

sourcetype=glassfish Options|

source=....\logs\glassfish\server.log Options|

...

The timestamp Splunk thinks is constantly at "5/17/11 12:14:27.000 PM" whereas the the actual timestamp is correct; in this case "2011-05-19T16:05:29.491".

Am I modifying the correct files?

Thanks and regards,
Bård

[torustad]

Yes, I have restarted Splunk.

Have I done the changes in the correct files (their names became illegible above - it is a Windows-server):

.../Splunk/etc/system/local/props.conf
.../Splunk/etc/apps/launcher/local/inputs.conf

Thanks and regards,
Bård

[dwaddle]

That is not the definition of the sourcetype glassfish as much as a definition of what to do with data OF the sourcetype glassfish. It appears like you have done the right thing by setting the sourcetype of your input in inputs.conf to glassfish. Did you restart splunk after making these changes?

[dwaddle]

I would look into TIME_FORMAT for this source/sourcetype. Setting TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD, and TIME_PREFIX are all useful tools for being sure that Splunk is properly parsing your time. Based on your example events, I would probably configure something like this in props.conf.

[glassfish]
TIME_PREFIX=^\[#\|
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=27

The docs cover this well at http://www.splunk.com/base/Documentation/4.2.1/Data/Configuretimestamprecognition