It's crashing on the maintailingthread while reading the disk_objects.log. This is on Windows and the crash looks like this:
Crashing thread: MainTailingThread for WatchedTailFile-WatchedFileState: path="C:\Program Files\Splunk\var\log\introspection\disk_objects.log [build 206881] 2014-05-09 04:01:09 Access violation, cannot read at address [0x000079646F626F00] Exception address: [0x0000000140614B21] Crashing thread: MainTailingThread MxCsr: [0x0000000000001FA0] SegDs: [0x000000000000002B] SegEs: [0x000000000000002B] SegFs: [0x0000000000000053] SegGs: [0x000000000000002B] SegSs: [0x000000000000002B] SegCs: [0x0000000000000033] EFlags: [0x0000000000010206] Rsp: [0x000000001192D1D0] Rip: [0x0000000140614B21] ? Dr0: [0x0000000000000000] Dr1: [0x0000000000000000] Dr2: [0x0000000000000000] Dr3: [0x0000000000000000] Dr6: [0x0000000000000000] Dr7: [0x0000000000000000] Rax: [0x000079646F626F00] Rcx: [0x0000000022C11268] Rdx: [0x000000001192E368] Rbx: [0x000000001192E2C0] Rbp: [0x0000000000000000] Rsi: [0x000000001192E368] Rdi: [0x0000000000000000] R8: [0x000000001192E2C0] R9: [0x0000000000000000] R10: [0x000000004E584490] R11: [0x000000004E584990] R12: [0x000000001192E420] R13: [0x0000000000000500] R14: [0x0000000022C11268] R15: [0x0000000000000000] DebugControl: [0x0000000000000000] LastBranchToRip: [0x0000000000000000] LastBranchFromRip: [0x0000000000000000] LastExceptionToRip: [0x0000000000000000] LastExceptionFromRip: [0x0000000000000000] OS: Windows Arch: x86-64
This issue has been identified as a bug affecting Splunk Enterprise 6.1 and 6.1.1 on Windows platforms only - the reference is SPL-83975.
It is triggered by the use of JSON-based indexed field extractions during the acquisition of two native files:
%SPLUNK_HOME%\var\log\introspection\resource_usage.log
%SPLUNK_HOME%\var\log\introspection\disk_objects.log
We hope to fix this issue in an upcoming maintenance release very soon, but in the meantime you can work-around it by adding the following configuration to %SPLUNK_HOME%\etc\system\local\props.conf
:
[splunk_disk_objects] INDEXED_EXTRACTIONS = [splunk_resource_usage] INDEXED_EXTRACTIONS =
This will suppress the INDEXED_EXTRACTIONS = json
directive that is applied by default to these sourcetypes in %SPLUNK_HOME%\etc\system\default\props.conf
.
This issue has been identified as a bug affecting Splunk Enterprise 6.1 and 6.1.1 on Windows platforms only - the reference is SPL-83975.
It is triggered by the use of JSON-based indexed field extractions during the acquisition of two native files:
%SPLUNK_HOME%\var\log\introspection\resource_usage.log
%SPLUNK_HOME%\var\log\introspection\disk_objects.log
We hope to fix this issue in an upcoming maintenance release very soon, but in the meantime you can work-around it by adding the following configuration to %SPLUNK_HOME%\etc\system\local\props.conf
:
[splunk_disk_objects] INDEXED_EXTRACTIONS = [splunk_resource_usage] INDEXED_EXTRACTIONS =
This will suppress the INDEXED_EXTRACTIONS = json
directive that is applied by default to these sourcetypes in %SPLUNK_HOME%\etc\system\default\props.conf
.
SPL-83975 was reported under Windows but also seen in some instances on Linux. This bug has been fixed as of 6.1.4 as referenced here