https://community.splunk.com/t5/Monitoring-Splunk/Splunk-crashing/m-p/555673#M5068 <P>Hi,</P><P>We recently upgraded from&nbsp;7.3.1 to 8.1.4.&nbsp; Shortly after we have seen Splunkd crashing.&nbsp; The crash log is below.&nbsp; Any help would be greatly appreciated. Thanks&nbsp;</P><P>[build 17f862b42a7c] 2021-06-14 15:24:21<BR />Access violation, cannot read at address [0x00000000000003C0]<BR />Exception address: [0x00007FF6A9059B66]<BR />Crashing thread: indexerPipe<BR />MxCsr: [0x0000000000001FA0]<BR />SegDs: [0x000000000000002B]<BR />SegEs: [0x000000000000002B]<BR />SegFs: [0x0000000000000053]<BR />SegGs: [0x000000000000002B]<BR />SegSs: [0x000000000000002B]<BR />SegCs: [0x0000000000000033]<BR />EFlags: [0x0000000000010202]<BR />Rsp: [0x000000610CAFEED0]<BR />Rip: [0x00007FF6A9059B66] ?<BR />Dr0: [0x0000000000000000]<BR />Dr1: [0x0000000000000000]<BR />Dr2: [0x0000000000000000]<BR />Dr3: [0x0000000000000000]<BR />Dr6: [0x0000000000000000]<BR />Dr7: [0x0000000000000000]<BR />Rax: [0x00000000000003A8]<BR />Rcx: [0x0000000000000000]<BR />Rdx: [0x0000000000000003]<BR />Rbx: [0x00000000000003A8]<BR />Rbp: [0x000000610CAFF030]<BR />Rsi: [0x0000000000000003]<BR />Rdi: [0x000000610CAFF4C8]<BR />R8: [0x000000610CAFF4C8]<BR />R9: [0x0000000000000000]<BR />R10: [0x000000610CAFF000]<BR />R11: [0x00000210282052E0]<BR />R12: [0x0000000000000024]<BR />R13: [0x0000021028205320]<BR />R14: [0x000000610CAFF024]<BR />R15: [0x0000000000000000]<BR />DebugControl: [0x0000021000000001]<BR />LastBranchToRip: [0x0000000000000000]<BR />LastBranchFromRip: [0x0000000000000000]<BR />LastExceptionToRip: [0x0000000000000000]<BR />LastExceptionFromRip: [0x0000000000000000]</P><P>OS: Windows<BR />Arch: x86-64</P><P>Backtrace:<BR />[0x00007FF6A9059B66] ?<BR />Args: [0x0000021028205344] [0x000000610CAFF4C8] [0x00000210256EA5D0]<BR />[0x00007FF6A9033055] ?<BR />Args: [0x0000021028BAB7E0] [0x0000000000000000] [0x0000000060C31E4D]<BR />[0x00007FF6A9032956] ?<BR />Args: [0x000000610CAFF730] [0x000000610CAFFA38] [0x0000000060C31E4D]<BR />[0x00007FF6A9048795] ?<BR />Args: [0x01D75E9B8846C480] [0x00000210266D6ED0] [0x00000210209CF980]<BR />[0x00007FF6A907F4C1] ?<BR />Args: [0x0000000000000000] [0x000000610CAFF780] [0x00007FF6AB85C701]<BR />[0x00007FF6A907EDC5] ?<BR />Args: [0x000000610CAFFA38] [0x000000610CAFFA38] [0x00000210209CF980]<BR />[0x00007FF6A907ECC5] ?<BR />Args: [0x0000000060C774A5] [0x0000000000000001] [0x0000000060C774A5]<BR />[0x00007FF6A908A218] ?<BR />Args: [0x00000210243F2558] [0x000000610CAFFAC0] [0x00000210209CF980]<BR />[0x00007FF6A961CF2E] ?<BR />Args: [0x00000210243F2558] [0x0000000000000001] [0x0000000000000000]<BR />[0x00007FF6A973964D] ?<BR />Args: [0x00000210205AF7D0] [0x00000210205AF7D0] [0x0000000000000000]<BR />[0x00007FF6A9969F5E] ?<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />[0x00007FFA0651FB80] o__realloc_base + 96/528<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />[0x00007FFA069A84D4] BaseThreadInitThunk + 20/1472<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />[0x00007FFA090E1791] RtlUserThreadStart + 33/784<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />Crash dump written to: D:\Splunk\var\log\splunk\D__Splunk_bin_splunkd_exe_crash-2021-06-14-15-24-21.dmp</P><P>Splunk ran as local administrator<BR />SIPR-SPLUNK2 /Windows Server 10.0 or later<BR />GetLastError(): 5<BR />Threads running: 90<BR />Executable module base: 0x0x7ff6a8c40000<BR />Runtime: 41.899370s<BR />argv: [Splunkd -p 8089]<BR />Thread: "indexerPipe", did_join=0, ready_to_run=Y, main_thread=N, token=4180</P><P><BR />x86 CPUID registers:<BR />0: 00000015 756E6547 6C65746E 49656E69<BR />1: 00050654 00010800 FEDA3203 0F8BFBFF<BR />2: 76036301 00F0B6FF 00000000 00C30000<BR />3: 00000000 00000000 00000000 00000000<BR />4: 00000121 01C0003F 0000003F 00000000<BR />5: 00000000 00000000 00000000 00000000<BR />6: 00000000 00000000 00000000 00000000<BR />7: 00000000 D09F6FB9 00000000 9C000400<BR />8: 00000000 00000000 00000000 00000000<BR />9: 00000000 00000000 00000000 00000000<BR />A: 00000000 00000000 00000000 00000000<BR />B: 00000000 00000001 00000100 00000000<BR />C: 00000000 00000000 00000000 00000000<BR /><span class="lia-unicode-emoji" title=":anguished_face:">😧</span> 000000FF 00000A80 00000A80 00000000<BR />E: 00000000 00000000 00000000 00000000<BR />F: 00000000 00000000 00000000 00000000<BR />10: 00000000 00000000 00000000 00000000<BR />11: 00000000 00000000 00000000 00000000<BR />12: 00000000 00000000 00000000 00000000<BR />13: 00000000 00000000 00000000 00000000<BR />14: 00000000 00000000 00000000 00000000<BR />15: 00000000 00000000 00000000 00000000<BR />80000000: 80000008 00000000 00000000 00000000<BR />80000001: 00000000 00000000 00000121 2C100800<BR />80000002: 65746E49 2952286C 6F655820 2952286E<BR />80000003: 6C695320 20726576 30313134 55504320<BR />80000004: 32204020 4730312E 00007A48 00000000<BR />80000005: 00000000 00000000 00000000 00000000<BR />80000006: 00000000 00000000 01006040 00000000<BR />80000007: 00000000 00000000 00000000 00000000<BR />80000008: 0000302C 00000000 00000000 00000000<BR />terminating...</P> Mon, 14 Jun 2021 16:16:23 GMT steveo2 2021-06-14T16:16:23Z https://community.splunk.com/t5/Monitoring-Splunk/Splunk-crashing/m-p/555673#M5068 <P>Hi,</P><P>We recently upgraded from&nbsp;7.3.1 to 8.1.4.&nbsp; Shortly after we have seen Splunkd crashing.&nbsp; The crash log is below.&nbsp; Any help would be greatly appreciated. Thanks&nbsp;</P><P>[build 17f862b42a7c] 2021-06-14 15:24:21<BR />Access violation, cannot read at address [0x00000000000003C0]<BR />Exception address: [0x00007FF6A9059B66]<BR />Crashing thread: indexerPipe<BR />MxCsr: [0x0000000000001FA0]<BR />SegDs: [0x000000000000002B]<BR />SegEs: [0x000000000000002B]<BR />SegFs: [0x0000000000000053]<BR />SegGs: [0x000000000000002B]<BR />SegSs: [0x000000000000002B]<BR />SegCs: [0x0000000000000033]<BR />EFlags: [0x0000000000010202]<BR />Rsp: [0x000000610CAFEED0]<BR />Rip: [0x00007FF6A9059B66] ?<BR />Dr0: [0x0000000000000000]<BR />Dr1: [0x0000000000000000]<BR />Dr2: [0x0000000000000000]<BR />Dr3: [0x0000000000000000]<BR />Dr6: [0x0000000000000000]<BR />Dr7: [0x0000000000000000]<BR />Rax: [0x00000000000003A8]<BR />Rcx: [0x0000000000000000]<BR />Rdx: [0x0000000000000003]<BR />Rbx: [0x00000000000003A8]<BR />Rbp: [0x000000610CAFF030]<BR />Rsi: [0x0000000000000003]<BR />Rdi: [0x000000610CAFF4C8]<BR />R8: [0x000000610CAFF4C8]<BR />R9: [0x0000000000000000]<BR />R10: [0x000000610CAFF000]<BR />R11: [0x00000210282052E0]<BR />R12: [0x0000000000000024]<BR />R13: [0x0000021028205320]<BR />R14: [0x000000610CAFF024]<BR />R15: [0x0000000000000000]<BR />DebugControl: [0x0000021000000001]<BR />LastBranchToRip: [0x0000000000000000]<BR />LastBranchFromRip: [0x0000000000000000]<BR />LastExceptionToRip: [0x0000000000000000]<BR />LastExceptionFromRip: [0x0000000000000000]</P><P>OS: Windows<BR />Arch: x86-64</P><P>Backtrace:<BR />[0x00007FF6A9059B66] ?<BR />Args: [0x0000021028205344] [0x000000610CAFF4C8] [0x00000210256EA5D0]<BR />[0x00007FF6A9033055] ?<BR />Args: [0x0000021028BAB7E0] [0x0000000000000000] [0x0000000060C31E4D]<BR />[0x00007FF6A9032956] ?<BR />Args: [0x000000610CAFF730] [0x000000610CAFFA38] [0x0000000060C31E4D]<BR />[0x00007FF6A9048795] ?<BR />Args: [0x01D75E9B8846C480] [0x00000210266D6ED0] [0x00000210209CF980]<BR />[0x00007FF6A907F4C1] ?<BR />Args: [0x0000000000000000] [0x000000610CAFF780] [0x00007FF6AB85C701]<BR />[0x00007FF6A907EDC5] ?<BR />Args: [0x000000610CAFFA38] [0x000000610CAFFA38] [0x00000210209CF980]<BR />[0x00007FF6A907ECC5] ?<BR />Args: [0x0000000060C774A5] [0x0000000000000001] [0x0000000060C774A5]<BR />[0x00007FF6A908A218] ?<BR />Args: [0x00000210243F2558] [0x000000610CAFFAC0] [0x00000210209CF980]<BR />[0x00007FF6A961CF2E] ?<BR />Args: [0x00000210243F2558] [0x0000000000000001] [0x0000000000000000]<BR />[0x00007FF6A973964D] ?<BR />Args: [0x00000210205AF7D0] [0x00000210205AF7D0] [0x0000000000000000]<BR />[0x00007FF6A9969F5E] ?<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />[0x00007FFA0651FB80] o__realloc_base + 96/528<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />[0x00007FFA069A84D4] BaseThreadInitThunk + 20/1472<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />[0x00007FFA090E1791] RtlUserThreadStart + 33/784<BR />Args: [0x0000000000000000] [0x0000000000000000] [0x0000000000000000]<BR />Crash dump written to: D:\Splunk\var\log\splunk\D__Splunk_bin_splunkd_exe_crash-2021-06-14-15-24-21.dmp</P><P>Splunk ran as local administrator<BR />SIPR-SPLUNK2 /Windows Server 10.0 or later<BR />GetLastError(): 5<BR />Threads running: 90<BR />Executable module base: 0x0x7ff6a8c40000<BR />Runtime: 41.899370s<BR />argv: [Splunkd -p 8089]<BR />Thread: "indexerPipe", did_join=0, ready_to_run=Y, main_thread=N, token=4180</P><P><BR />x86 CPUID registers:<BR />0: 00000015 756E6547 6C65746E 49656E69<BR />1: 00050654 00010800 FEDA3203 0F8BFBFF<BR />2: 76036301 00F0B6FF 00000000 00C30000<BR />3: 00000000 00000000 00000000 00000000<BR />4: 00000121 01C0003F 0000003F 00000000<BR />5: 00000000 00000000 00000000 00000000<BR />6: 00000000 00000000 00000000 00000000<BR />7: 00000000 D09F6FB9 00000000 9C000400<BR />8: 00000000 00000000 00000000 00000000<BR />9: 00000000 00000000 00000000 00000000<BR />A: 00000000 00000000 00000000 00000000<BR />B: 00000000 00000001 00000100 00000000<BR />C: 00000000 00000000 00000000 00000000<BR /><span class="lia-unicode-emoji" title=":anguished_face:">😧</span> 000000FF 00000A80 00000A80 00000000<BR />E: 00000000 00000000 00000000 00000000<BR />F: 00000000 00000000 00000000 00000000<BR />10: 00000000 00000000 00000000 00000000<BR />11: 00000000 00000000 00000000 00000000<BR />12: 00000000 00000000 00000000 00000000<BR />13: 00000000 00000000 00000000 00000000<BR />14: 00000000 00000000 00000000 00000000<BR />15: 00000000 00000000 00000000 00000000<BR />80000000: 80000008 00000000 00000000 00000000<BR />80000001: 00000000 00000000 00000121 2C100800<BR />80000002: 65746E49 2952286C 6F655820 2952286E<BR />80000003: 6C695320 20726576 30313134 55504320<BR />80000004: 32204020 4730312E 00007A48 00000000<BR />80000005: 00000000 00000000 00000000 00000000<BR />80000006: 00000000 00000000 01006040 00000000<BR />80000007: 00000000 00000000 00000000 00000000<BR />80000008: 0000302C 00000000 00000000 00000000<BR />terminating...</P> Mon, 14 Jun 2021 16:16:23 GMT https://community.splunk.com/t5/Monitoring-Splunk/Splunk-crashing/m-p/555673#M5068 steveo2 2021-06-14T16:16:23Z https://community.splunk.com/t5/Monitoring-Splunk/Splunk-crashing/m-p/555683#M5069 <P>Open a support case with Splunk.</P> Mon, 14 Jun 2021 17:54:56 GMT https://community.splunk.com/t5/Monitoring-Splunk/Splunk-crashing/m-p/555683#M5069 richgalloway 2021-06-14T17:54:56Z