Installation

Splunk crash after upgrade from 6.6.3 to 7.1.4

vishaltaneja070
Motivator

Splunk Keep crashing. Below is crash log

[build 5a7a840afcb3] 2019-02-12 02:31:45
 Access violation, cannot read at address [0xFFFFFFFFFFFFFFF0]
 Exception address: [0x00007FF7B0A544AD]
 Crashing thread: SchedulerThread
    MxCsr:  [0x0000000000001FA0]
    SegDs:  [0x000000000000002B]
    SegEs:  [0x000000000000002B]
    SegFs:  [0x0000000000000053]
    SegGs:  [0x000000000000002B]
    SegSs:  [0x000000000000002B]
    SegCs:  [0x0000000000000033]
    EFlags:  [0x0000000000010282]
    Rsp:  [0x000000CF468F99D0]
    Rip:  [0x00007FF7B0A544AD] ?
    Dr0:  [0x0000000000000000]
    Dr1:  [0x0000000000000000]
    Dr2:  [0x0000000000000000]
    Dr3:  [0x0000000000000000]
    Dr6:  [0x0000000000000000]
    Dr7:  [0x0000000000000000]
    Rax:  [0x0000000000000001]
    Rcx:  [0xFFFFFFFFFFFFFFE0]
    Rdx:  [0x00007F286B5DDBE8]
    Rbx:  [0x000000CF4D0E2130]
    Rbp:  [0x000000CF468FA850]
    Rsi:  [0x000000CF53457701]
    Rdi:  [0x0000000000000000]
    R8:  [0x0000000000000000]
    R9:  [0x0000000000008000]
    R10:  [0x00007FF7B1ED7718]
    R11:  [0x000000CF468F9B30]
    R12:  [0x000000CF468F9D60]
    R13:  [0x0000000000000002]
    R14:  [0x000000CF468F9DB0]
    R15:  [0x0000000000000000]
    DebugControl:  [0x000000CF52EF64E0]
    LastBranchToRip:  [0x0000000000000000]
    LastBranchFromRip:  [0x0000000000000000]
    LastExceptionToRip:  [0x0000000000000000]
    LastExceptionFromRip:  [0x0000000000000000]

 OS: Windows
 Arch: x86-64

 Backtrace:
  [0x00007FF7B0A544AD] ?
Args:  [0x000000CF53575320]  [0x000000CF468F9E40]  [0x0000000000000000]
  [0x00007FF7B0A538FD] ?
Args:  [0x000000CF468FA820]  [0x000000CF53457710]  [0x000000CF468FA850]
  [0x00007FF7B0B74EC3] ?
Args:  [0x0000000000000000]  [0x00007FF7B117F684]  [0x000000CF4AE84080]
  [0x00007FF7B0B73E3B] ?
Args:  [0x0000000000000000]  [0x000000CF4AE84080]  [0x0000000000000000]
  [0x00007FF7B161259E] ?
Args:  [0x0000000000000000]  [0x0000000000000000]  [0x000000CF468FD598]
  [0x00007FF7B16DF9CC] ?
Args:  [0x0000000000000000]  [0x000000CF468FD7F0]  [0x0000000000000000]
  [0x00007FF7B0C6DA42] ?
Args:  [0x000000CF468FD870]  [0x0000000000000000]  [0x0000000000000000]
  [0x00007FF7B0C73D0A] ?
Args:  [0xFFFFFFFFFFFFFFFE]  [0x00007FF7B0A4FB5F]  [0x000000CF49A2DD70]
  [0x00007FF7B0C737C4] ?
Args:  [0x0000000000000000]  [0x000000CF4B258C60]  [0x000000CF4B258D40]
  [0x00007FF7B0C85C1F] ?
Args:  [0x000000CF4066ABF0]  [0x000000CF468FED98]  [0x0000000000000005]
  [0x00007FF7B0C96BDC] ?
Args:  [0x000000CF468FEBB8]  [0x00007FF7B1F2C2A0]  [0x0000000000000000]
  [0x00007FF7B0CA4F8D] ?
Args:  [0x000000CF40797880]  [0x00007FF7B1F2C460]  [0x0000000000000000]
  [0x00007FF7B0C9F9A5] ?
Args:  [0x000000CF468FEF38]  [0x000000CF4066ABF0]  [0x000000CF4066ABF8]
  [0x00007FF7B0CB400B] ?
Args:  [0x000000CF468FF2A8]  [0x000000CF468FF490]  [0x000000CF4066ABF0]
  [0x00007FF7B114BF3F] ?
Args:  [0x0000000000000000]  [0x000000000AD2CFFE]  [0x0000000000007530]
  [0x00007FF7B114F79C] ?
Args:  [0x000000CF3E811028]  [0x000000000AD2CF54]  [0x0000000000000000]
  [0x00007FF7B0CAA9E5] ?
Args:  [0x000000CF4162C190]  [0x000000CF4162C190]  [0x000000CF3EC28CF0]
  [0x00007FF7B1136C82] ?
Args:  [0x0000000000000000]  [0x00007FF99CE5BDC0]  [0x00007FF99CE5BDC0]
  [0x00007FF99CE5BE1D] crt_at_quick_exit + 125/784
Args:  [0x00007FF99CE5BDC0]  [0x000000CF3EC28CF0]  [0x0000000000000000]
  [0x00007FF9BDC813D2] BaseThreadInitThunk + 34/96
Args:  [0x00007FF9BDC813B0]  [0x0000000000000000]  [0x0000000000000000]
  [0x00007FF9BE0254F4] RtlUserThreadStart + 52/1008
Args:  [0x0000000000000000]  [0x0000000000000000]  [0x0000000000000000]
 Crash dump written to: D:\Splunk\var\log\splunk\D__Splunk_bin_splunkd_exe_crash-2019-02-12-02-31-45.dmp

Splunk ran as local administrator
Windows Server 2012 R2
GetLastError(): 0
Threads running: 75
Executable module base: 0x00007FF7B06D0000
Runtime: 74.515457s
argv: [Splunkd -p 8089]
Thread: "SchedulerThread", did_join=0, ready_to_run=Y, main_thread=N
First 4 bytes of Thread token @0xcf4162c1a4:
00000000  94 06 00 00                                       |....|
00000004

First 512 bytes of Timeout object @0xcf4066abf0:
00000000  30 bc f2 b1 f7 7f 00 00  a0 71 f1 40 cf 00 00 00  |0........q.@....|
00000010  a0 ef 8f 46 cf 00 00 00  00 00 00 00 00 00 00 00  |...F............|
00000020  00 00 00 00 00 00 00 00  54 cf d2 0a 00 00 00 00  |........T.......|
00000030  30 75 00 00 00 00 00 00  a0 ab fb 40 cf 00 00 00  |0u.........@....|
00000040  b0 86 da 3f cf 00 00 00  c0 99 31 3f cf 00 00 00  |...?......1?....|
00000050  69 00 63 00 73 00 5c 00  10 00 00 00 00 00 00 00  |i.c.s.\.........|
00000060  1f 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  01 00 5f 00 61 00 70 00  ff ff ff ff ff ff ff ff  |.._.a.p.........|
00000080  ff ff ff ff 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 00 00 00 00  d0 07 00 02 00 00 00 00  |................|
000000a0  58 bc f2 b1 f7 7f 00 00  30 db 0b 41 cf 00 00 00  |X.......0..A....|
000000b0  00 00 00 00 00 00 00 00  70 00 61 00 74 00 74 00  |........p.a.t.t.|
000000c0  03 00 00 00 6f 00 6e 00  a0 71 f1 40 cf 00 00 00  |....o.n..q.@....|
000000d0  d0 73 ff 3f cf 00 00 00  49 00 00 00 00 00 00 00  |.s.?....I.......|
000000e0  ff ff ff ff ff ff ff ff  ff ff ff ff 00 00 00 00  |................|
000000f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000100  d0 07 00 02 00 00 00 00  d0 08 7b 40 cf 00 00 00  |..........{@....|
00000110  00 00 00 00 00 00 00 00  a0 82 79 40 cf 00 00 00  |..........y@....|
00000120  00 00 00 00 00 00 00 00  ff ff ff ff ff ff ff ff  |................|
00000130  ff ff ff ff 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 00 00 00 00  d0 07 00 02 00 00 00 00  |................|
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000170  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000190  00 00 00 00 00 00 00 00  01 00 5c 00 61 00 70 00  |..........\.a.p.|
000001a0  60 4f fe 3f cf 00 00 00  20 a1 02 40 cf 00 00 00  |`O.?.... ..@....|
000001b0  08 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001c0  01 00 00 00 00 00 00 00  00 00 00 00 6f 00 6e 00  |............o.n.|
000001d0  00 a4 8a 41 cf 00 00 00  b0 a2 02 40 cf 00 00 00  |...A.......@....|
000001e0  08 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  01 00 00 00 00 00 00 00  00 00 00 00 75 3b 00 80  |............u;..|
00000200
_when = 181587.796, _initialInterval = 30.000

x86 CPUID registers:
         0: 0000000D 756E6547 6C65746E 49656E69
         1: 000306A0 04010800 FEBA2203 0FABFBFF
         2: 76036301 00F0B2FF 00000000 00CA0000
         3: 00000000 00000000 00000000 00000000
         4: 00000121 01C0003F 0000003F 00000000
         5: 00000000 00000000 00000000 00000000
         6: 00000077 00000002 00000009 00000000
         7: 00000000 00000281 00000000 00000000
         8: 00000000 00000000 00000000 00000000
         9: 00000000 00000000 00000000 00000000
         A: 07300401 0000007F 00000000 00000000
         B: 00000000 00000001 00000100 00000004
         C: 00000000 00000000 00000000 00000000
         😧 00000007 00000340 00000340 00000000
  80000000: 80000008 00000000 00000000 00000000
  80000001: 00000000 00000000 00000001 28100800
  80000002: 20202020 6E492020 286C6574 58202952
  80000003: 286E6F65 43202952 45205550 36322D35
  80000004: 76203039 20402032 30302E33 007A4847
  80000005: 00000000 00000000 00000000 00000000
  80000006: 00000000 00000000 01006040 00000000
  80000007: 00000000 00000000 00000000 00000100
  80000008: 00003028 00000000 00000000 00000000
terminating...
Labels (1)
0 Karma

sloshburch
Splunk Employee
Splunk Employee

Did you open a support case?

0 Karma

vishaltaneja070
Motivator

@SloshBurch Yes i have opened waiting for their response.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...