Splunk Crashes on Startup

MHS · ‎09-12-2019

Everything was running fine and then it started crashing. The crash log references the scheduler. Not sure what I can do about this.

[build 088f49762779] 2019-09-09 12:36:00
Received fatal signal 11 (Segmentation fault).
 Cause:
   Unknown signal origin (si_code=128, si_addr=[0x0000000000000000]).
 Crashing thread: SchedulerThread
 Registers:
    RIP:  [0x000055B54ECDF85E] _ZN12FieldAliaser12addAliasListERSt6vectorI9AliasInfoSaIS1_EERK3StrS7_S7_ + 366 (splunkd + 0x1EDF85E)
    RDI:  [0x00007F89755F6C80]
    RSI:  [0x79745F64726F6365]
    RBP:  [0x00007F89755F6CA0]
    RSP:  [0x00007F89755F6BE0]
    RAX:  [0x00007F897584B9E0]
    RBX:  [0x00007F89755F6C80]
    RCX:  [0x00007F89755F6CB0]
    RDX:  [0x0000000000000000]
    R8:  [0x0000000000000001]
    R9:  [0x00007F8988F1D000]
    R10:  [0x0000000000000020]
    R11:  [0x00007F898CC77864]
    R12:  [0x0000000000000000]
    R13:  [0x0000000000000008]
    R14:  [0x00007F897591D280]
    R15:  [0x00007F89755F6DC0]
    EFL:  [0x0000000000010246]
    TRAPNO:  [0x000000000000000D]
    ERR:  [0x0000000000000000]
    CSGSFS:  [0xB5BD000000000033]
    OLDMASK:  [0x0000000000000000]

 OS: Linux
 Arch: x86-64

 Backtrace (PIC build):
  [0x000055B54ECDF85E] _ZN12FieldAliaser12addAliasListERSt6vectorI9AliasInfoSaIS1_EERK3StrS7_S7_ + 366 (splunkd + 0x1EDF85E)
  [0x000055B54ECDFE4F] _ZN12FieldAliaser12addAliasListERK3StrS2_S2_P18SearchResultsFilesb + 111 (splunkd + 0x1EDFE4F)
  [0x000055B54ECE08BE] _ZN12FieldAliaser4initERK20PropertyPagesContextb + 1694 (splunkd + 0x1EE08BE)
  [0x000055B54DC4813B] _ZN14SearchOperator8evalArgsERK10AstOptions + 9131 (splunkd + 0xE4813B)
  [0x000055B54DC421DF] _ZN14SearchOperator8evalArgsER17SearchResultsInfo + 191 (splunkd + 0xE421DF)
  [0x000055B54EAA2F16] _ZN14DispatchThread8evaluateEb + 1974 (splunkd + 0x1CA2F16)
  [0x000055B54ED1CE93] _ZN16SummarizerHelper20getExtendedSummaryIdEP13ScopedPointerI14SearchPipelineER17SearchResultsInfoR3StrS7_S7_S7_RSt6vectorISt4pairIS6_S6_ESaISA_EESD_bb + 579 (splunkd + 0x1F1CE93
)
  [0x000055B54DE59EE4] ? (splunkd + 0x1059EE4)
  [0x000055B54DE5F922] _ZN15SummaryDirector20getExtendedSummaryIdERK11SavedSplunkR9StrVectorS4_S4_RSt6vectorISt4pairI3StrS7_ESaIS8_EESB_bb + 546 (splunkd + 0x105F922)
  [0x000055B54DE60FDD] _ZN15SummaryDirector20getExtendedSummaryIdERK11SavedSplunkR3StrS4_S4_RSt6vectorISt4pairIS3_S3_ESaIS7_EESA_b + 173 (splunkd + 0x1060FDD)
  [0x000055B54DE63375] _ZN15SummaryDirector21toAutoSummarySearchesERK17SavedSplunkVectorRS0_bPK19SavedSearchSelector + 581 (splunkd + 0x1063375)
  [0x000055B54DDE9225] _ZN16SearchDispatcher25appendAutoSummarySearchesElRSt3setI19SavedSearchSelectorSt4lessIS1_ESaIS1_EERm + 421 (splunkd + 0xFE9225)
  [0x000055B54DDFB49F] _ZN16SearchDispatcher27getPrioritizedSavedSearchesEllPSt3setI19SavedSearchSelectorSt4lessIS1_ESaIS1_EEPm + 3167 (splunkd + 0xFFB49F)
  [0x000055B54DDFCF37] _ZN16SearchDispatcher7executeEv + 2535 (splunkd + 0xFFCF37)
  [0x000055B54DDE09F2] _ZN16SchedulerTimeout12when_expiredEP8Interval + 50 (splunkd + 0xFE09F2)
  [0x000055B54E52BD1A] _ZN11TimeoutHeap18runExpiredTimeoutsER13MonotonicTime + 362 (splunkd + 0x172BD1A)
  [0x000055B54E461076] _ZN9EventLoop3runEv + 230 (splunkd + 0x1661076)
  [0x000055B54D8ACCC8] _ZN13SavedSplunker4mainEv + 1960 (splunkd + 0xAACCC8)
  [0x000055B54E529159] _ZN6Thread8callMainEPv + 105 (splunkd + 0x1729159)
  [0x00007F898B95BDD5] ? (libpthread.so.0 + 0x7DD5)
  [0x00007F898B684EAD] clone + 109 (libc.so.6 + 0xFDEAD)
 Linux / navstlsplunk01.navvis.local / 3.10.0-957.10.1.el7.x86_64 / #1 SMP Mon Mar 18 15:06:45 UTC 2019 / x86_64
 /etc/redhat-release: CentOS Linux release 7.6.1810 (Core) 
 glibc version: 2.17
 glibc release: stable
 MAP: 55b54ce00000-55b54fc39000 r-xp 00000000 fd:02 2148546384                 /home/splunk/bin/splunkd
 MAP: 55b54fc39000-55b54fd03000 rw-p 02e38000 fd:02 2148546384                 /home/splunk/bin/splunkd
 MAP: 55b54fd03000-55b54fd99000 rw-p 00000000 00:00 0 
 MAP: 7f88f5bea000-7f88f5bff000 r-xp 00000000 fd:00 33554633                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
 MAP: 7f88f5bff000-7f88f5dfe000 ---p 00015000 fd:00 33554633                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
 MAP: 7f88f5dfe000-7f88f5dff000 r--p 00014000 fd:00 33554633                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
 MAP: 7f88f5dff000-7f88f5e00000 rw-p 00015000 fd:00 33554633                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
 MAP: 7f88f5e00000-7f88f6600000 rw-p 00000000 00:00 0 
 MAP: 7f88f67fe000-7f88f67ff000 ---p 00000000 00:00 0 
 MAP: 7f88f67ff000-7f88f69ff000 rw-p 00000000 00:00 0 
 MAP: 7f88f69ff000-7f88f6a00000 ---p 00000000 00:00 0 
 MAP: 7f88f6a00000-7f88f6c00000 rw-p 00000000 00:00 0 
 MAP: 7f88f6c00000-7f88f6e00000 rw-p 00000000 00:00 0 
 MAP: 7f88f6fff000-7f88f7000000 ---p 00000000 00:00 0 
 MAP: 7f88f7000000-7f88f7200000 rw-p 00000000 00:00 0 
 MAP: 7f88f7200000-7f88fa000000 rw-p 00000000 00:00 0 
 MAP: 7f88fa1ff000-7f88fa200000 ---p 00000000 00:00 0 
 MAP: 7f88fa200000-7f88fa400000 rw-p 00000000 00:00 0 
 MAP: 7f88fa400000-7f88ffa00000 rw-p 00000000 00:00 0 
 MAP: 7f88ffc00000-7f8900000000 rw-p 00000000 00:00 0 
 MAP: 7f89001ff000-7f8900200000 ---p 00000000 00:00 0 
 MAP: 7f8900200000-7f8900400000 rw-p 00000000 00:00 0 
 MAP: 7f8900400000-7f8900a00000 rw-p 00000000 00:00 0 
 MAP: 7f8900bfd000-7f8900bfe000 ---p 00000000 00:00 0 
 MAP: 7f8900bfe000-7f8900dfe000 rw-p 00000000 00:00 0 
 MAP: 7f8900dfe000-7f8900dff000 ---p 00000000 00:00 0 
 MAP: 7f8900dff000-7f8900fff000 rw-p 00000000 00:00 0 
 MAP: 7f8900fff000-7f8901000000 ---p 00000000 00:00 0 
 MAP: 7f8901000000-7f8901200000 rw-p 00000000 00:00 0 
 MAP: 7f8901200000-7f8903600000 rw-p 00000000 00:00 0 
 MAP: 7f89037ff000-7f8903800000 ---p 00000000 00:00 0 
 MAP: 7f8903800000-7f8903c00000 rw-p 00000000 00:00 0 
 MAP: 7f8903c00000-7f8908000000 rw-p 00000000 00:00 0 
 MAP: 7f8908200000-7f8908600000 rw-p 00000000 00:00 0 
 MAP: 7f8908800000-7f8909600000 rw-p 00000000 00:00 0 
 MAP: 7f8909800000-7f890ba00000 rw-p 00000000 00:00 0 
 MAP: 7f890ba00000-7f890be00000 rw-p 00000000 00:00 0 
 MAP: 7f890bfff000-7f890c000000 ---p 00000000 00:00 0 
 MAP: 7f890c000000-7f890c200000 rw-p 00000000 00:00 0 
 MAP: 7f890c200000-7f890c400000 rw-p 00000000 00:00 0 
 MAP: 7f890c5fc000-7f890c5fd000 ---p 00000000 00:00 0 
 MAP: 7f890c5fd000-7f890c7fd000 rw-p 00000000 00:00 0 
 MAP: 7f890c7fd000-7f890c7fe000 ---p 00000000 00:00 0 
 MAP: 7f890c7fe000-7f890c9fe000 rw-p 00000000 00:00 0 
 MAP: 7f890c9fe000-7f890c9ff000 ---p 00000000 00:00 0 
 MAP: 7f890c9ff000-7f890cbff000 rw-p 00000000 00:00 0 
 MAP: 7f890cbff000-7f890cc00000 ---p 00000000 00:00 0 
 MAP: 7f890cc00000-7f890ce00000 rw-p 00000000 00:00 0 
 MAP: 7f890ce00000-7f8910800000 rw-p 00000000 00:00 0 
 MAP: 7f8910a00000-7f8911000000 rw-p 00000000 00:00 0 
 MAP: 7f8911200000-7f8911800000 rw-p 00000000 00:00 0 
 MAP: 7f8911a00000-7f8912000000 rw-p 00000000 00:00 0 
 MAP: 7f89121ff000-7f8912200000 ---p 00000000 00:00 0 
 MAP: 7f8912200000-7f8912600000 rw-p 00000000 00:00 0 
 MAP: 7f8912600000-7f8915400000 rw-p 00000000 00:00 0 
 MAP: 7f8915600000-7f891de00000 rw-p 00000000 00:00 0 
 MAP: 7f891e000000-7f891e600000 rw-p 00000000 00:00 0 
 MAP: 7f891e800000-7f891ea00000 rw-p 00000000 00:00 0 
 MAP: 7f891ec00000-7f891f200000 rw-p 00000000 00:00 0 
 MAP: 7f891f400000-7f891f600000 rw-p 00000000 00:00 0 
 MAP: 7f891f800000-7f891fe00000 rw-p 00000000 00:00 0 
 MAP: 7f8920000000-7f8920800000 rw-p 00000000 00:00 0 
 MAP: 7f8920a00000-7f8921e00000 rw-p 00000000 00:00 0 
 MAP: 7f8922000000-7f8922800000 rw-p 00000000 00:00 0 
 MAP: 7f8922a00000-7f8923000000 rw-p 00000000 00:00 0 
 MAP: 7f8923200000-7f8923600000 rw-p 00000000 00:00 0 
 MAP: 7f8923800000-7f8924000000 rw-p 00000000 00:00 0 
 MAP: 7f8924200000-7f8927600000 rw-p 00000000 00:00 0 
 MAP: 7f8927800000-7f892ae00000 rw-p 00000000 00:00 0 
 MAP: 7f892b000000-7f892c000000 rw-p 00000000 00:00 0 
 MAP: 7f892c200000-7f892c800000 rw-p 00000000 00:00 0 
 MAP: 7f892ca00000-7f892d600000 rw-p 00000000 00:00 0 
 MAP: 7f892d800000-7f892dc00000 rw-p 00000000 00:00 0 
 MAP: 7f892de00000-7f892ec00000 rw-p 00000000 00:00 0 
 MAP: 7f892ee00000-7f892f000000 rw-p 00000000 00:00 0 
 MAP: 7f892f200000-7f8930800000 rw-p 00000000 00:00 0 
 MAP: 7f8930a00000-7f8931800000 rw-p 00000000 00:00 0 
 MAP: 7f8931a00000-7f8932800000 rw-p 00000000 00:00 0 
 MAP: 7f8932a00000-7f8935800000 rw-p 00000000 00:00 0 
 MAP: 7f89359f6000-7f89359f7000 ---p 00000000 00:00 0 
 MAP: 7f89359f7000-7f8935bf7000 rw-p 00000000 00:00 0 
 MAP: 7f8935bf7000-7f8935bf8000 ---p 00000000 00:00 0 
 MAP: 7f8935bf8000-7f8935df8000 rw-p 00000000 00:00 0 
 MAP: 7f8935dff000-7f8935e00000 ---p 00000000 00:00 0 
 MAP: 7f8935e00000-7f8936000000 rw-p 00000000 00:00 0 
 MAP: 7f8936000000-7f8936200000 rw-p 00000000 00:00 0 
 MAP: 7f89363fe000-7f89363ff000 ---p 00000000 00:00 0 
 MAP: 7f89363ff000-7f89365ff000 rw-p 00000000 00:00 0 
 MAP: 7f89365ff000-7f8936600000 ---p 00000000 00:00 0 
 MAP: 7f8936600000-7f8936800000 rw-p 00000000 00:00 0 
 MAP: 7f8936800000-7f8936a00000 rw-p 00000000 00:00 0 
 MAP: 7f8936c00000-7f8937e00000 rw-p 00000000 00:00 0 
 MAP: 7f8937fcd000-7f8937fe3000 r-xp 00000000 fd:00 33561479                   /usr/lib64/libresolv-2.17.so
 MAP: 7f8937fe3000-7f89381e2000 ---p 00016000 fd:00 33561479                   /usr/lib64/libresolv-2.17.so
 MAP: 7f89381e2000-7f89381e3000 r--p 00015000 fd:00 33561479                   /usr/lib64/libresolv-2.17.so
 MAP: 7f89381e3000-7f89381e4000 rw-p 00016000 fd:00 33561479                   /usr/lib64/libresolv-2.17.so
 MAP: 7f89381e4000-7f89381e6000 rw-p 00000000 00:00 0 
 MAP: 7f89381e6000-7f89381eb000 r-xp 00000000 fd:00 33561461                   /usr/lib64/libnss_dns-2.17.so
 MAP: 7f89381eb000-7f89383eb000 ---p 00005000 fd:00 33561461                   /usr/lib64/libnss_dns-2.17.so
 MAP: 7f89383eb000-7f89383ec000 r--p 00005000 fd:00 33561461                   /usr/lib64/libnss_dns-2.17.so
 MAP: 7f89383ec000-7f89383ed000 rw-p 00006000 fd:00 33561461                   /usr/lib64/libnss_dns-2.17.so
 MAP: 7f89383ed000-7f89383f9000 r-xp 00000000 fd:00 33561464                   /usr/lib64/libnss_files-2.17.so
 MAP: 7f89383f9000-7f89385f8000 ---p 0000c000 fd:00 33561464                   /usr/lib64/libnss_files-2.17.so
 MAP: 7f89385f8000-7f89385f9000 r--p 0000b000 fd:00 33561464                   /usr/lib64/libnss_files-2.17.so
 MAP: 7f89385f9000-7f89385fa000 rw-p 0000c000 fd:00 33561464                   /usr/lib64/libnss_files-2.17.so
 MAP: 7f89385fa000-7f8938600000 rw-p 00000000 00:00 0 
 MAP: 7f8938600000-7f893d400000 rw-p 00000000 00:00 0 
 MAP: 7f893d600000-7f893e400000 rw-p 00000000 00:00 0 
 MAP: 7f893e600000-7f893fe00000 rw-p 00000000 00:00 0 
 MAP: 7f8940000000-7f8940800000 rw-p 00000000 00:00 0 
 MAP: 7f89409ff000-7f8940a00000 ---p 00000000 00:00 0 
 MAP: 7f8940a00000-7f8940c00000 rw-p 00000000 00:00 0 
 MAP: 7f8940c00000-7f8943800000 rw-p 00000000 00:00 0 
 MAP: 7f89439ff000-7f8943a00000 ---p 00000000 00:00 0 
 MAP: 7f8943a00000-7f8943c00000 rw-p 00000000 00:00 0 
 MAP: 7f8943c00000-7f8944e00000 rw-p 00000000 00:00 0 
 MAP: 7f8944ffd000-7f8944ffe000 ---p 00000000 00:00 0 
 MAP: 7f8944ffe000-7f89451fe000 rw-p 00000000 00:00 0 
 MAP: 7f89451fe000-7f89451ff000 ---p 00000000 00:00 0 
 MAP: 7f89451ff000-7f89453ff000 rw-p 00000000 00:00 0 
 MAP: 7f89453ff000-7f8945400000 ---p 00000000 00:00 0 
 MAP: 7f8945400000-7f8945600000 rw-p 00000000 00:00 0 
 MAP: 7f8945600000-7f8946800000 rw-p 00000000 00:00 0 
 MAP: 7f89469ff000-7f8946a00000 ---p 00000000 00:00 0 
 MAP: 7f8946a00000-7f8946c00000 rw-p 00000000 00:00 0 
 MAP: 7f8946c00000-7f8947200000 rw-p 00000000 00:00 0 
 MAP: 7f8947400000-7f8949a00000 rw-p 00000000 00:00 0 
 MAP: 7f8949bff000-7f8949c00000 ---p 00000000 00:00 0 
 MAP: 7f8949c00000-7f8949e00000 rw-p 00000000 00:00 0 
 MAP: 7f8949e00000-7f894a000000 rw-p 00000000 00:00 0 
 MAP: 7f894a1ff000-7f894a200000 ---p 00000000 00:00 0 
 MAP: 7f894a200000-7f894a400000 rw-p 00000000 00:00 0 
 MAP: 7f894a400000-7f896c000000 rw-p 00000000 00:00 0 
 MAP: 7f896c1ff000-7f896c200000 ---p 00000000 00:00 0 
 MAP: 7f896c200000-7f896c400000 rw-p 00000000 00:00 0 
 MAP: 7f896c400000-7f896c800000 rw-p 00000000 00:00 0 
 MAP: 7f896ca00000-7f896f200000 rw-p 00000000 00:00 0 
 MAP: 7f896f3ff000-7f896f400000 ---p 00000000 00:00 0 
 MAP: 7f896f400000-7f896fc00000 rw-p 00000000 00:00 0 
 MAP: 7f896fc00000-7f8970000000 rw-p 00000000 00:00 0 
 MAP: 7f89701ff000-7f8970200000 ---p 00000000 00:00 0 
 MAP: 7f8970200000-7f8970400000 rw-p 00000000 00:00 0 
 MAP: 7f8970400000-7f8970600000 rw-p 00000000 00:00 0 
 MAP: 7f89707ff000-7f8970800000 ---p 00000000 00:00 0 
 MAP: 7f8970800000-7f8970a00000 rw-p 00000000 00:00 0 
 MAP: 7f8970a00000-7f8970c00000 rw-p 00000000 00:00 0 
 MAP: 7f8970e00000-7f8971600000 rw-p 00000000 00:00 0 
 MAP: 7f89717fe000-7f89717ff000 ---p 00000000 00:00 0 
 MAP: 7f89717ff000-7f89719ff000 rw-p 00000000 00:00 0 
 MAP: 7f89719ff000-7f8971a00000 ---p 00000000 00:00 0 
 MAP: 7f8971a00000-7f8971c00000 rw-p 00000000 00:00 0 
 MAP: 7f8971c00000-7f8972e00000 rw-p 00000000 00:00 0 
 MAP: 7f8972fff000-7f8973000000 ---p 00000000 00:00 0 
 MAP: 7f8973000000-7f8973200000 rw-p 00000000 00:00 0 
 MAP: 7f8973200000-7f8973400000 rw-p 00000000 00:00 0 
 MAP: 7f89735ff000-7f8973600000 ---p 00000000 00:00 0 
 MAP: 7f8973600000-7f8973800000 rw-p 00000000 00:00 0 
 MAP: 7f8973800000-7f8973a00000 rw-p 00000000 00:00 0 
 MAP: 7f8973bff000-7f8973c00000 ---p 00000000 00:00 0 
 MAP: 7f8973c00000-7f8973e00000 rw-p 00000000 00:00 0 
 MAP: 7f8973e00000-7f8974000000 rw-p 00000000 00:00 0 
 MAP: 7f89741ff000-7f8974200000 ---p 00000000 00:00 0 
 MAP: 7f8974200000-7f8974400000 rw-p 00000000 00:00 0 
 MAP: 7f8974400000-7f8974600000 rw-p 00000000 00:00 0 
 MAP: 7f89747ff000-7f8974800000 ---p 00000000 00:00 0 
  MAP: 7f8974800000-7f8974a00000 rw-p 00000000 00:00 0 
 MAP: 7f8974a00000-7f8974c00000 rw-p 00000000 00:00 0 
 MAP: 7f8974dff000-7f8974e00000 ---p 00000000 00:00 0 
 MAP: 7f8974e00000-7f8975000000 rw-p 00000000 00:00 0 
 MAP: 7f8975000000-7f8975200000 rw-p 00000000 00:00 0 
 MAP: 7f89753fe000-7f89753ff000 ---p 00000000 00:00 0 
 MAP: 7f89753ff000-7f89755ff000 rw-p 00000000 00:00 0 
 MAP: 7f89755ff000-7f8975600000 ---p 00000000 00:00 0 
 MAP: 7f8975600000-7f8975800000 rw-p 00000000 00:00 0 
 MAP: 7f8975800000-7f8976000000 rw-p 00000000 00:00 0 
 MAP: 7f89761fe000-7f89761ff000 ---p 00000000 00:00 0 
 MAP: 7f89761ff000-7f89763ff000 rw-p 00000000 00:00 0 
 MAP: 7f89763ff000-7f8976400000 ---p 00000000 00:00 0 
 MAP: 7f8976400000-7f8976600000 rw-p 00000000 00:00 0 
 MAP: 7f8976600000-7f8976a00000 rw-p 00000000 00:00 0 
 MAP: 7f8976bff000-7f8976c00000 ---p 00000000 00:00 0 
 MAP: 7f8976c00000-7f8976e00000 rw-p 00000000 00:00 0 
 MAP: 7f8976e00000-7f8977000000 rw-p 00000000 00:00 0 
 MAP: 7f89771ff000-7f8977200000 ---p 00000000 00:00 0 
 MAP: 7f8977200000-7f8977400000 rw-p 00000000 00:00 0 
 MAP: 7f8977400000-7f8977a00000 rw-p 00000000 00:00 0 
 MAP: 7f8977bff000-7f8977c00000 ---p 00000000 00:00 0 
 MAP: 7f8977c00000-7f8977e00000 rw-p 00000000 00:00 0 
 MAP: 7f8977e00000-7f8978200000 rw-p 00000000 00:00 0 
 MAP: 7f89783fe000-7f89783ff000 ---p 00000000 00:00 0 
 MAP: 7f89783ff000-7f89785ff000 rw-p 00000000 00:00 0 
 MAP: 7f89785ff000-7f8978600000 ---p 00000000 00:00 0 
 MAP: 7f8978600000-7f8978800000 rw-p 00000000 00:00 0 
 MAP: 7f8978800000-7f8978e00000 rw-p 00000000 00:00 0 
 MAP: 7f8978ffe000-7f8978fff000 ---p 00000000 00:00 0 
 MAP: 7f8978fff000-7f89791ff000 rw-p 00000000 00:00 0 
 MAP: 7f89791ff000-7f8979200000 ---p 00000000 00:00 0 
 MAP: 7f8979200000-7f8979400000 rw-p 00000000 00:00 0 
 MAP: 7f8979400000-7f8979600000 rw-p 00000000 00:00 0 
 MAP: 7f89797ff000-7f8979800000 ---p 00000000 00:00 0 
 MAP: 7f8979800000-7f8979a00000 rw-p 00000000 00:00 0 
 MAP: 7f8979a00000-7f897a000000 rw-p 00000000 00:00 0 
 MAP: 7f897a1fd000-7f897a1fe000 ---p 00000000 00:00 0 
 MAP: 7f897a1fe000-7f897a3fe000 rw-p 00000000 00:00 0 
 MAP: 7f897a3fe000-7f897a3ff000 ---p 00000000 00:00 0 
 MAP: 7f897a3ff000-7f897a5ff000 rw-p 00000000 00:00 0 
 MAP: 7f897a5ff000-7f897a600000 ---p 00000000 00:00 0 
 MAP: 7f897a600000-7f897a800000 rw-p 00000000 00:00 0 
 MAP: 7f897a800000-7f897ac00000 rw-p 00000000 00:00 0 
 MAP: 7f897adfe000-7f897adff000 ---p 00000000 00:00 0 
 MAP: 7f897adff000-7f897afff000 rw-p 00000000 00:00 0 
 MAP: 7f897afff000-7f897b000000 ---p 00000000 00:00 0 
 MAP: 7f897b000000-7f897b200000 rw-p 00000000 00:00 0 
 MAP: 7f897b200000-7f897b400000 rw-p 00000000 00:00 0 
 MAP: 7f897b5ff000-7f897b600000 ---p 00000000 00:00 0 
 MAP: 7f897b600000-7f897b800000 rw-p 00000000 00:00 0 
 MAP: 7f897b800000-7f897ba00000 rw-p 00000000 00:00 0 
 MAP: 7f897bbff000-7f897bc00000 ---p 00000000 00:00 0 
 MAP: 7f897bc00000-7f897be00000 rw-p 00000000 00:00 0 
 MAP: 7f897be00000-7f897c000000 rw-p 00000000 00:00 0 
 MAP: 7f897c1fe000-7f897c1ff000 ---p 00000000 00:00 0 
  MAP: 7f897c1ff000-7f897c3ff000 rw-p 00000000 00:00 0 
 MAP: 7f897c3ff000-7f897c400000 ---p 00000000 00:00 0 
 MAP: 7f897c400000-7f897c600000 rw-p 00000000 00:00 0 
 MAP: 7f897c600000-7f897c800000 rw-p 00000000 00:00 0 
 MAP: 7f897c9ff000-7f897ca00000 ---p 00000000 00:00 0 
 MAP: 7f897ca00000-7f897cc00000 rw-p 00000000 00:00 0 
 MAP: 7f897cc00000-7f897d000000 rw-p 00000000 00:00 0 
 MAP: 7f897d1ff000-7f897d200000 ---p 00000000 00:00 0 
 MAP: 7f897d200000-7f897da00000 rw-p 00000000 00:00 0 
 MAP: 7f897da00000-7f897e200000 rw-p 00000000 00:00 0 
 MAP: 7f897e3fe000-7f897e3ff000 ---p 00000000 00:00 0 
 MAP: 7f897e3ff000-7f897e5ff000 rw-p 00000000 00:00 0 
 MAP: 7f897e7ff000-7f897e800000 ---p 00000000 00:00 0 
 MAP: 7f897e800000-7f897ea00000 rw-p 00000000 00:00 0 
 MAP: 7f897ea00000-7f897ee00000 rw-p 00000000 00:00 0 
 MAP: 7f897effe000-7f897efff000 ---p 00000000 00:00 0 
 MAP: 7f897efff000-7f897f1ff000 rw-p 00000000 00:00 0 
 MAP: 7f897f1ff000-7f897f200000 ---p 00000000 00:00 0 
 MAP: 7f897f200000-7f897f400000 rw-p 00000000 00:00 0 
 MAP: 7f897f400000-7f897f800000 rw-p 00000000 00:00 0 
 MAP: 7f897f9ff000-7f897fa00000 ---p 00000000 00:00 0 
 MAP: 7f897fa00000-7f897fc00000 rw-p 00000000 00:00 0 
 MAP: 7f897fc00000-7f897fe00000 rw-p 00000000 00:00 0 
 MAP: 7f897fffd000-7f897fffe000 ---p 00000000 00:00 0 
 MAP: 7f897fffe000-7f89801fe000 rw-p 00000000 00:00 0 
 MAP: 7f89801fe000-7f89801ff000 ---p 00000000 00:00 0 
 MAP: 7f89801ff000-7f89803ff000 rw-p 00000000 00:00 0 
 MAP: 7f89803ff000-7f8980400000 ---p 00000000 00:00 0 
 MAP: 7f8980400000-7f8980600000 rw-p 00000000 00:00 0 
 MAP: 7f8980600000-7f8980800000 rw-p 00000000 00:00 0 
 MAP: 7f89809ff000-7f8980a00000 ---p 00000000 00:00 0 
 MAP: 7f8980a00000-7f8980c00000 rw-p 00000000 00:00 0 
 MAP: 7f8980c00000-7f8980e00000 rw-p 00000000 00:00 0 
 MAP: 7f8980ffe000-7f8980fff000 ---p 00000000 00:00 0 
 MAP: 7f8980fff000-7f89811ff000 rw-p 00000000 00:00 0 
 MAP: 7f8981200000-7f8981800000 rw-p 00000000 00:00 0 
 MAP: 7f89819fe000-7f89819ff000 ---p 00000000 00:00 0 
 MAP: 7f89819ff000-7f8981bff000 rw-p 00000000 00:00 0 
 MAP: 7f8981bff000-7f8981c00000 ---p 00000000 00:00 0 
 MAP: 7f8981c00000-7f8981e00000 rw-p 00000000 00:00 0 
 MAP: 7f8981e00000-7f8982000000 rw-p 00000000 00:00 0 
 MAP: 7f89821ff000-7f8982200000 ---p 00000000 00:00 0 
 MAP: 7f8982200000-7f8982400000 rw-p 00000000 00:00 0 
 MAP: 7f8982400000-7f8982600000 rw-p 00000000 00:00 0 
 MAP: 7f89827ff000-7f8982800000 ---p 00000000 00:00 0 
 MAP: 7f8982800000-7f8982a00000 rw-p 00000000 00:00 0 
 MAP: 7f8982a00000-7f8983000000 rw-p 00000000 00:00 0 
 MAP: 7f89831ff000-7f8983200000 ---p 00000000 00:00 0 
 MAP: 7f8983200000-7f8983400000 rw-p 00000000 00:00 0 
 MAP: 7f8983400000-7f8983600000 rw-p 00000000 00:00 0 
 MAP: 7f89837ff000-7f8983800000 ---p 00000000 00:00 0 
 MAP: 7f8983800000-7f8983a00000 rw-p 00000000 00:00 0 
 MAP: 7f8983a00000-7f8983e00000 rw-p 00000000 00:00 0 
 MAP: 7f8983ffe000-7f8983fff000 ---p 00000000 00:00 0 
 MAP: 7f8983fff000-7f89841ff000 rw-p 00000000 00:00 0 
 MAP: 7f89841ff000-7f8984200000 ---p 00000000 00:00 0 
  MAP: 7f8984200000-7f8984400000 rw-p 00000000 00:00 0 
 MAP: 7f8984400000-7f8984800000 rw-p 00000000 00:00 0 
 MAP: 7f89849ff000-7f8984a00000 ---p 00000000 00:00 0 
 MAP: 7f8984a00000-7f8984c00000 rw-p 00000000 00:00 0 
 MAP: 7f8984c00000-7f8987400000 rw-p 00000000 00:00 0 
 MAP: 7f898755f000-7f89875ff000 rwxp 00000000 00:00 0 
 MAP: 7f89875ff000-7f8987600000 ---p 00000000 00:00 0 
 MAP: 7f8987600000-7f8987800000 rw-p 00000000 00:00 0 
 MAP: 7f8987800000-7f8987a00000 rw-p 00000000 00:00 0 
 MAP: 7f8987a0f000-7f8987bff000 rwxp 00000000 00:00 0 
 MAP: 7f8987bff000-7f8987c00000 ---p 00000000 00:00 0 
 MAP: 7f8987c00000-7f8987e00000 rw-p 00000000 00:00 0 
 MAP: 7f8987e00000-7f8988000000 rw-p 00000000 00:00 0 
 MAP: 7f898800f000-7f89881ff000 rwxp 00000000 00:00 0 
 MAP: 7f89881ff000-7f8988200000 ---p 00000000 00:00 0 
 MAP: 7f8988200000-7f8988400000 rw-p 00000000 00:00 0 
 MAP: 7f8988400000-7f8988800000 rw-p 00000000 00:00 0 
 MAP: 7f898880e000-7f89889fe000 rwxp 00000000 00:00 0 
 MAP: 7f89889fe000-7f89889ff000 ---p 00000000 00:00 0 
 MAP: 7f89889ff000-7f8988bff000 rw-p 00000000 00:00 0 
 MAP: 7f8988bff000-7f8988c00000 ---p 00000000 00:00 0 
 MAP: 7f8988c00000-7f8988e00000 rw-p 00000000 00:00 0 
 MAP: 7f8988e00000-7f898a000000 rw-p 00000000 00:00 0 
 MAP: 7f898a00f000-7f898a1ff000 rwxp 00000000 00:00 0 
 MAP: 7f898a1ff000-7f898a200000 ---p 00000000 00:00 0 
 MAP: 7f898a200000-7f898a400000 rw-p 00000000 00:00 0 
 MAP: 7f898a400000-7f898a800000 rw-p 00000000 00:00 0 
 MAP: 7f898a80f000-7f898a9ff000 rwxp 00000000 00:00 0 
 MAP: 7f898a9ff000-7f898aa00000 ---p 00000000 00:00 0 
 MAP: 7f898aa00000-7f898ac00000 rw-p 00000000 00:00 0 
 MAP: 7f898ac00000-7f898ae00000 rw-p 00000000 00:00 0 
 MAP: 7f898ae05000-7f898aeb5000 rwxp 00000000 00:00 0 
 MAP: 7f898aeb5000-7f898aeb6000 ---p 00000000 00:00 0 
 MAP: 7f898aeb6000-7f898aebf000 rw-p 00000000 00:00 0 
 MAP: 7f898aebf000-7f898aec0000 ---p 00000000 00:00 0 
 MAP: 7f898aec0000-7f898b000000 rwxp 00000000 00:00 0 
 MAP: 7f898b000000-7f898b400000 rw-p 00000000 00:00 0 
 MAP: 7f898b403000-7f898b583000 rwxp 00000000 00:00 0 
 MAP: 7f898b583000-7f898b587000 rw-p 00000000 00:00 0 
 MAP: 7f898b587000-7f898b749000 r-xp 00000000 fd:00 33561437                   /usr/lib64/libc-2.17.so
 MAP: 7f898b749000-7f898b949000 ---p 001c2000 fd:00 33561437                   /usr/lib64/libc-2.17.so
 MAP: 7f898b949000-7f898b94d000 r--p 001c2000 fd:00 33561437                   /usr/lib64/libc-2.17.so
 MAP: 7f898b94d000-7f898b94f000 rw-p 001c6000 fd:00 33561437                   /usr/lib64/libc-2.17.so
 MAP: 7f898b94f000-7f898b954000 rw-p 00000000 00:00 0 
 MAP: 7f898b954000-7f898b96b000 r-xp 00000000 fd:00 33561476                   /usr/lib64/libpthread-2.17.so
 MAP: 7f898b96b000-7f898bb6a000 ---p 00017000 fd:00 33561476                   /usr/lib64/libpthread-2.17.so
 MAP: 7f898bb6a000-7f898bb6b000 r--p 00016000 fd:00 33561476                   /usr/lib64/libpthread-2.17.so
 MAP: 7f898bb6b000-7f898bb6c000 rw-p 00017000 fd:00 33561476                   /usr/lib64/libpthread-2.17.so
 MAP: 7f898bb6c000-7f898bb70000 rw-p 00000000 00:00 0 
 MAP: 7f898bb70000-7f898bc71000 r-xp 00000000 fd:00 33561450                   /usr/lib64/libm-2.17.so
 MAP: 7f898bc71000-7f898be70000 ---p 00101000 fd:00 33561450                   /usr/lib64/libm-2.17.so
 MAP: 7f898be70000-7f898be71000 r--p 00100000 fd:00 33561450                   /usr/lib64/libm-2.17.so
 MAP: 7f898be71000-7f898be72000 rw-p 00101000 fd:00 33561450                   /usr/lib64/libm-2.17.so
 MAP: 7f898be72000-7f898bea8000 r-xp 00000000 fd:02 668053                     /home/splunk/lib/libbson-1.0.so.0.0.0
 MAP: 7f898bea8000-7f898beac000 rw-p 00035000 fd:02 668053                     /home/splunk/lib/libbson-1.0.so.0.0.0
 MAP: 7f898beac000-7f898beb2000 rw-p 00000000 00:00 0 
 MAP: 7f898beb2000-7f898bf2e000 r-xp 00000000 fd:02 669570                     /home/splunk/lib/libmongoc-1.0.so.0.0.0
 MAP: 7f898bf2e000-7f898bf31000 rw-p 0007c000 fd:02 669570                     /home/splunk/lib/libmongoc-1.0.so.0.0.0
 MAP: 7f898bf31000-7f898c028000 r-xp 00000000 fd:02 669574                     /home/splunk/lib/libsqlite3.so.0.8.6
 MAP: 7f898c028000-7f898c02c000 rw-p 000f6000 fd:02 669574                     /home/splunk/lib/libsqlite3.so.0.8.6
 MAP: 7f898c02c000-7f898c02d000 rw-p 00000000 00:00 0 
 MAP: 7f898c02d000-7f898c0ce000 r-xp 00000000 fd:02 669588                     /home/splunk/lib/libarchive.so.13.3.1
 MAP: 7f898c0ce000-7f898c0d2000 rw-p 000a1000 fd:02 669588                     /home/splunk/lib/libarchive.so.13.3.1
 MAP: 7f898c0d2000-7f898c0d4000 r-xp 00000000 fd:00 33561447                   /usr/lib64/libdl-2.17.so
 MAP: 7f898c0d4000-7f898c2d4000 ---p 00002000 fd:00 33561447                   /usr/lib64/libdl-2.17.so
 MAP: 7f898c2d4000-7f898c2d5000 r--p 00002000 fd:00 33561447                   /usr/lib64/libdl-2.17.so
 MAP: 7f898c2d5000-7f898c2d6000 rw-p 00003000 fd:00 33561447                   /usr/lib64/libdl-2.17.so
 MAP: 7f898c2d6000-7f898c583000 r-xp 00000000 fd:02 669569                     /home/splunk/lib/libcrypto.so.1.0.0
 MAP: 7f898c583000-7f898c5af000 rw-p 002ad000 fd:02 669569                     /home/splunk/lib/libcrypto.so.1.0.0
 MAP: 7f898c5af000-7f898c5b3000 rw-p 00000000 00:00 0 
 MAP: 7f898c5b3000-7f898c609000 r-xp 00000000 fd:02 669573                     /home/splunk/lib/libxmlsec1-openssl.so.1.2.24
 MAP: 7f898c609000-7f898c60d000 rw-p 00055000 fd:02 669573                     /home/splunk/lib/libxmlsec1-openssl.so.1.2.24
 MAP: 7f898c60d000-7f898c60e000 rw-p 00000000 00:00 0 
 MAP: 7f898c60e000-7f898c689000 r-xp 00000000 fd:02 669580                     /home/splunk/lib/libxmlsec1.so.1.2.24
 MAP: 7f898c689000-7f898c68d000 rw-p 0007a000 fd:02 669580                     /home/splunk/lib/libxmlsec1.so.1.2.24
 MAP: 7f898c68d000-7f898c846000 r-xp 00000000 fd:02 668050                     /home/splunk/lib/libxml2.so.2.9.8
 MAP: 7f898c846000-7f898c850000 rw-p 001b8000 fd:02 668050                     /home/splunk/lib/libxml2.so.2.9.8
 MAP: 7f898c850000-7f898c851000 rw-p 00000000 00:00 0 
 MAP: 7f898c851000-7f898c858000 r-xp 00000000 fd:00 33561481                   /usr/lib64/librt-2.17.so
 MAP: 7f898c858000-7f898ca57000 ---p 00007000 fd:00 33561481                   /usr/lib64/librt-2.17.so
 MAP: 7f898ca57000-7f898ca58000 r--p 00006000 fd:00 33561481                   /usr/lib64/librt-2.17.so
 MAP: 7f898ca58000-7f898ca59000 rw-p 00007000 fd:00 33561481                   /usr/lib64/librt-2.17.so
 MAP: 7f898ca59000-7f898ca7b000 r-xp 00000000 fd:00 33561430                   /usr/lib64/ld-2.17.so
 MAP: 7f898ca7b000-7f898ca7d000 rw-p 00000000 00:00 0 
 MAP: 7f898ca7d000-7f898ca98000 r-xp 00000000 fd:02 668051                     /home/splunk/lib/libz.so.1.2.11
 MAP: 7f898ca98000-7f898ca99000 rw-p 0001b000 fd:02 668051                     /home/splunk/lib/libz.so.1.2.11
 MAP: 7f898ca99000-7f898ca9a000 rw-p 00000000 00:00 0 
 MAP: 7f898ca9a000-7f898caab000 r-xp 00000000 fd:02 669590                     /home/splunk/lib/libbz2.so.1.0.3
 MAP: 7f898caab000-7f898caad000 rw-p 00010000 fd:02 669590                     /home/splunk/lib/libbz2.so.1.0.3
 MAP: 7f898caad000-7f898caaf000 rw-p 00000000 00:00 0 
 MAP: 7f898caaf000-7f898cb1b000 r-xp 00000000 fd:02 669581                     /home/splunk/lib/libssl.so.1.0.0
 MAP: 7f898cb1b000-7f898cb25000 rw-p 0006b000 fd:02 669581                     /home/splunk/lib/libssl.so.1.0.0
 MAP: 7f898cb25000-7f898cb77000 r-xp 00000000 fd:02 669481                     /home/splunk/lib/libxslt.so.1.1.30
 MAP: 7f898cb77000-7f898cb79000 rw-p 00052000 fd:02 669481                     /home/splunk/lib/libxslt.so.1.1.30
 MAP: 7f898cb79000-7f898cb7a000 rw-p 00000000 00:00 0 
 MAP: 7f898cb7a000-7f898cc1f000 r-xp 00000000 fd:02 669575                     /home/splunk/lib/libpcre2-8.so
 MAP: 7f898cc1f000-7f898cc20000 rw-p 000a5000 fd:02 669575                     /home/splunk/lib/libpcre2-8.so
 MAP: 7f898cc20000-7f898cc21000 rw-p 00000000 00:00 0 
 MAP: 7f898cc26000-7f898cc27000 rw-p 00000000 00:00 0 
 MAP: 7f898cc27000-7f898cc75000 r-xp 00000000 fd:02 669479                     /home/splunk/lib/libjemalloc.so.2
 MAP: 7f898cc75000-7f898cc78000 rw-p 0004d000 fd:02 669479                     /home/splunk/lib/libjemalloc.so.2
 MAP: 7f898cc78000-7f898cc7a000 rw-p 00000000 00:00 0 
 MAP: 7f898cc7a000-7f898cc7b000 r--p 00021000 fd:00 33561430                   /usr/lib64/ld-2.17.so
 MAP: 7f898cc7b000-7f898cc7c000 rw-p 00022000 fd:00 33561430                   /usr/lib64/ld-2.17.so
 MAP: 7f898cc7c000-7f898cc7d000 rw-p 00000000 00:00 0 
 MAP: 7ffd4cf03000-7ffd4cf24000 rw-p 00000000 00:00 0                          [stack]
 MAP: 7ffd4cfb3000-7ffd4cfb5000 r-xp 00000000 00:00 0                          [vdso]
 MAP: ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Last errno: 0
Threads running: 78
Runtime: 427255.981480s
argv: [splunkd -p 8089 start]
Regex JIT enabled

using CLOCK_MONOTONIC
Thread: "SchedulerThread", did_join=0, ready_to_run=Y, main_thread=N
First 8 bytes of Thread token @0x7f8975813290:
00000000  00 e7 5f 75 89 7f 00 00                           |.._u....|
00000008

First 512 bytes of Timeout object @0x7f8975836000:
00000000  c8 fd c6 4f b5 55 00 00  00 00 00 00 00 00 00 00  |...O.U..........|
00000010  d0 d3 5f 75 89 7f 00 00  00 00 00 00 00 00 00 00  |.._u............|
00000020  00 00 00 00 00 00 00 00  d2 87 06 00 00 00 00 00  |................|
00000030  88 24 1d 22 00 00 00 00  78 e6 00 00 00 00 00 00  |.$."....x.......|
00000040  c0 05 d8 4f b5 55 00 00  60 8b d1 4f b5 55 00 00  |...O.U..`..O.U..|
00000050  c0 10 82 75 89 7f 00 00  10 00 00 00 00 00 00 00  |...u............|
00000060  10 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000a0  00 00 00 00 00 00 00 00  08 fe c6 4f b5 55 00 00  |...........O.U..|
000000b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000c0  00 00 00 00 00 00 00 00  b8 60 83 75 89 7f 00 00  |.........`.u....|
000000d0  b8 60 83 75 89 7f 00 00  00 00 00 00 00 00 00 00  |.`.u............|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000f0  03 00 00 00 00 00 00 00  00 70 82 75 89 7f 00 00  |.........p.u....|
00000100  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000110  00 30 9c 75 89 7f 00 00  00 d8 80 75 89 7f 00 00  |.0.u.......u....|
00000120  00 f8 84 75 89 7f 00 00  29 00 00 00 00 00 00 00  |...u....).......|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000170  60 61 83 75 89 7f 00 00  60 61 83 75 89 7f 00 00  |`a.u....`a.u....|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001a0  90 61 83 75 89 7f 00 00  90 61 83 75 89 7f 00 00  |.a.u.....a.u....|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000200
_when = 427986.572335240, _initialInterval = 59.000

x86 CPUID registers:
         0: 0000000B 756E6547 6C65746E 49656E69
         1: 000206F2 02080800 82BA2203 17CBFBFF
         2: 55035A01 00F0B2FF 00000000 00CA0000
         3: 00000000 00000000 00000000 00000000
         4: 1C000000 00000000 00000000 00000000
         5: 00000000 00000000 00000000 00000000
         6: 00000000 00000000 00000000 00000000
         7: 00000000 00000000 00000000 00000000
         8: 00000000 00000000 00000000 00000000
         9: 00000000 00000000 00000000 00000000
         A: 07000000 00000004 00000000 00000000
                  B: 00000000 00000000 00000000 00000002
  80000000: 80000008 00000000 00000000 00000000
  80000001: 00000000 00000000 00000001 2C100800
  80000002: 20202020 49202020 6C65746E 20295228
  80000003: 6E6F6558 20295228 20555043 202D3745
  80000004: 30373834 20402020 30342E32 007A4847
  80000005: 00000000 00000000 00000000 00000000
  80000006: 00000000 00000000 01006040 00000000
  80000007: 00000000 00000000 00000000 00000000
  80000008: 0000302C 00001000 00000000 00000000
terminating...




00000020  00 00 00 00 00 00 00 00  d2 87 06 00 00 00 00 00  |................|
00000030  88 24 1d 22 00 00 00 00  78 e6 00 00 00 00 00 00  |.$."....x.......|
00000040  c0 05 d8 4f b5 55 00 00  60 8b d1 4f b5 55 00 00  |...O.U..`..O.U..|
00000050  c0 10 82 75 89 7f 00 00  10 00 00 00 00 00 00 00  |...u............|
00000060  10 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000a0  00 00 00 00 00 00 00 00  08 fe c6 4f b5 55 00 00  |...........O.U..|
000000b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000c0  00 00 00 00 00 00 00 00  b8 60 83 75 89 7f 00 00  |.........`.u....|
000000d0  b8 60 83 75 89 7f 00 00  00 00 00 00 00 00 00 00  |.`.u............|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000f0  03 00 00 00 00 00 00 00  00 70 82 75 89 7f 00 00  |.........p.u....|
00000100  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000110  00 30 9c 75 89 7f 00 00  00 d8 80 75 89 7f 00 00  |.0.u.......u....|
00000120  00 f8 84 75 89 7f 00 00  29 00 00 00 00 00 00 00  |...u....).......|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000170  60 61 83 75 89 7f 00 00  60 61 83 75 89 7f 00 00  |`a.u....`a.u....|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001a0  90 61 83 75 89 7f 00 00  90 61 83 75 89 7f 00 00  |.a.u.....a.u....|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000200
_when = 427986.572335240, _initialInterval = 59.000

x86 CPUID registers:
         0: 0000000B 756E6547 6C65746E 49656E69
         1: 000206F2 02080800 82BA2203 17CBFBFF
         2: 55035A01 00F0B2FF 00000000 00CA0000
         3: 00000000 00000000 00000000 00000000
         4: 1C000000 00000000 00000000 00000000
         5: 00000000 00000000 00000000 00000000
         6: 00000000 00000000 00000000 00000000
         7: 00000000 00000000 00000000 00000000
         8: 00000000 00000000 00000000 00000000
         9: 00000000 00000000 00000000 00000000
         A: 07000000 00000004 00000000 00000000
         B: 00000000 00000000 00000000 00000002
  80000000: 80000008 00000000 00000000 00000000
  80000001: 00000000 00000000 00000001 2C100800
  80000002: 20202020 49202020 6C65746E 20295228
  80000003: 6E6F6558 20295228 20555043 202D3745
  80000004: 30373834 20402020 30342E32 007A4847
  80000005: 00000000 00000000 00000000 00000000
  80000006: 00000000 00000000 01006040 00000000
  80000007: 00000000 00000000 00000000 00000000
  80000008: 0000302C 00001000 00000000 00000000
terminating...

richgalloway · ‎09-12-2019

What is at the end of splunkd.log?
Consider opening a Support request.

---
If this reply helps you, Karma would be appreciated.

MHS · ‎09-12-2019

I guess the question is this. Can I pause or disable scheduled queries to see if we can bring this system back up and troubleshoot what is going on it it.

Sorry for the original post coming across a bit pissy. Just frustrated.

jacobpevans · ‎09-12-2019

You can disable all scheduled searches by removing all instances of savedsearches.conf. You could also rename them to anything else (instead of deleting) while you investigate.

Cheers,
Jacob

If you feel this response answered your question, please do not forget to mark it as such. If it did not, but you do have the answer, feel free to answer your own post and accept that as the answer.

Splunk Crashes on Startup

scheduler activity

Get Inspired! We’ve Got Validation that Your Hard Work is Paying Off

What's New in Splunk Enterprise 9.4: Features to Power Your Digital Resilience

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)