Re: Splunk 8.0.2 and proxy conf

lies2yk · ‎07-08-2020

Hello

I'm having a strange problem on splunk 8 (clean install) and using the "proxyConfig" directive.

Inside the server.conf file I configured the following:
[ProxyConfig]
http_proxy = http://127.0.0.1:3128
https_proxy = https://127.0.0.1:3128
proxy_rules = *
no_proxy = localhost, 127.0.0.1, :: 1

Unfortunately, when I try to install an app via the web interface, the interface crashes immediately after I enter my credentials for downloading the app.
After a few seconds he throws me out and the login screen returns.

Please help me...

Thanks

richgalloway · ‎07-08-2020

Have you checked splunkd.log?

---
If this reply helps you, Karma would be appreciated.

lies2yk · ‎07-08-2020

Here the log.
in this extraction you see Splunk 7.6 and proxy listen in port 3129 because i'm trying various release, but the same issue i've in 8.0.x and proxy configured on port 3128.

_ZN9EventLoop3runEv + 667 (splunkd + 0x181688B)
  [0x000055DE4DD40400] _ZN15TcpOutboundLoop3runEv + 16 (splunkd + 0x18DF400)
  [0x000055DE4DC8E8F7] _ZN21HttpClientTransaction22runSyncAndShutdownLoopEP15TcpOutboundLoop + 103 (splunkd + 0x182D8F7)
  [0x000055DE4DC8EA75] _ZN21HttpClientTransaction7runSyncEv + 69 (splunkd + 0x182DA75)
  [0x000055DE4DB855B1] _ZN18ApplicationUpdater20fetchUpdateFileByUriERK7FullUriRK3StrR28ApplicationUpdateTransaction + 145 (splunkd + 0x17245B1)
  [0x000055DE4DB85CF1] _ZN18ApplicationUpdater20fetchUpdateFileByUriERK7FullUriRK3StrR8Pathnameb + 385 (splunkd + 0x1724CF1)
  [0x000055DE4D9CD20E] _ZN21LocalAppsAdminHandler13handleInstallER10ConfigInfoPK3StrS4_bbb + 2174 (splunkd + 0x156C20E)
  [0x000055DE4D9CDD6A] _ZN21LocalAppsAdminHandler12handleCreateER10ConfigInfo + 426 (splunkd + 0x156CD6A)
  [0x000055DE4D64A52D] _ZN14MConfigHandler2goER10ConfigInfo + 189 (splunkd + 0x11E952D)
  [0x000055DE4D64B07F] _ZN29AdminManagerReplyDataProvider2goEv + 783 (splunkd + 0x11EA07F)
  [0x000055DE4D7273B8] _ZN33ServicesEndpointReplyDataProvider9rawHandleEv + 88 (splunkd + 0x12C63B8)
  [0x000055DE4D71CEBF] _ZN18RawRestHttpHandler10getPreBodyEP21HttpServerTransaction + 31 (splunkd + 0x12BBEBF)
  [0x000055DE4DCA8390] _ZN32HttpThreadedCommunicationHandler11communicateER17TcpSyncDataBuffer + 272 (splunkd + 0x1847390)
  [0x000055DE4D057432] _ZN16TcpChannelThread4mainEv + 242 (splunkd + 0xBF6432)
  [0x000055DE4DD41638] _ZN6Thread8callMainEPv + 120 (splunkd + 0x18E0638)
  [0x00007FD2A54DAEA5] ? (libpthread.so.0 + 0x7EA5)
  [0x00007FD2A52038CD] clone + 109 (libc.so.6 + 0xFE8CD)
 Linux / ssplkshwsl01v / 3.10.0-1062.7.1.el7.x86_64 / #1 SMP Wed Nov 13 08:44:42 EST 2019 / x86_64
 Last few lines of stderr (may contain info on assertion failure, but also could be old):
    2020-07-08 13:39:16.950 +0200 splunkd started (build 47d8552a4d84) pid=47562
    splunkd: /opt/splunk/src/util/HttpClientRequest.cpp:1895: void HttpClientTransaction::_handleRedirect(): Assertion `_redirectReply == REPLY_EATING_NORMAL' failed.
    2020-07-08 13:43:07.379 +0200 splunkd started (build 47d8552a4d84) pid=48059
    splunkd: /opt/splunk/src/util/HttpClientRequest.cpp:1895: void HttpClientTransaction::_handleRedirect(): Assertion `_redirectReply == REPLY_EATING_NORMAL' failed.
    2020-07-08 13:46:35.317 +0200 splunkd started (build 47d8552a4d84) pid=48582
    splunkd: /opt/splunk/src/util/HttpClientRequest.cpp:1895: void HttpClientTransaction::_handleRedirect(): Assertion `_redirectReply == REPLY_EATING_NORMAL' failed.
 /etc/redhat-release: Red Hat Enterprise Linux Server release 7.6 (Maipo)
 glibc version: 2.17
 glibc release: stable
Last errno: 0
Threads running: 71
Runtime: 53.409999s
argv: [splunkd -p 8089 restart]
Regex JIT enabled
using CLOCK_MONOTONIC
Thread: "TcpChannelThread", did_join=0, ready_to_run=Y, main_thread=N
First 8 bytes of Thread token @0x7fd2826ac510:
00000000  00 e7 1f 78 d2 7f 00 00                           |...x....|
00000008
commandForThread=0, nextIdle=0x7fd2826a1200, requestAfterThread=0, _tpfd=0x7fd282622800, writeCorkCount=0, terminateCallback=(nil), ioError=No error, lastError=No error, terminateError=No error
giveCmd @0x7fd2826ac680: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, cmd=0, ok=Y, chan=(nil)
writeDataAvail @0x7fd2826ac6e0: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, chan=(nil)
wbuf: ptr=0x7fd2826ac780, size=0x8000, rptr=0x0, wptr=0x0
HttpListeningConnection: _transactionActive=Y, _haveHadTransaction=Y, _alreadyLoggedTimeout=N
HttpTcpConnection: peer=10.223.x.x, _desiredCompressionLevel=6
RestHttpServerTransaction: _restPath="apps/local", namespaced=N, context=-/-, session=[user=admin, refcnt=7, touched=1594208848, refreshEligible=1594209030, removed=N, id=9c9fcfeebea8f184466a7f3affad8eba, created=1594208805, expires=1594212405, initialLife=3600, createdBy=BFD1E27C-45EB-4B06-8A2C-330A2E2ABA35, ip=10.223.239.59, csrf=3758827135434845639]
HttpServerTransaction: _state=6, _shouldLog=Y, _startTime=1594208848.524956575153
REQUEST: POST /it-IT/splunkd/__raw/services/apps/local HTTP/1.1
    Host: 10.187.80.196:8000
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0
    Accept: text/javascript, text/html, application/xml, text/xml, */*
    Accept-Language: it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3
    Accept-Encoding: gzip, deflate
    X-Splunk-Form-Key: 3758827135434845639
    Content-Type: application/x-www-form-urlencoded; charset=UTF-8
    X-Requested-With: XMLHttpRequest
    Content-Length: 154
    Origin: http://10.x.x.x:8000
    Connection: keep-alive
    Cookie: session_id_8000=5ee84200ab939d8fcd6be5586971ec00ee6f8bcf; splunkweb_csrf_token_8000=3758827135434845639; splunkd_8000=FSoqtVC0ZhAOn3Pt8kcBvhOaOm0Z9D8Gji5M5^TZSup_8EuvGKgseg4Qdso7Vw4XQ_Q3^yd2eHeJczDPZIaM_C_32bhM8nuewdP22HyZMlY9qW4Mp_XfQVTk0X8e; csrftoken=4QQp5WpLEnIkk2Vw6UxiWQOMoCuXZs3Vrk4SOKB3lmCLuLf4Ks4JEZ770LBSApmX; sessionid=pm4b2jwciiwt2zx1e8o2r11h1pvr6eka
  _bytesReceived=154, _maximumRequestDataSize=524288000, _totalBytesExpectedOfRequestData=154
  _bytesLeftInRequestDataChunk=0, _requestTransferEncodingIsChunked=N, _receivingRequestDataForever=N
  _needToSetupRequestGunzip=N, _owedConsume=0, _wantSavedRequestData=N
  _100continue=0, _expectDisconnect=N, _overrideSourceState=0
POST arguments: {["filename"] = "true", ["auth"] = "pm4b2jwciiwt2zx1e8o2r11h1pvr6eka", ["name"] = "https://splunkbase.splunk.com/app/4335/release/3.2.1/download/"}
REPLY: 200 
    Set-Cookie: splunkd_8000=FSoqtVC0ZhAOn3Pt8kcBvhOaOm0Z9D8Gji5M5^TZSup_8EuvGKgseg4Qdso7Vw4XQ_Q3^yd2eHeJczDPZIaM_C_32bhM8nuewdP22HyZMlY9qW4Mp_XfQVTk0X8e; Path=/; HttpOnly; Max-Age=3600; Expires=Wed, 08 Jul 2020 12:47:28 GMT
    Set-Cookie: splunkweb_csrf_token_8000=3758827135434845639; Path=/; Max-Age=157680000; Expires=Mon, 07 Jul 2025 11:47:28 GMT
admin_handler="localapps"
MConfigHandler: name=localapps, _atomFormat=2, _customAction=
  caller args: id="https://splunkbase.splunk.com/app/4335/release/3.2.1/download/": { auth -> { _dataType=string _isMultiValue=Y, _values: ["pm4b2jwciiwt2zx1e8o2r11h1pvr6eka"] }, filename -> { _dataType=string _isMultiValue=Y, _values: ["true"] } }
  _docShowEntry=Y, _didFilter=N, _didPaginate=N
  _maxCount=30, posOffset=0, _requestedAction=1
  _shouldFilter=N, _shouldReload=N, _shouldAutoList=N, _sortSpecified=N
  _strict_mode=N, _list_new=N, _force_stanza_overwite=N, _force_app_context_on_write=Y
  sort keys: ["name"]
  sort modes: ["auto"]
  supported args: ["auth" type=0, "author" type=0, "configured" type=0, "deploy" type=0, "description" type=0, "explicit_appname" type=0, "filename" type=0, "label" type=0, "local_bundle" type=0, "name" type=0 (required), "session" type=0, "template" type=0, "update" type=0, "version" type=0, "visible" type=0]
  Paginator: offset=0, count=30
  _customStatusCode=0, _supportedActions=0x1EF, hasSession=Y
  _forceBoolNormalization=N  _forceRuleValidation=N, _contextMode=0, _didCapCheck=Y
  _ranSetup=Y, _restartRequired=N, _listingOne=N
  _userName=admin, _appName=search
ServicesEndpointReplyDataProvider: _setupState=0, _outputMode=2, _explicitOutputMode=Y
GET args: {}
  _allowedMethods={GET,POST,PUT,DELETE,HEAD,OPTIONS}, _preconditionState=0
  _wantsSeparateThread=N, _alreadyBuiltHeaders=N, _needToSendBody=Y
  _bodyBytesWritten=0, _chunkedState=0, _isLastTransaction=N
  _varyBy=0x8, _redirectUrl="", _downloadFilename="", _totalScheduledLength=0
  _willSendDataLater=N, _toSendState=0, _toSendSafe=Y
  _knowCompleteLength=N, _desiredCompressionLevel=6
  _replyContentEncoding= << identity, _cacheControl=0x0, _maxCacheSeconds=4294967295, _dontIncludeFrameOptions=N
In TcpChannel 0x7fd282622800, _tcloop=0x7fd28a8ae2f8, no async write data, _data._shouldKill=N, r/w_timeouts=5.000/300.000, timeout_count=0
SSL: inactive
rbuf: ptr=0x7fd2826228a0, size=0x2000, rptr=0x0, wptr=0x0
TcpChannelAcceptor: , tcloop=0x7fd28a8ae2f8, _disabledReasons=0, _activeCount=6, _inflightSubordinateAccepts=0
HttpListener: ssl=N, _maxActiveConnections=1365, _wellBelowConnectionLimit=Y, _maxThreads=1365
SplunkdHttpListener: PORT: _allowGzip=Y, bind=http://:8000
  conf: _sslopt={rootCAPath="", caCertFile="", certFile="", privateKeyFile="", privateKeyPassword_set=N, commonNameToCheck="", altNameToCheck="", allowSslRenegotiation=Y, sslVersions="SSL3,TLS1.0,TLS1.1,TLS1.2", cipherSuite="", ecdhCurves="", useCompression==N, useSSL=2, quietShutdown==NdhFile="", shouldVerifyClientCert=N}, _allowSslRenegotiation=Y, _frameOptionsSameOrigin=Y, _strictTransportSecurityHeader=N, _allowBasicAuth=N, _allowCookieAuth=N
  conf: _streamInWriteTimeout=5.000, _maxContentLength=524288000, _maxThreads=1365, _maxSockets=1365, _forceHttp10=0
_thread=0x7fd2826ac500: commandForThread=0, nextIdle=0x7fd2826a1200, requestAfterThread=0, _tpfd=0x7fd282622800, writeCorkCount=0, terminateCallback=(nil), ioError=No error, lastError=No error, terminateError=No error
giveCmd @0x7fd2826ac680: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, cmd=0, ok=Y, chan=(nil)
writeDataAvail @0x7fd2826ac6e0: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, chan=(nil)
wbuf: ptr=0x7fd2826ac780, size=0x8000, rptr=0x0, wptr=0x0
First 512 bytes of PolledFd object @0x7fd2a1b05c40:
00000000  08 27 5c 4f de 55 00 00  00 00 00 00 00 00 00 00  |.'\O.U..........|
00000010  20 a2 67 4f de 55 00 00  00 00 00 00 00 00 00 00  | .gO.U..........|
00000020  e0 73 ba a1 d2 7f 00 00  00 00 00 00 00 00 00 00  |.s..............|
00000030  1e 00 00 00 00 00 00 00  5e 00 00 00 61 00 00 00  |........^...a...|
00000040  80 25 bc a1 d2 7f 00 00  1b 20 00 00 d2 7f 00 00  |.%....... ......|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  c8 71 5c 4f de 55 00 00  00 00 00 00 00 00 00 00  |.q\O.U..........|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  68 c6 1f 78 d2 7f 00 00  77 d4 8c 00 00 00 00 00  |h..x....w.......|
00000090  26 20 c2 3a 00 00 00 00  a0 86 01 00 00 00 00 00  |& .:............|
000000a0  40 5c b0 a1 d2 7f 00 00  00 00 00 00 d2 7f 00 00  |@\..............|
000000b0  60 c6 1f 78 d2 7f 00 00  80 a8 8a 8a d2 7f 00 00  |`..x............|
000000c0  00 77 ba a1 d2 7f 00 00  01 01 00 00 d2 7f 00 00  |.w..............|
000000d0  20 5d b0 a1 d2 7f 00 00  00 00 00 00 00 00 00 00  | ]..............|
000000e0  00 00 00 00 00 00 00 00  00 01 b0 a1 d2 7f 00 00  |................|
000000f0  ff ff ff ff d2 7f 00 00  48 5d b0 a1 d2 7f 00 00  |........H]......|
00000100  00 00 00 00 00 00 00 00  00 89 86 8a d2 7f 00 00  |................|
00000110  60 5d b0 a1 d2 7f 00 00  18 20 00 00 1b 20 00 00  |`]....... ... ..|
00000120  e0 2e 00 00 00 00 00 00  a0 86 01 00 00 00 00 00  |................|
00000130  a0 86 01 00 00 00 00 00  d0 07 00 00 00 00 00 00  |................|
00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000170  00 00 00 00 00 00 00 00  c0 5d b0 a1 d2 7f 00 00  |.........]......|
00000180  00 00 00 00 00 00 00 00  30 c2 67 4f de 55 00 00  |........0.gO.U..|
00000190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001a0  00 00 00 00 00 00 00 00  f1 02 00 00 00 00 00 00  |................|
000001b0  48 54 54 50 2f 31 2e 31  20 33 30 32 20 46 6f 75  |HTTP/1.1 302 Fou|
000001c0  6e 64 0d 0a 43 6f 6e 74  65 6e 74 2d 54 79 70 65  |nd..Content-Type|
000001d0  3a 20 74 65 78 74 2f 68  74 6d 6c 3b 20 63 68 61  |: text/html; cha|
000001e0  72 73 65 74 3d 75 74 66  2d 38 0d 0a 44 61 74 65  |rset=utf-8..Date|
000001f0  3a 20 57 65 64 2c 20 30  38 20 4a 75 6c 20 32 30  |: Wed, 08 Jul 20|
00000200
TcpClientConnection:  peer=127.0.0.1, port=3129In TcpOutputLoop 0x7fd2a1b05c40, _toloopp=0x7fd2781fc660, _tstate=1, no async write data, isTerminated=N, destLoop=(nil), c/r/w/s timeouts=12.000/100.000/100.000/2.000, paused=0, timeout_count=0, ssl_shutdown_returned_zero=N
SSL: version="TLSv1.2", state="SSL negotiation finished successfully", cipher="ECDHE-RSA-AES128-GCM-SHA256", compression="none"
serr: No error, _wantEvents=8216, _setEvents=8219
rbuf: ptr=0x7fd2a1b05df0, size=0x4000, rptr=0x0, wptr=0x2F1
HttpClientConnection: _hc_state=12, _gunzip_initialized=N  _had_previous_transactions=Y, _can_reuse_connection=Y
ApplicationUpdateTransaction: file="/opt/splunk/var/run/98f4c71e0bb74726.tar.gz", failureStr="", open=N
HttpClientTransaction: Connecting to host=http://127.0.0.1:3129
Request details: GET https://cdn.apps.splunk.com/media/private/4461d6f0-ad99-11ea-b41b-06add55d78f8.tgz?response-content-disposition=attachment%3Bfilename%3D%22misp42splunk_321.tgz%22&Expires=1594210348&Signature=Ui3RerZf5bPFUMmx2nWPhPo1c8FWt4u7M-BxHaEFYySybOhTRMlgbdLcElE03VN8B18qw97j16XbT~YbcrSR7Cup69HFk53~sORx1Th9~4bnaAJ4defPvIJ-31CMHkjy6MwFA6QWsBf~obVtNOtn6zzT2eY7E48fPMnmqk1KuljlUS04HK40Yz5QGOpHgekM96SFdezdOp3cJV9wcxcYztIWwKNEdOQZFnlXfmvqfM0gywIIDvtoj4Y2Knj~pXrku83hf3igDNEWdZXzXqSkqodQbkclQ-6tYn9GObcK5QJt6sDpxu4trYwM-eBEYdQcgaR98PQbWUK-swW07UThIg__&Key-Pair-Id=APKAISM7Q7KZPNKOIT7A
    X-Auth-Token: pm4b2jwciiwt2zx1e8o2r11h1pvr6eka
  _lastError=No error, _terminateEloopAfter=Y
  _connect_done=Y, _addrElem=0, _connectErrorPriority=0, _resolveError=""
  _useHttp11=Y, _allowTrailers=Y, _use_idle_connection=Y, _avoid_idle_connection_for_next_only=N, _last_on_connection=N, _send_content_type_even_if_no_body=N, _sniToSend=""
  _interpret_redirects=Y, _redirects_left=29, _redirectReply=2
  _doneSendingRequestData=N, _requestBytesExpected=0
  RESPONSE: HTTP/1.1 302 Found
    Content-Type: text/html; charset=utf-8
    Date: Wed, 08 Jul 2020 11:47:28 GMT
    Location: https://cdn.apps.splunk.com/media/private/4461d6f0-ad99-11ea-b41b-06add55d78f8.tgz?response-content-disposition=attachment%3Bfilename%3D%22misp42splunk_321.tgz%22&Expires=1594210348&Signature=Ui3RerZf5bPFUMmx2nWPhPo1c8FWt4u7M-BxHaEFYySybOhTRMlgbdLcElE03VN8B18qw97j16XbT~YbcrSR7Cup69HFk53~sORx1Th9~4bnaAJ4defPvIJ-31CMHkjy6MwFA6QWsBf~obVtNOtn6zzT2eY7E48fPMnmqk1KuljlUS04HK40Yz5QGOpHgekM96SFdezdOp3cJV9wcxcYztIWwKNEdOQZFnlXfmvqfM0gywIIDvtoj4Y2Knj~pXrku83hf3igDNEWdZXzXqSkqodQbkclQ-6tYn9GObcK5QJt6sDpxu4trYwM-eBEYdQcgaR98PQbWUK-swW07UThIg__&Key-Pair-Id=APKAISM7Q7KZPNKOIT7A
    Server: Apache
    Vary: Cookie
    Content-Length: 0
    Connection: keep-alive
  _bytesRx=0, _maybeCompressedBytesRx=0, _bytesExpected=0, _maxResponseSize=576460752303423487
  _acceptAndPass=identity, _acceptAndDecompress=identity, _activeDecompressPolicy=0, _remoteIndicatedCompression=identity
  _connectTimeout=10.000, _readTimeout=100.000, _writeTimeout=100.000
TcpClientConnectionPool: allowSsl=Y, _idleCount=0, _maxIdle=25, _addressOrder=0
  _sslShutdownTimeout=2.000, _idleTimeout=28.000, _idle_connection_trimmer_scheduled=N
x86 CPUID registers:
         0: 0000000D 756E6547 6C65746E 49656E69
         1: 00050654 1E010800 FEFA3203 0FABFBFF
         2: 76036301 00F0B5FF 00000000 00C30000
         3: 00000000 00000000 00000000 00000000
         4: 00000000 00000000 00000000 00000000
         5: 00000000 00000000 00000000 00000000
         6: 00000004 00000000 00000000 00000000
         7: 00000000 00000000 00000000 00000000
         8: 00000000 00000000 00000000 00000000
         9: 00000000 00000000 00000000 00000000
         A: 07300401 0000007F 00000000 00000000
         B: 00000000 00000000 000000FD 0000001E
         C: 00000000 00000000 00000000 00000000
         😧 00000000 00000000 00000000 00000000
  80000000: 80000008 00000000 00000000 00000000
  80000001: 00000000 00000000 00000101 2C100800
  80000002: 65746E49 2952286C 6F655820 2952286E
  80000003: 6C6F4720 31352064 43203831 40205550
  80000004: 332E3220 7A484730 00000000 00000000
  80000005: 00000000 00000000 00000000 00000000
  80000006: 00000000 00000000 01006040 00000000
  80000007: 00000000 00000000 00000000 00000100
  80000008: 00003028 00000000 00000000 00000000
terminating...

07-08-2020 13:46:36.009 +0200 INFO  ProxyConfig - Successfully initialized https_proxy=http://127.0.0.1:3129 from server.conf for splunkd.

07-08-2020 13:46:36.009 +0200 INFO  ProxyConfig - Successfully initialized http_proxy=http://127.0.0.1:3129 from server.conf for splunkd.

lies2yk · ‎07-08-2020

And here the complete crash log: Sorry for long text....!

[build 47d8552a4d84] 2020-07-08 15:41:18
Received fatal signal 6 (Aborted).
 Cause:
   Signal sent by PID 54177 running under UID 0.
 Crashing thread: TcpChannelThread
 Registers:
    RIP:  [0x00007FD784903377] gsignal + 55 (libc.so.6 + 0x36377)
    RDI:  [0x000000000000D3A1]
    RSI:  [0x000000000000D4A7]
    RBP:  [0x00007FD784A57C20]
    RSP:  [0x00007FD758DFBE28]
    RAX:  [0x0000000000000000]
    RBX:  [0x00007FD785FE6000]
    RCX:  [0xFFFFFFFFFFFFFFFF]
    RDX:  [0x0000000000000006]
    R8:  [0x00000000000000A0]
    R9:  [0xFEFEFEFEFF092D63]
    R10:  [0x0000000000000008]
    R11:  [0x0000000000000202]
    R12:  [0x0000556F90E44920]
    R13:  [0x0000556F90F6A3C0]
    R14:  [0x00007FD758DFC050]
    R15:  [0x00007FD786038E08]
    EFL:  [0x0000000000000202]
    TRAPNO:  [0x0000000000000000]
    ERR:  [0x0000000000000000]
    CSGSFS:  [0xFFFF000000000033]
    OLDMASK:  [0x0000000000000000]

 OS: Linux
 Arch: x86-64

 Backtrace (PIC build):
  [0x00007FD784903377] gsignal + 55 (libc.so.6 + 0x36377)
  [0x00007FD784904A68] abort + 328 (libc.so.6 + 0x37A68)
  [0x00007FD7848FC196] ? (libc.so.6 + 0x2F196)
  [0x00007FD7848FC242] ? (libc.so.6 + 0x2F242)
  [0x0000556F8FDD2684] _ZN21HttpClientTransaction15_handleRedirectEv + 852 (splunkd + 0x182F684)
  [0x0000556F8FDD3572] _ZN21HttpClientTransaction18_finishTransactionEv + 274 (splunkd + 0x1830572)
  [0x0000556F8FDD476B] _ZN20HttpClientConnection10parseReplyEPKcS1_ + 1691 (splunkd + 0x183176B)
  [0x0000556F8FDD5860] _ZN20HttpClientConnection13dataAvailableEv + 160 (splunkd + 0x1832860)
  [0x0000556F8FE80C77] _ZN11TcpOutbound6_do_ioE18PollableDescriptor + 423 (splunkd + 0x18DDC77)
  [0x0000556F8FE81C51] _ZN11TcpOutbound11when_eventsE18PollableDescriptor + 33 (splunkd + 0x18DEC51)
  [0x0000556F8FDB88B6] _ZN8PolledFd8do_eventEv + 134 (splunkd + 0x18158B6)
  [0x0000556F8FDB988B] _ZN9EventLoop3runEv + 667 (splunkd + 0x181688B)
  [0x0000556F8FE82400] _ZN15TcpOutboundLoop3runEv + 16 (splunkd + 0x18DF400)
  [0x0000556F8FDD08F7] _ZN21HttpClientTransaction22runSyncAndShutdownLoopEP15TcpOutboundLoop + 103 (splunkd + 0x182D8F7)
  [0x0000556F8FDD0A75] _ZN21HttpClientTransaction7runSyncEv + 69 (splunkd + 0x182DA75)
  [0x0000556F8FCC75B1] _ZN18ApplicationUpdater20fetchUpdateFileByUriERK7FullUriRK3StrR28ApplicationUpdateTransaction + 145 (splunkd + 0x17245B1)
  [0x0000556F8FCC7CF1] _ZN18ApplicationUpdater20fetchUpdateFileByUriERK7FullUriRK3StrR8Pathnameb + 385 (splunkd + 0x1724CF1)
  [0x0000556F8FB0F20E] _ZN21LocalAppsAdminHandler13handleInstallER10ConfigInfoPK3StrS4_bbb + 2174 (splunkd + 0x156C20E)
  [0x0000556F8FB0FD6A] _ZN21LocalAppsAdminHandler12handleCreateER10ConfigInfo + 426 (splunkd + 0x156CD6A)
  [0x0000556F8F78C52D] _ZN14MConfigHandler2goER10ConfigInfo + 189 (splunkd + 0x11E952D)
  [0x0000556F8F78D07F] _ZN29AdminManagerReplyDataProvider2goEv + 783 (splunkd + 0x11EA07F)
  [0x0000556F8F8693B8] _ZN33ServicesEndpointReplyDataProvider9rawHandleEv + 88 (splunkd + 0x12C63B8)
  [0x0000556F8F85EEBF] _ZN18RawRestHttpHandler10getPreBodyEP21HttpServerTransaction + 31 (splunkd + 0x12BBEBF)
  [0x0000556F8FDEA390] _ZN32HttpThreadedCommunicationHandler11communicateER17TcpSyncDataBuffer + 272 (splunkd + 0x1847390)
  [0x0000556F8F199432] _ZN16TcpChannelThread4mainEv + 242 (splunkd + 0xBF6432)
  [0x0000556F8FE83638] _ZN6Thread8callMainEPv + 120 (splunkd + 0x18E0638)
  [0x00007FD784CA2EA5] ? (libpthread.so.0 + 0x7EA5)
  [0x00007FD7849CB8CD] clone + 109 (libc.so.6 + 0xFE8CD)
 Linux / ssplkshwsl01v / 3.10.0-1062.7.1.el7.x86_64 / #1 SMP Wed Nov 13 08:44:42 EST 2019 / x86_64
 Last few lines of stderr (may contain info on assertion failure, but also could be old):
    2020-07-08 15:37:44.361 +0200 Interrupt signal received sent by PID 52646 (UID 0)
    2020-07-08 15:38:02.990 +0200 splunkd started (build 47d8552a4d84) pid=52734
    2020-07-08 15:38:37.273 +0200 Interrupt signal received sent by PID 53122 (UID 0)
    2020-07-08 15:38:55.899 +0200 splunkd started (build 47d8552a4d84) pid=53213
    2020-07-08 15:39:41.400 +0200 Interrupt signal received sent by PID 53607 (UID 0)
    2020-07-08 15:39:59.024 +0200 splunkd started (build 47d8552a4d84) pid=53694
    2020-07-08 15:40:29.952 +0200 Interrupt signal received sent by PID 54019 (UID 0)
    2020-07-08 15:40:46.579 +0200 splunkd started (build 47d8552a4d84) pid=54177
    splunkd: /opt/splunk/src/util/HttpClientRequest.cpp:1895: void HttpClientTransaction::_handleRedirect(): Assertion `_redirectReply == REPLY_EATING_NORMAL' failed.

 /etc/redhat-release: Red Hat Enterprise Linux Server release 7.6 (Maipo)
 glibc version: 2.17
 glibc release: stable
Last errno: 0
Threads running: 72
Runtime: 32.336326s
argv: [splunkd -p 8089 restart]
Regex JIT enabled

using CLOCK_MONOTONIC
Thread: "TcpChannelThread", did_join=0, ready_to_run=Y, main_thread=N
First 8 bytes of Thread token @0x7fd764aa7c10:
00000000  00 e7 df 58 d7 7f 00 00                           |...X....|
00000008
commandForThread=0, nextIdle=0x7fd764a9ca40, requestAfterThread=0, _tpfd=0x7fd764a22800, writeCorkCount=0, terminateCallback=(nil), ioError=No error, lastError=No error, terminateError=No error
giveCmd @0x7fd764aa7d80: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, cmd=0, ok=Y, chan=(nil)
writeDataAvail @0x7fd764aa7de0: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, chan=0x7fd764a27800
wbuf: ptr=0x7fd764aa7e80, size=0x8000, rptr=0x0, wptr=0x0
HttpListeningConnection: _transactionActive=Y, _haveHadTransaction=Y, _alreadyLoggedTimeout=N
HttpTcpConnection: peer=10.22.x.x, _desiredCompressionLevel=6
RestHttpServerTransaction: _restPath="apps/local", namespaced=N, context=-/-, session=[user=admin, refcnt=7, touched=1594215678, refreshEligible=1594215881, removed=N, id=d385f669fbf4984aa1cb560528dfaa30, created=1594215656, expires=1594219256, initialLife=3600, createdBy=BFD1E27C-45EB-4B06-8A2C-330A2E2ABA35, ip=10.22.x.x, csrf=10733806047818363417]
HttpServerTransaction: _state=6, _shouldLog=Y, _startTime=1594215678.515153535249
REQUEST: POST /it-IT/splunkd/__raw/services/apps/local HTTP/1.1
    Host: 10.x.x.x:8000
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0
    Accept: text/javascript, text/html, application/xml, text/xml, */*
    Accept-Language: it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3
    Accept-Encoding: gzip, deflate
    X-Splunk-Form-Key: 10733806047818363417
    Content-Type: application/x-www-form-urlencoded; charset=UTF-8
    X-Requested-With: XMLHttpRequest
    Content-Length: 154
    Origin: http://10.x.x.x:8000
    Connection: keep-alive
    Cookie: session_id_8000=a9c42a41b5e763284dac4c49d4b06841b1d82666; splunkweb_csrf_token_8000=10733806047818363417; splunkd_8000=5QQJr40SASizD5Al8z600hxQLcvPoDcT1hdAK8WOYqoGS2fxjtXzRwbZc4bLMrQRY6EcJjHeLVOvdGeFwnQVaH1Dy^kFIoQM1mLhG74CGNKFmNcYJzRnBkIVI7FvPUm3fC; csrftoken=fvvvLb5LgzUUPaulTpMMAo0bIHZDZUHFHhBRJLv7ouWAnZA0TKxJZdtNMv5MX26K; sessionid=pm4b2jwciiwt2zx1e8o2r11h1pvr6eka
  _bytesReceived=154, _maximumRequestDataSize=524288000, _totalBytesExpectedOfRequestData=154
  _bytesLeftInRequestDataChunk=0, _requestTransferEncodingIsChunked=N, _receivingRequestDataForever=N
  _needToSetupRequestGunzip=N, _owedConsume=0, _wantSavedRequestData=N
  _100continue=0, _expectDisconnect=N, _overrideSourceState=0
POST arguments: {["filename"] = "true", ["auth"] = "pm4b2jwciiwt2zx1e8o2r11h1pvr6eka", ["name"] = "https://splunkbase.splunk.com/app/4335/release/3.2.1/download/"}
REPLY: 200 
    Set-Cookie: splunkd_8000=5QQJr40SASizD5Al8z600hxQLcvPoDcT1hdAK8WOYqoGS2fxjtXzRwbZc4bLMrQRY6EcJjHeLVOvdGeFwnQVaH1Dy^kFIoQM1mLhG74CGNKFmNcYJzRnBkIVI7FvPUm3fC; Path=/; HttpOnly; Max-Age=3600; Expires=Wed, 08 Jul 2020 14:41:18 GMT
    Set-Cookie: splunkweb_csrf_token_8000=10733806047818363417; Path=/; Max-Age=157680000; Expires=Mon, 07 Jul 2025 13:41:18 GMT
admin_handler="localapps"
MConfigHandler: name=localapps, _atomFormat=2, _customAction=
  caller args: id="https://splunkbase.splunk.com/app/4335/release/3.2.1/download/": { auth -> { _dataType=string _isMultiValue=Y, _values: ["pm4b2jwciiwt2zx1e8o2r11h1pvr6eka"] }, filename -> { _dataType=string _isMultiValue=Y, _values: ["true"] } }
  _docShowEntry=Y, _didFilter=N, _didPaginate=N
  _maxCount=30, posOffset=0, _requestedAction=1
  _shouldFilter=N, _shouldReload=N, _shouldAutoList=N, _sortSpecified=N
  _strict_mode=N, _list_new=N, _force_stanza_overwite=N, _force_app_context_on_write=Y
  sort keys: ["name"]
  sort modes: ["auto"]
  supported args: ["auth" type=0, "author" type=0, "configured" type=0, "deploy" type=0, "description" type=0, "explicit_appname" type=0, "filename" type=0, "label" type=0, "local_bundle" type=0, "name" type=0 (required), "session" type=0, "template" type=0, "update" type=0, "version" type=0, "visible" type=0]
  Paginator: offset=0, count=30
  _customStatusCode=0, _supportedActions=0x1EF, hasSession=Y
  _forceBoolNormalization=N  _forceRuleValidation=N, _contextMode=0, _didCapCheck=Y
  _ranSetup=Y, _restartRequired=N, _listingOne=N
  _userName=admin, _appName=search
ServicesEndpointReplyDataProvider: _setupState=0, _outputMode=2, _explicitOutputMode=Y
GET args: {}
  _allowedMethods={GET,POST,PUT,DELETE,HEAD,OPTIONS}, _preconditionState=0
  _wantsSeparateThread=N, _alreadyBuiltHeaders=N, _needToSendBody=Y
  _bodyBytesWritten=0, _chunkedState=0, _isLastTransaction=N
  _varyBy=0x8, _redirectUrl="", _downloadFilename="", _totalScheduledLength=0
  _willSendDataLater=N, _toSendState=0, _toSendSafe=Y
  _knowCompleteLength=N, _desiredCompressionLevel=6
  _replyContentEncoding= << identity, _cacheControl=0x0, _maxCacheSeconds=4294967295, _dontIncludeFrameOptions=N
In TcpChannel 0x7fd764a22800, _tcloop=0x7fd769ea82f8, no async write data, _data._shouldKill=N, r/w_timeouts=5.000/300.000, timeout_count=0
SSL: inactive
rbuf: ptr=0x7fd764a228a0, size=0x2000, rptr=0x0, wptr=0x0
TcpChannelAcceptor: , tcloop=0x7fd769ea82f8, _disabledReasons=0, _activeCount=6, _inflightSubordinateAccepts=0
HttpListener: ssl=N, _maxActiveConnections=1365, _wellBelowConnectionLimit=Y, _maxThreads=1365
SplunkdHttpListener: PORT: _allowGzip=Y, bind=http://:8000
  conf: _sslopt={rootCAPath="", caCertFile="", certFile="", privateKeyFile="", privateKeyPassword_set=N, commonNameToCheck="", altNameToCheck="", allowSslRenegotiation=Y, sslVersions="SSL3,TLS1.0,TLS1.1,TLS1.2", cipherSuite="", ecdhCurves="", useCompression==N, useSSL=2, quietShutdown==NdhFile="", shouldVerifyClientCert=N}, _allowSslRenegotiation=Y, _frameOptionsSameOrigin=Y, _strictTransportSecurityHeader=N, _allowBasicAuth=N, _allowCookieAuth=N
  conf: _streamInWriteTimeout=5.000, _maxContentLength=524288000, _maxThreads=1365, _maxSockets=1365, _forceHttp10=0
_thread=0x7fd764aa7c00: commandForThread=0, nextIdle=0x7fd764a9ca40, requestAfterThread=0, _tpfd=0x7fd764a22800, writeCorkCount=0, terminateCallback=(nil), ioError=No error, lastError=No error, terminateError=No error
giveCmd @0x7fd764aa7d80: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, cmd=0, ok=Y, chan=(nil)
writeDataAvail @0x7fd764aa7de0: _queuedOn=(nil), ran=N, wantWake=N, wantFailIfLoopDone=N, chan=0x7fd764a27800
wbuf: ptr=0x7fd764aa7e80, size=0x8000, rptr=0x0, wptr=0x0

First 512 bytes of PolledFd object @0x7fd781abc580:
00000000  08 47 70 91 6f 55 00 00  00 00 00 00 00 00 00 00  |.Gp.oU..........|
00000010  20 c2 7b 91 6f 55 00 00  00 00 00 00 00 00 00 00  | .{.oU..........|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 00 00 00  95 00 00 00 40 00 00 00  |............@...|
00000040  40 1a ab 81 d7 7f 00 00  1b 20 00 00 00 00 00 00  |@........ ......|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  c8 91 70 91 6f 55 00 00  00 00 00 00 00 00 00 00  |..p.oU..........|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  68 c6 df 58 d7 7f 00 00  25 ef 8c 00 00 00 00 00  |h..X....%.......|
00000090  06 7b c8 35 00 00 00 00  a0 86 01 00 00 00 00 00  |.{.5............|
000000a0  80 c5 ab 81 d7 7f 00 00  00 00 00 00 00 00 00 00  |................|
000000b0  60 c6 df 58 d7 7f 00 00  80 78 ea 69 d7 7f 00 00  |`..X.....x.i....|
000000c0  00 d0 a5 81 d7 7f 00 00  01 01 00 00 00 00 00 00  |................|
000000d0  60 c6 ab 81 d7 7f 00 00  00 00 00 00 00 00 00 00  |`...............|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000f0  ff ff ff ff 00 00 00 00  88 c6 ab 81 d7 7f 00 00  |................|
00000100  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000110  00 00 00 00 00 00 00 00  18 20 00 00 1b 20 00 00  |......... ... ..|
00000120  e0 2e 00 00 00 00 00 00  a0 86 01 00 00 00 00 00  |................|
00000130  a0 86 01 00 00 00 00 00  d0 07 00 00 00 00 00 00  |................|
00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000170  00 00 00 00 00 00 00 00  00 c7 ab 81 d7 7f 00 00  |................|
00000180  00 00 00 00 00 00 00 00  30 e2 7b 91 6f 55 00 00  |........0.{.oU..|
00000190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001a0  00 00 00 00 00 00 00 00  f1 02 00 00 00 00 00 00  |................|
000001b0  48 54 54 50 2f 31 2e 31  20 33 30 32 20 46 6f 75  |HTTP/1.1 302 Fou|
000001c0  6e 64 0d 0a 43 6f 6e 74  65 6e 74 2d 54 79 70 65  |nd..Content-Type|
000001d0  3a 20 74 65 78 74 2f 68  74 6d 6c 3b 20 63 68 61  |: text/html; cha|
000001e0  72 73 65 74 3d 75 74 66  2d 38 0d 0a 44 61 74 65  |rset=utf-8..Date|
000001f0  3a 20 57 65 64 2c 20 30  38 20 4a 75 6c 20 32 30  |: Wed, 08 Jul 20|
00000200
TcpClientConnection:  peer=127.0.0.1, port=3128In TcpOutputLoop 0x7fd781abc580, _toloopp=0x7fd758dfc660, _tstate=1, no async write data, isTerminated=N, destLoop=(nil), c/r/w/s timeouts=12.000/100.000/100.000/2.000, paused=0, timeout_count=0, ssl_shutdown_returned_zero=N
SSL: version="TLSv1.2", state="SSL negotiation finished successfully", cipher="ECDHE-RSA-AES128-GCM-SHA256", compression="none"

serr: No error, _wantEvents=8216, _setEvents=8219
rbuf: ptr=0x7fd781abc730, size=0x4000, rptr=0x0, wptr=0x2F1
HttpClientConnection: _hc_state=12, _gunzip_initialized=N  _had_previous_transactions=Y, _can_reuse_connection=Y
ApplicationUpdateTransaction: file="/opt/splunk/var/run/9e609a5b8abf3c36.tar.gz", failureStr="", open=N
HttpClientTransaction: Connecting to host=http://127.0.0.1:3128
Request details: GET https://cdn.apps.splunk.com/media/private/4461d6f0-ad99-11ea-b41b-06add55d78f8.tgz?response-content-disposition=attachment%3Bfilename%3D%22misp42splunk_321.tgz%22&Expires=1594217178&Signature=E-fAq-mlh5a1h5vxCyFLx39m2e6k1w54a3C4bY58UcXeagrZU-A2rakFGsVCjbmFbeX-NtHqfa1~Gv7Pdh27eX9QZrp6alHVxd1E9QnhbF3C9aEZAPlaeCbMzXCoPPFsyJt5H1l1joux0lSTU5tcr3VLPtAtYO5bcn6B5vD3lOvjvQ0zX-0WyXhofkCXTi4SMUOeXse51hvwX16DB4BVoYp8sdJ28yqmoao2-BIZPxLbOI6VNUE275T13jPcNFPqhRTRwFEKEqgSCqopfoeddR3gFqO5ngoEJufA622HcoZnwyFmbkkQn9CUAAQORUKDPaz7MXIHoPBoTlopdXAPbw__&Key-Pair-Id=APKAISM7Q7KZPNKOIT7A
    X-Auth-Token: pm4b2jwciiwt2zx1e8o2r11h1pvr6eka
  _lastError=No error, _terminateEloopAfter=Y
  _connect_done=Y, _addrElem=0, _connectErrorPriority=0, _resolveError=""
  _useHttp11=Y, _allowTrailers=Y, _use_idle_connection=Y, _avoid_idle_connection_for_next_only=N, _last_on_connection=N, _send_content_type_even_if_no_body=N, _sniToSend=""
  _interpret_redirects=Y, _redirects_left=29, _redirectReply=2
  _doneSendingRequestData=N, _requestBytesExpected=0
  RESPONSE: HTTP/1.1 302 Found
    Content-Type: text/html; charset=utf-8
    Date: Wed, 08 Jul 2020 13:41:18 GMT
    Location: https://cdn.apps.splunk.com/media/private/4461d6f0-ad99-11ea-b41b-06add55d78f8.tgz?response-content-disposition=attachment%3Bfilename%3D%22misp42splunk_321.tgz%22&Expires=1594217178&Signature=E-fAq-mlh5a1h5vxCyFLx39m2e6k1w54a3C4bY58UcXeagrZU-A2rakFGsVCjbmFbeX-NtHqfa1~Gv7Pdh27eX9QZrp6alHVxd1E9QnhbF3C9aEZAPlaeCbMzXCoPPFsyJt5H1l1joux0lSTU5tcr3VLPtAtYO5bcn6B5vD3lOvjvQ0zX-0WyXhofkCXTi4SMUOeXse51hvwX16DB4BVoYp8sdJ28yqmoao2-BIZPxLbOI6VNUE275T13jPcNFPqhRTRwFEKEqgSCqopfoeddR3gFqO5ngoEJufA622HcoZnwyFmbkkQn9CUAAQORUKDPaz7MXIHoPBoTlopdXAPbw__&Key-Pair-Id=APKAISM7Q7KZPNKOIT7A
    Server: Apache
    Vary: Cookie
    Content-Length: 0
    Connection: keep-alive
  _bytesRx=0, _maybeCompressedBytesRx=0, _bytesExpected=0, _maxResponseSize=576460752303423487
  _acceptAndPass=identity, _acceptAndDecompress=identity, _activeDecompressPolicy=0, _remoteIndicatedCompression=identity
  _connectTimeout=10.000, _readTimeout=100.000, _writeTimeout=100.000
TcpClientConnectionPool: allowSsl=Y, _idleCount=2, _maxIdle=25, _addressOrder=0
  _sslShutdownTimeout=2.000, _idleTimeout=28.000, _idle_connection_trimmer_scheduled=N


x86 CPUID registers:
         0: 0000000D 756E6547 6C65746E 49656E69
         1: 00050654 10010800 FEFA3203 0FABFBFF
         2: 76036301 00F0B5FF 00000000 00C30000
         3: 00000000 00000000 00000000 00000000
         4: 00000000 00000000 00000000 00000000
         5: 00000000 00000000 00000000 00000000
         6: 00000004 00000000 00000000 00000000
         7: 00000000 00000000 00000000 00000000
         8: 00000000 00000000 00000000 00000000
         9: 00000000 00000000 00000000 00000000
         A: 07300401 0000007F 00000000 00000000
         B: 00000000 00000000 000000FD 00000010
         C: 00000000 00000000 00000000 00000000
         😧 00000000 00000000 00000000 00000000
  80000000: 80000008 00000000 00000000 00000000
  80000001: 00000000 00000000 00000101 2C100800
  80000002: 65746E49 2952286C 6F655820 2952286E
  80000003: 6C6F4720 31352064 43203831 40205550
  80000004: 332E3220 7A484730 00000000 00000000
  80000005: 00000000 00000000 00000000 00000000
  80000006: 00000000 00000000 01006040 00000000
  80000007: 00000000 00000000 00000000 00000100
  80000008: 00003028 00000000 00000000 00000000
terminating...

Splunk 8.0.2 and proxy conf

Linux

.conf24 | Registration Open!

ICYMI - Check out the latest releases of Splunk Edge Processor

Introducing the 2024 SplunkTrust!