Here are the contents of the /opt/splunk/etc/deployment-apps:
[splunk@splunk deployment-apps]$ ls
README Splunk_TA_nix Splunk_TA_windows
[splunk@splunk deployment-apps]$ pwd
/opt/splunk/etc/deployment-appsHowever upon running either splunk reload deploy-server or splunk restart, while no errors are being generated, the deployment apps displayed on the web UI are not updating:
The entire path /opt/splunk/etc/deployment-apps and its subfolders are owned by the splunk user.
Here are the results when running splunk reload deploy-server -debug:
[splunk@splunk deployment-apps]$ splunk reload deploy-server -debug > reload
Will setenv SPLUNK_CLI_DEBUG to "v".
In check_and_set_splunk_os_user(): In env found SPLUNK_OS_USER var: "splunk"
In cmd_rc
Will exec (detach=no): LDAPCONF=/disk2/splunk/etc/openldap/ldap.conf HOSTNAME=splunk.cdc.health.local SPLUNK_SERVER_NAME=Splunkd SPLUNK_HOME=/disk2/splunk USER=splunk LD_LIBRARY_PATH=/disk2/splunk/lib:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64/jli PATH=/disk2/splunk/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/splunk/.local/bin:/home/splunk/bin:/disk2/splunk/bin PWD=/opt/splunk/etc/deployment-apps SPLUNK_DB=/export/intindx NODE_PATH=/disk2/splunk/lib/node_modules SPLUNK_OS_USER=splunk /disk2/splunk/bin/splunkd btool web list settings --no-log
Will exec (detach=no): LDAPCONF=/disk2/splunk/etc/openldap/ldap.conf HOSTNAME=splunk.cdc.health.local SPLUNK_SERVER_NAME=Splunkd SPLUNK_HOME=/disk2/splunk USER=splunk LD_LIBRARY_PATH=/disk2/splunk/lib:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64/jli PATH=/disk2/splunk/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/splunk/.local/bin:/home/splunk/bin:/disk2/splunk/bin PWD=/opt/splunk/etc/deployment-apps SPLUNK_DB=/export/intindx NODE_PATH=/disk2/splunk/lib/node_modules SPLUNK_OS_USER=splunk /disk2/splunk/bin/splunkd btool server list general --no-log
Will exec (detach=no): LDAPCONF=/disk2/splunk/etc/openldap/ldap.conf HOSTNAME=splunk.cdc.health.local SPLUNK_SERVER_NAME=Splunkd SPLUNK_HOME=/disk2/splunk USER=splunk LD_LIBRARY_PATH=/disk2/splunk/lib:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64/jli PATH=/disk2/splunk/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/splunk/.local/bin:/home/splunk/bin:/disk2/splunk/bin PWD=/opt/splunk/etc/deployment-apps SPLUNK_DB=/export/intindx NODE_PATH=/disk2/splunk/lib/node_modules SPLUNK_OS_USER=splunk /disk2/splunk/bin/splunkd btool server list kvstore --no-log
Will exec (detach=no): LDAPCONF=/disk2/splunk/etc/openldap/ldap.conf HOSTNAME=splunk.cdc.health.local SPLUNK_SERVER_NAME=Splunkd SPLUNK_HOME=/disk2/splunk USER=splunk LD_LIBRARY_PATH=/disk2/splunk/lib:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64/jli PATH=/disk2/splunk/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/splunk/.local/bin:/home/splunk/bin:/disk2/splunk/bin PWD=/opt/splunk/etc/deployment-apps SPLUNK_DB=/export/intindx NODE_PATH=/disk2/splunk/lib/node_modules SPLUNK_OS_USER=splunk /disk2/splunk/bin/splunkd btool server list dfs --no-log
dynamic conf init: httpport=8000 mgmtHostPort=8089 bind_ip= rest_hostname=127.0.0.1 kvStorePort=8191
base splunkdURL from splunkd: https://127.0.0.1:8089
Will setenv SPLUNK_CLI_DEBUG to "v".
cmd: reload
obj: deploy-server
cli args:
In build_full_rest_url(): Composing URL from base=https://127.0.0.1:8089 + relative=/static/splunkrc_cmds.xml
In build_full_rest_url(): Composed URL=https://127.0.0.1:8089/static/splunkrc_cmds.xml
In make_simple_rest_call_online(): using_basic_auth=0
In make_simple_rest_call_online(): [Re-]Initialized HTTP request headers:
<none>
In make_simple_rest_call_online(): HTTP request response_code=200
In make_simple_rest_call_online(): HTTP response headers:
ETag: "s-t2y1XwAAAAD4ZQsAAAAAAJ8CCgAAAAAANsQDAAAAAAA"
Date: Thu, 01 Apr 2021 20:44:37 GMT
Expires: Thu, 01 Apr 2021 21:44:37 GMT
Cache-Control: max-age=3600
Content-Type: text/xml
X-Content-Type-Options: nosniff
Last-Modified: Wed, 18 Nov 2020 18:49:27 GMT
Content-Length: 246838
Connection: Keep-Alive
X-Frame-Options: SAMEORIGIN
Server: Splunkd
Online REST call to /static/splunkrc_cmds.xml returned 0
Visited node: deploy-server
Entered node:deploy-server/common
Entered node:deploy-server/cmd
Metadata:
cmd: reload
obj: deploy-server
rel_url: (no value)
eai_id: (no value)
implied_arg_name: (no value)
action: 0
args map count: 0
args map:
default args count: 0
initial args count: 0
hooks count: 0
object help
command array count: 0
In call_pcl, will exec_python
Will exec (detach=no): HOSTNAME=splunk.cdc.health.local USER=splunk PATH=/disk2/splunk/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/splunk/.local/bin:/home/splunk/bin:/disk2/splunk/bin PWD=/opt/splunk/etc/deployment-apps SPLUNK_HOME=/disk2/splunk SPLUNK_DB=/export/intindx SPLUNK_SERVER_NAME=Splunkd SPLUNK_OS_USER=splunk NODE_PATH=/disk2/splunk/lib/node_modules PYTHONPATH= LD_LIBRARY_PATH=/disk2/splunk/lib:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64/jli LDAPCONF=/disk2/splunk/etc/openldap/ldap.conf /disk2/splunk/bin/python3.7 -u /disk2/splunk/lib/python3.7/site-packages/splunk/clilib/cli.py reload deploy-server
DEBUG (cli) Running in debug mode.
DEBUG (cli) Command: reload
DEBUG (cli) Subcmd: deploy-server
DEBUG (cli) Begin parsed arguments:
DEBUG (cli)
DEBUG (cli) End parsed arguments.
DEBUG (cli) noAuthReq: False
DEBUG (cli) username: None
DEBUG (cli) password: None
DEBUG (cli_common) Running btool for 'web.conf'.
WARNING (cli_common) btool returned something in stderr: 'Will exec (detach=no): HOSTNAME=splunk.cdc.health.local USER=splunk PATH=/disk2/splunk/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/splunk/.local/bin:/home/splunk/bin:/disk2/splunk/bin PWD=/opt/splunk/etc/deployment-apps SPLUNK_HOME=/disk2/splunk SPLUNK_DB=/export/intindx SPLUNK_SERVER_NAME=Splunkd SPLUNK_OS_USER=splunk NODE_PATH=/disk2/splunk/lib/node_modules LD_LIBRARY_PATH=/disk2/splunk/lib:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64/jli LDAPCONF=/disk2/splunk/etc/openldap/ldap.conf /disk2/splunk/bin/splunkd btool web list
'
DEBUG (cli) Contents of ./splunk/authToken file: <auth><username>splunkadmin</username><sessionkey></sessionkey></auth>
DEBUG (cli) authInfo:
DEBUG (rcUtils) cmd: reload, obj: deploy-server, restArgList: {'authstr':', 'owner': 'splunkadmin'}
DEBUG (rcUtils) endpoint: deployments
DEBUG (rcUtils) layeredFind:default_eai_parms:
DEBUG (rcUtils) authstr poped from argList
DEBUG (rcUtils) layeredFind:app_context:
DEBUG (rcUtils) namespace:
DEBUG (rcUtils) layeredFind:uri: /deployment/server/config/_reload
DEBUG (rcUtils) layeredFind:eai_id:
DEBUG (rcUtils) layeredFind:required:
DEBUG (rcUtils) eai_key_list: []
DEBUG (rcUtils) eai_key_list: []
DEBUG (rcUtils) layeredFind:args:
DEBUG (rcUtils) layeredFind:prehooks:
DEBUG (rcUtils) after prehooks, eaiArgsList: {}
DEBUG (rcUtils) Before buildEndpoint uri: /deployment/server/config/_reload
DEBUG (rcUtils) Before buildEndpoint entityName:
DEBUG (rcUtils) uri: /services/deployment/server/config/_reload
DEBUG (rcUtils) layeredFind:type: edit
DEBUG (rcUtils) eaiArgsList: {}
DEBUG (rcUtils) In sanitizeArgs: target: , argsMap: {}, argsDict: {}
DEBUG (rcUtils) In sanitizeArgs: target: , argsMap: {}, argsDict: {}
DEBUG (rcUtils) In sanitizeArgs: target: /deployment/server/config/_reload, argsMap: {}, argsDict: {}
DEBUG (rcUtils) In sanitizeArgs: target: /deployment/server/config/_reload, argsMap: {}, argsDict: {}
DEBUG (rcUtils) postargs: {}
DEBUG (rcUtils) getargs: {}
DEBUG (__init__) simpleRequest > POST https://127.0.0.1:8089/services/deployment/server/config/_reload [] sessionSource=direct timeout=30
DEBUG (cli_common) Running btool for 'server.conf'.
WARNING (cli_common) btool returned something in stderr: 'Will exec (detach=no): HOSTNAME=splunk.cdc.health.local USER=splunk PATH=/disk2/splunk/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/splunk/.local/bin:/home/splunk/bin:/disk2/splunk/bin PWD=/opt/splunk/etc/deployment-apps SPLUNK_HOME=/disk2/splunk SPLUNK_DB=/export/intindx SPLUNK_SERVER_NAME=Splunkd SPLUNK_OS_USER=splunk NODE_PATH=/disk2/splunk/lib/node_modules LD_LIBRARY_PATH=/disk2/splunk/lib:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64:/disk2/splunk/bin/jars/vendors/java/OpenJDK8U-jre_x64_linux_hotspot_8u242b08/lib/amd64/jli LDAPCONF=/disk2/splunk/etc/openldap/ldap.conf /disk2/splunk/bin/splunkd btool server list
'
DEBUG (__init__) simpleRequest < server responded status=200 responseTime=0.1081s
DEBUG (rcUtils) In checkStatus: type: edit, server_response: {'date': 'Thu, 01 Apr 2021 20:44:37 GMT', 'expires': 'Thu, 26 Oct 1978 00:00:00 GMT', 'cache-control': 'no-store, no-cache, must-revalidate, max-age=0', 'content-type': 'text/xml; charset=UTF-8', 'x-content-type-options': 'nosniff', 'content-length': '3828', 'vary': 'Authorization', 'connection': 'Close', 'x-frame-options': 'SAMEORIGIN', 'server': 'Splunkd', 'status': '200'}
DEBUG (rcDisplay) In displayDeployment: kwargs: {'cmd': 'reload', 'obj': 'deploy-server', 'type': 'edit', 'serverResponse': {'date': 'Thu, 01 Apr 2021 20:44:37 GMT', 'expires': 'Thu, 26 Oct 1978 00:00:00 GMT', 'cache-control': 'no-store, no-cache, must-revalidate, max-age=0', 'content-type': 'text/xml; charset=UTF-8', 'x-content-type-options': 'nosniff', 'content-length': '3828', 'vary': 'Authorization', 'connection': 'Close', 'x-frame-options': 'SAMEORIGIN', 'server': 'Splunkd', 'status': '200'}, 'serverContent': b'<?xml version="1.0" encoding="UTF-8"?>\n<!--This is to override browser formatting; see server.conf[httpServer] to disable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .-->\n<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?>\n<feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">\n <title>deploymentserver</title>\n <id>https://127.0.0.1:8089/services/deployment/server/config</id>\n <updated>2021-04-01T20:44:37+00:00</updated>\n <generator build="152fb4b2bb96" version="8.0.6"/>\n <author>\n <name>Splunk</name>\n </author>\n <link href="/services/deployment/server/config/_reload" rel="_reload"/>\n <link href="/services/deployment/server/config/attributesUnsupportedInUI" rel="attributesUnsupportedInUI"/>\n <link href="/services/deployment/server/config/authentication" rel="authentication"/>\n <link href="/services/deployment/server/config/listIsDisabled" rel="listIsDisabled"/>\n <opensearch:totalResults>1</opensearch:totalResults>\n <opensearch:itemsPerPage>30</opensearch:itemsPerPage>\n <opensearch:startIndex>0</opensearch:startIndex>\n <s:messages/>\n <entry>\n <title>config</title>\n <id>https://127.0.0.1:8089/services/deployment/server/config/config</id>\n <updated>1970-01-01T00:00:00+00:00</updated>\n <link href="/services/deployment/server/config/config" rel="alternate"/>\n <author>\n <name>system</name>\n </author>\n <link href="/services/deployment/server/config/config" rel="list"/>\n <link href="/services/deployment/server/config/config/_reload" rel="_reload"/>\n <link href="/services/deployment/server/config/config" rel="edit"/>\n <content type="text/xml">\n <s:dict>\n <s:key name="currentDownloads">0</s:key>\n <s:key name="disabled">0</s:key>\n <s:key name="eai:acl">\n <s:dict>\n <s:key name="app"></s:key>\n <s:key name="can_list">1</s:key>\n <s:key name="can_write">1</s:key>\n <s:key name="modifiable">0</s:key>\n <s:key name="owner">system</s:key>\n <s:key name="perms">\n <s:dict>\n <s:key name="read">\n <s:list>\n <s:item>admin</s:item>\n <s:item>splunk-system-role</s:item>\n </s:list>\n </s:key>\n <s:key name="write">\n <s:list>\n <s:item>admin</s:item>\n <s:item>splunk-system-role</s:item>\n </s:list>\n </s:key>\n </s:dict>\n </s:key>\n <s:key name="removable">0</s:key>\n <s:key name="sharing">system</s:key>\n </s:dict>\n </s:key>\n <s:key name="loadTime">1617309877</s:key>\n <s:key name="repositoryLocation">$SPLUNK_HOME/etc/deployment-apps</s:key>\n </s:dict>\n </content>\n </entry>\n</feed>\n', 'sessionKey': '', 'eaiArgsList': {}}
1 Solution
