Hi All, We are trying to upgrade the splunk universal forwarder from version 8.1.0 to 9.0.3 using ansible scripts. But we are getting error when the script tries to start the forwarder. Herewith attached error and ansible playbook. Ansible playbook: - name: Splunk Upgrade | Copy tgz to target copy: src: /pub/splunk/splunkpackages/{{ splunk_package }} dest: /tmp/{{ splunk_package }} - name: Splunk Upgrade | Check for SYSV scripts stat: path: /etc/rc.d/init.d/splunk register: splunk_sysv - name: Splunk Upgrade | Stop Splunk shell: | {{ splunk_home }}/bin/splunk stop tar -cvf /opt/splunk_config_backup.tar {{ splunk_home }}/etc/ - name: Splunk Upgrade | Clean up SYSV scripts shell: | rm /etc/rc.d/init.d/splunk /opt/splunkforwarder/bin/splunk disable boot-start when: splunk_sysv.stat.exists ignore_errors: yes - name: Splunk Upgrade | Upgrade Forwarder and restart ==> in this task it getting failed shell: | cd /opt tar -xzvf /tmp/{{ splunk_package }} chown -R splunk:splunk /opt/splunkforwarder {{ splunk_home }}/bin/splunk start --accept-license --answer-yes --no-prompt register: splunk_upgrade - name: Splunk Upgrade | Convert SYSV to Systemd shell: | {{ splunk_home }}/bin/splunk stop chown -R splunk:splunk /opt/splunkforwarder /opt/splunkforwarder/bin/splunk enable boot-start -user splunk when: splunk_sysv.stat.exists - name: Splunk Upgrade | start and enable splunk service: name: SplunkForwarder.service enabled: true state: started - name: Splunk Upgrade | Cleanup tgz file: state: absent path: /tmp/{{ splunk_package }}   Error in splunk forwarder log:   ... View more

Hi Team, Recently, I have configured splunk in my project to monitoring the application logs. I could find there is some log count mismatch between log file in server and event count in splunk logs. it is not happening in all time only some times like 2 or 3 times in a month then remaining days the event count is matching with log file count in server.  Could you please share suggestion to troubleshoot the issue. Splunk enterprise licensed version: 9.0.3 server kernel: Linux red hat Universal forwarder version: 9.0.3 server kernel: Linux red hat Example: Log file size is 500MB and total log count in log file is  1520713 and total event count in splunk after indexing is 1520794 which is higher than the server log file.  logs count in application log file = 1520713  event count in splunk search = 1520794  which is higher than actual log file.  I have verified the splunkd logs and there is no error. verified limits conf and props ocnf as well and there is no specific config related to it.  index conf: [monitor:///app/log/audit.log] index = xxxx disabled = false ignoreOlderThan = 7d recursive = false limits.conf: [thruput] maxKBps = 512   ... View more

Hi, I have requirement to open an additional non ssl http rest port in splunk and bind it to localhost for my splunk security issue. In splunk documentation server. conf file they have mentioned as below. How to open non ssl port and bound to localhost?  ############################################################################ # Open an additional non-SSL HTTP REST port, bound to the localhost # interface (and therefore not accessible from outside the machine) Local # REST clients like the CLI can use this to avoid SSL overhead when not # sending data across the network. ############################################################################ [httpServerListener:127.0.0.1:8090] ssl = false    ... View more

Hi everyone, i have a splunk universal forwarder installed in linux machine and configured some log files to forward to indexer. But i am getting below error and data not getting ingested in splunk. input type: File Error: 0100 ERROR Metrics - Metric with name thruput:thruput already registered 0100 ERROR Metrics - Metric with name thruput:idxSummary already registered   ... View more

Hi, There is an application which is used by multiple teams and we are ingesting the application logs for each team in a single index. Here we want to restrict each team people should be accessible only their teams logs not all the data in the index. How do i implement it in splunk? Thanks in advance. Gowtham ... View more

Hi, IThere is an application which is used by multiple teams and we are ingesting the application logs for each team in a single index. Here we want to restrict each team people should be accessible only their teams logs not all the data in the index. How do i implement it in splunk? Thanks in advance. Gowtham ... View more