Hello Everyone, Recently I have installed Splunk db connect app (3.16.0) in my Splunk heavy forwarder (9.1.1). As per the documentation I have installed jre and installed the my sql add on. And created Identities, connections and inputs. But when I check for the data it is not getting ingested. So I enabled the debug mode and checked the logs and got the hec token error. But the hec token is configured in inputs.conf file and same I could see in the Splunk web gui under Data inputs -> HTTP event collector. could you please help if anyone faced this error before? Error: ERROR org.easybatch.core.job.BatchJob - Unable to write records java.io.IOException: There are no Http Event Collectors available at this time. ERROR c.s.d.s.dbinput.recordwriter.CheckpointUpdater - action=skip_checkpoint_update_batch_writing_failed java.io.IOException: There are no Http Event Collectors available at this time.   ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch java.io.IOException: There are no Http Event Collectors available at this time. ... View more

Hello Everyone, Recently, I am trying to ingest the logs from my server. But it is not getting indexed. The log file which I am trying to ingest has different timestamp with same events. Events in log file: 1712744099:{"jsonefd":"1.0","result":"1357","id":1} 1712744400:{"jsonefd":"1.0","result":"1357","id":1} 1712745680:{"jsonefd":"1.0","result":"1357","id":1} 1714518017:{"jsonefd":"1.0","result":"1378","id":1} 1715299221:{"jsonefd":"1.0","result":"1366","id":1} I tried with crcsalt but still no luck. Kindly help if anyone faced this issue before.  I would like to ingest the events even the events are same with different timestamps. ... View more

Hello All, Logs are not indexing into splunk. My configurations are below  inputs.conf: [monitor:///usr/logs/Client*.log*] index = admin crcSalt = <SOURCE> disabled = false recursive = false props.conf: [source::(...(usr/logs/Client*.log*))] sourcetype = auth_log My logs files pattern: Client_11.186.145.54:1_q1234567.log Client_11.186.145.54:1_q1234567.log.~~ Client_12.187.146.53:2_s1234567.log Client_12.187.146.53:2_s1234567.log.~~ Client_1.1.1.1:2_p1244567.log Client_1.1.1.1:2_p1244567.log.~~ In some of log files it starts with below line: ===== JLSLog: Maximum log file size is 5000000 and then log events So for this one i tried with below config one by one but nothing worked out adding crcSalt=<SOURCE> in monitor stanze, tried with adding SEDCMD in props.conf SEDCMD-removeheadersfooters=s/\=\=\=\=\=\sJLSLog:\s((Maximum\slog\sfile\ssize\sis\s\d+)|Initial\slog\slevel\sis\sLow)//g and tried with regex in transforms.conf transforms.conf [ignore_lines_starting_with_equals] REGEX = ^===(.*) DEST_KEY = queue FORMAT = nullQueue props.conf: [auth_log] SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)=== TRANSFORMS-null = ignore_lines_starting_with_equals When i checked in splunkd logs there is no error captured and in list inputstatus it is showing                  percent = 100.00                 type = finished reading / open file please help me out of this issue if anyone faced before and fixed it. but the weird scenario is sometimes only  the first line of log file is indexed  ===== JLSLog: Maximum log file size is 5000000 host/server details: os: Solaris 10 splunk universal forwarder version 7.3.9 splunk enterprise version: 9.1.1 Here restriction is the host os cant be upgraded as of now so i need to strict on 7.3.9 splunk forwarder version. ... View more

Hi All, We are trying to upgrade the splunk universal forwarder from version 8.1.0 to 9.0.3 using ansible scripts. But we are getting error when the script tries to start the forwarder. Herewith attached error and ansible playbook. Ansible playbook: - name: Splunk Upgrade | Copy tgz to target copy: src: /pub/splunk/splunkpackages/{{ splunk_package }} dest: /tmp/{{ splunk_package }} - name: Splunk Upgrade | Check for SYSV scripts stat: path: /etc/rc.d/init.d/splunk register: splunk_sysv - name: Splunk Upgrade | Stop Splunk shell: | {{ splunk_home }}/bin/splunk stop tar -cvf /opt/splunk_config_backup.tar {{ splunk_home }}/etc/ - name: Splunk Upgrade | Clean up SYSV scripts shell: | rm /etc/rc.d/init.d/splunk /opt/splunkforwarder/bin/splunk disable boot-start when: splunk_sysv.stat.exists ignore_errors: yes - name: Splunk Upgrade | Upgrade Forwarder and restart ==> in this task it getting failed shell: | cd /opt tar -xzvf /tmp/{{ splunk_package }} chown -R splunk:splunk /opt/splunkforwarder {{ splunk_home }}/bin/splunk start --accept-license --answer-yes --no-prompt register: splunk_upgrade - name: Splunk Upgrade | Convert SYSV to Systemd shell: | {{ splunk_home }}/bin/splunk stop chown -R splunk:splunk /opt/splunkforwarder /opt/splunkforwarder/bin/splunk enable boot-start -user splunk when: splunk_sysv.stat.exists - name: Splunk Upgrade | start and enable splunk service: name: SplunkForwarder.service enabled: true state: started - name: Splunk Upgrade | Cleanup tgz file: state: absent path: /tmp/{{ splunk_package }}   Error in splunk forwarder log:   ... View more

Hi Team, Recently, I have configured splunk in my project to monitoring the application logs. I could find there is some log count mismatch between log file in server and event count in splunk logs. it is not happening in all time only some times like 2 or 3 times in a month then remaining days the event count is matching with log file count in server.  Could you please share suggestion to troubleshoot the issue. Splunk enterprise licensed version: 9.0.3 server kernel: Linux red hat Universal forwarder version: 9.0.3 server kernel: Linux red hat Example: Log file size is 500MB and total log count in log file is  1520713 and total event count in splunk after indexing is 1520794 which is higher than the server log file.  logs count in application log file = 1520713  event count in splunk search = 1520794  which is higher than actual log file.  I have verified the splunkd logs and there is no error. verified limits conf and props ocnf as well and there is no specific config related to it.  index conf: [monitor:///app/log/audit.log] index = xxxx disabled = false ignoreOlderThan = 7d recursive = false limits.conf: [thruput] maxKBps = 512   ... View more

Hi, I have requirement to open an additional non ssl http rest port in splunk and bind it to localhost for my splunk security issue. In splunk documentation server. conf file they have mentioned as below. How to open non ssl port and bound to localhost?  ############################################################################ # Open an additional non-SSL HTTP REST port, bound to the localhost # interface (and therefore not accessible from outside the machine) Local # REST clients like the CLI can use this to avoid SSL overhead when not # sending data across the network. ############################################################################ [httpServerListener:127.0.0.1:8090] ssl = false    ... View more

Hi everyone, i have a splunk universal forwarder installed in linux machine and configured some log files to forward to indexer. But i am getting below error and data not getting ingested in splunk. input type: File Error: 0100 ERROR Metrics - Metric with name thruput:thruput already registered 0100 ERROR Metrics - Metric with name thruput:idxSummary already registered   ... View more

Hi, There is an application which is used by multiple teams and we are ingesting the application logs for each team in a single index. Here we want to restrict each team people should be accessible only their teams logs not all the data in the index. How do i implement it in splunk? Thanks in advance. Gowtham ... View more

Hi, IThere is an application which is used by multiple teams and we are ingesting the application logs for each team in a single index. Here we want to restrict each team people should be accessible only their teams logs not all the data in the index. How do i implement it in splunk? Thanks in advance. Gowtham ... View more