I have a sample JSON object containing multiple values for same metric_name which is CPU_usage. How to convert it in to multiple metric points whose metric name is same i.e CPU_usage. samplejson: [ [-] { [-] epochtime: 1573532862 value: 5.29 } { [-] epochtime: 1573532562 value: 5.34 } ... View more

I need to create a scripted input in inputs.conf that runs scripts by passing arguments at an interval of 60 secs. Basically script contains a curl command to ping websites. I need to create around 600 scripted inputs to run the same script but passing different arguments to the script as defined in each input stanza. Does Splunk execute all the scripts at a time? Or does it have any handling mechanism to execute only a few inputs in parallel at a time and and pick other inputs once completed and so on? ... View more

Splunk is indexing events in wrong format. On Splunk forwarder, I am seeing these errors: WARN UTF8Processor - Using charset UTF-8, as the monitor is believed over the raw text which may be UTF-16LE - data_source="C:\Program Files\SplunkUniversalForwarder\var\log\XXX.log", data_host="xxx", data_sourcetype="config" A few events are indexed in the below format: \xFF\xFEC\x00:\x00\\x00P\x00r\x00o The input file data is in proper format which is output of Splunk btool cmd copied to file and ingested to Splunk. May I know how can we handle this? ... View more

Does Splunk Universal Forwarder forward audit event logs to Splunk _audit index? I can see Splunk HF's are forwarding audit events, but couldn't find which app has inputs.conf which enable reading audit logs and forward to _audit index. May I know which app consists inputs to read and send data to _audit index in Splunk? ... View more

I have a Powershell script on windows UF servers. We have created a powershell input and pointed to the script. The output is forwarded to Indexers. I created props.conf on the indexers to merge the events of the script to single event based on regex. But the properties are not being applied to the data. The parsing properties are working when I manually ingested same data from file and line breaking takes place as expected. Then I tested this powershell input on HF placing props.conf on HF. It is still not working. Is there any reason for this? Does powershell script output formatted in different way? ... View more

In Inputs.conf, it says that we can run powershell scripts using the below stanza. Does the universal forwarder have the capability to run this input alone? [powershell://] Is a UF required to have a Powershell to be installed on Windows host separately to run this input? OR Does Powershell come as a package with UF Installation? ... View more

We have Windows servers blocked for executing batch scripts. So, how do I run the below Splunk CLI command scheduled everyday on universal forwarders and index that data? Splunk btool inputs list --debug I thought to create a scripted input and use the .path file to run the command, but it is not working. inputs.conf [script://.\bin\xyz.path] And create a path file xyz.path under bin directory of the app. xyz.path contails below line $SPLUNK_HOME\bin\splunk btool inputs list --debug Is there any other way to do it on UF's ... View more

I would like to run a scheduled Splunk btool command using scripted input to index configs every few hours. I cannot put this command in .sh or any script file and give it as input to scripted input in Splunk due to limitation of running scripts on our Windows universal forwarders. So, I have put path file under bin dir of app, and pointed the .path file in scripted input like [Script//./bin/file.path] And The path file contains the below command: /opt/splunk/bin/splunk cmd btool inputs list —debug But it is not running the Splunk btool cmd when pointed from the .path file. It's not indexing data. The path file can only point and run external scripts in .sh or .exe formats. Is there any possibility to run the btool command on UFs without using .exe scripts on Windows in scheduled based by Splunk inputs? My requirement is to index config data every day ... View more

We have around 1K+ universal forwarder servers where we have deployed apps manually without using DS. Is there any way to track the configuration changes (inputs.conf or outputs.conf) by any un-authorized user? One way is to use btool and get all current configurations copied to filesystem in a scheduled manner and ingest configurations to Splunk and compare them to track changes. But this approach has limitations due to license and storage for these extra logs. May I know whether there is any way to implement configuration tracking? ... View more

I send out a email report everyday in csv format. One column in csv is the _raw field. The new lines in the raw data are being replaced by /n character in the Report. Is there any way to overcome this. I could able to download csv file of results without any issues. I can see _raw field value consisting of multiple lines. Only when csv report is emailed the newlines are replaced by /n field. ... View more

I need to assign number each event sorted in decending _time order. Ex Event. _time Count Event1. 11:54:51. 1 Event2. 11:53:57 2 Event3. 11:53:52. 3 I can use |streamstats count. But does this guarantee events in descending order for historical searches on clustered indexers? Using sorting on _time is effecting query performance. So Is there any way to assign a increment number count based on descending order of _time. ... View more

I am consuming messages using the JMS modular input. For one connection, I need to refresh the queue connection by disabling it and enabling it again to start re-consuming. I need to do it every 5 min. It is giving the following error message from: "python /opt/splunk/etc/apps/jms_ta/bin/jms.py" Exception in thread "Thread-40" java.lang.OutOfMemoryError: Java heap space Do you know how to handle this? @Damien Dallimore ... View more

When search affinity is disabled, search head can search across all indexers across multiple sites. But primaries on respective indexer information is provided to search head by indexer itself. If this is the case how search head searches primary copy only once eventhough it is present on two sites. How search head manages to search specific primary copy only one time if it is present on other site as well. Is there any mechanism like S.H gets primary copy details from both sites indexers at first, then it will make decision which primary to search and accordingly intiate search on indexers? ... View more

If a new indexer is added to the cluster. Do we need to manually push the cluster-bundle from master to indexers OR Cluster-bundle will be automatically pushed to new indexer once it communicates with the master? ... View more

Hi, I have a server.conf file under system/local directory which has following stanza [general] pass4SymmKey = $1$xxxxxxxxxx I expect that this password is the encrypted form of pass4SymmKey in server.conf at system/default. But even though I changed pass4SymmKey in server.conf at system/default , the pass4SymmKey in server.conf at system/local directory is not getting updated. Then I removed the pass4SymmKey in server.conf at system/local directory. After restart , the same password is getting generated. Do you know from which source pass4SymmKey in server.conf at system/local might get generated other than pass4SymmKey in server.conf at system/default ? ... View more