About jkerai

jkerai · ‎10-22-2013

You can edit props.conf in $SPLUNK_HOME/etc/master-apps/_cluster/local/props.conf on master and push cluster-bundle with command 'splunk apply cluster-bundle'. Peers will restart and props.conf, in $SPLUNK_HOME/etc/slave-apps/_cluster/local/props.conf, will be layered when splunkd start.

jkerai · ‎10-04-2013

Are you still experiencing the issue? Can you restart master and give it a try again. Seems like master is confused for whatever reason.

jkerai · ‎10-04-2013

Something like '* | convert ctime(_indextime) as it | table host it'.

jkerai · ‎10-04-2013

Clustering is not expected to use 20GB of RAM. In our tests we have not seen it go over 3GB. So, I would think that there is a associated memory leak. Which version of splunk is it? Please file a support case and we will help find the leak.

jkerai · ‎10-04-2013

Unfortunately yes. You cannot use DS in this instance on search head as saved changes would get overwritten by DS. DS needs an enhancement to exclude set of directories/files when updating app.

jkerai · ‎10-04-2013

Mag2sub, you may want to look at 6.0 as some of these limitations have been addressed.

jkerai · ‎10-04-2013

Couple of things to try: 1) [clustering] stanza for search head should be 'mode=searchhead'. 2) Check indexer's $SPLUNK_HOME/var/log/splunk/splunkd.log to see if there are any reported errors. Seems like it has not yet connected to master probably due to some error. 3) Access master's REST endpoint to confirm that you see indexers at https://splunk-server:8089/services/cluster/master/peers. This should list all the indexers seen by master. Seems like master does not see the indexer and thus search head also does not see them.

jkerai · ‎10-03-2013

Mix of 5.0.x and 6.0 are not supported. 6.0 search head makes additional REST requests from master, e.g. number of replicated copies, searchable copies of an index etc that are not supported on 5.0.x, thus not quite compatible. Would recommend you to follow ChrisG's suggestion about reading Upgrade a cluster topic.

jkerai · ‎10-03-2013

Can you check if you are able to connect to the ip port? You can do 'telnet 192.168.1.1 64514' or 'echo "test" | nc 192.168.1.1 64154'.

jkerai · ‎10-03-2013

The only issue was with HWF CLI validating that you have input configured to valid index that is present on HWF instance. Under the cover, CLI validates index, creates inputs.conf stanza with desired index setting and reloads inputs.conf. So, the workarounds are, to repeat 🙂 1) Either define the index to make validation code happy. This allows CLI to continue with creating inputs stanza with desried index setting. 2) or just edit inputs.conf directly specifying the index(but not defining them in HWF, skipping index validation step) and ask splunk explicitly to reload inputs.conf with REST endpoint at https://splunk-server:8089/services/data/inputs/monitor/_reload. UF does not perform index validation, as it does not data locally. So, just adding monitor with desired index would work fine.

jkerai · ‎10-03-2013

Unless there is a very specific reason to use regular/heavy weight forwarder(HWF), I would recommend using Universal forwarder(UF). If you use UF, you will not hit this bug. If you edit inputs.conf manually(you will need to for HWF for this case), you will need to either restart forwarder, or hit REST endpoint https://splunk-server:8089/services/data/inputs/monitor/_reload to ask splunkd to reload. Adding inputs via monitor is definitely simpler.

jkerai · ‎10-03-2013

If you are using Universal forwarder, then CLI would not throw the error. Seems like you are using regular/heavy weight forwarder, so CLI incorrectly validates and errors out. I would recommend just editing inputs.conf and specifying index. This way you can avoid adding indexes on forwarder. This is not yet addressed in 6.0. Have not seen any issue specific to authentication on search head. May want to get in touch with support and upload diag. What is the specific error message reported by search head?

jkerai · ‎10-03-2013

I meant you to add custom indexes on forwarder that were pushed out from master to cluster peers. As far as data not found on cluster search head is concerned, I think those were already forwarded to main index in your previous attempt. Could you check if they are in main index?

jkerai · ‎10-03-2013

Seems like you are using Heavy weight forwarder(HWF). Please define the index on the forwarder instance as well. The index on HWF would not take any space, but will merely allow validation to pass. Or, You can edit inputs.conf to add 'index= ' to specific input stanza. For Universal forwarder, we skip validation of index, so CLI with index param does not throw any error.

jkerai · ‎10-03-2013

Seems like CLI/REST endpoint is running into validation error. You can try one of the following: You may want to try adding this index (splunk add index -name 'idx1') on the forwarder as well to avoid this error. Edit inputs.conf and for monitor stanza add 'index=idx1' param and restart. Forwarding should happen to correct index. I will file a ticket for CLI/REST issue.

jkerai · ‎07-30-2013

Sorry, there is no workaround at present. We have a fix slated for 5.0.5 which will log messages to splunkd.log. It will log message for only first 10 forwarder instances not using ACK.

jkerai · ‎08-12-2011

Steve, are you seeing the remaining data (5000-2048) show up at all? We do try to read 2048 bytes at a time and queue it up for processing. One of the solutions could be to make UDP input configurable on splunk such that it tries to read configured number of bytes, 5000 in your case.

jkerai · ‎04-28-2011

It should be run on the indexer that will listen and receive data from from universal forwarder on port 9997. I am assuming that your indexer will serve the purpose of Deployment Server as well. If so, you can run the command on DeploymentServer/indexer.

jkerai · ‎04-27-2011

Could you provide the output of following command 'splunk version'. Can you check if you are running the command on indexer or Universal forwarder?

jkerai · ‎04-17-2011

Unfortunately this is a bug and we are trying to identify root cause.

jkerai · ‎04-17-2011

Is deployment client 32-bit or 64-bit? We verified with 64-bit and it seemed to untar downloaded app without any errors. Could you kindly open a case with support and provide diag from client and server?

jkerai · ‎04-13-2011

I would suggest using a sourcetype in place of source. Since the source will be set to each file under the directory being monitored, the transforms may not match. You can try the following: On fwd: [monitor:///home/tomcat1/apache-tomcat-6.0.24/logs] disabled=false host=prod_228_1 index=production sourcetype=apache_logs On indexer: [sourcetype::apache_logs] TRANSFORMS-null=setnull [setnull] REGEX = healthCheck DEST_KEY = queue FORMAT = nullQueue

jkerai · ‎04-13-2011

Could you provide forwarder's outputs.conf and indexer's inputs.conf. Seems like on indexer, the receiving port is misconfigured. Please see that it is configured as [splunktcp://9997]

jkerai · ‎04-13-2011

This can be safely ingored for Splunk Light Forwarder or Universal forwarder. Currently we perform check to see if the configured memory is less than 1GB and produce the above ERROR message(We will open a a BUG to take care of this). LWF should not have problem running with above ulimit settings.

jkerai · ‎01-15-2011

Yes, this configuration should work. You should be able to clone the data to 2 clusters of indexers for HA.

Posts	30
Solutions	5
Karma Given	17
Karma Received	30
Member Since	‎08-16-2010

Online Status	Offline
Date Last Visited	‎06-05-2020 02:02 AM

Re: Where do I edit props.conf in a cluster?

Re: cannot apply cluster-bundle

Re: How to display index time in table?

Re: Splunkd process on the indexer in clustering u...

Re: Deployment Server - when app is redeployed, wh...

Re: deploying Dashboard apps, search apps(search p...

Re: Received an empty peer list from the master. W...

Re: Planning a Splunk 5 Cluster upgrade to Splunk ...

Re: Linux Syslogd Config

Re: Forwarding events to custom index on Cluster p...

Re: Forwarding events to custom index on Cluster p...

Re: Forwarding events to custom index on Cluster p...

Re: Forwarding events to custom index on Cluster p...

Re: Forwarding events to custom index on Cluster p...

Re: Forwarding events to custom index on Cluster p...

Re: How to turn off indexer-acknowledgement warnin...

Re: UDP Events truncated at 2048 bytes

Re: deployment-server question

Re: deployment-server question

Re: Large number of errors processing data from fo...

Re: Deployment Clients not picking up config

Re: Filtering events on indexer received from Ligh...

Re: seeing meta tags with log entries

Re: Why is Splunk writing errors to splunkd.log wh...

Re: High Availability - Light Forwarders and combi...