Okay so we only add the line to the indexers indexes.conf stanza for those indexes that we are archiving. And not to foo_archive, just foo. coldToFrozenScript = "$SPLUNK_HOME/etc/apps/splunk_archiver/bin/coldToFrozen.sh" Somehow I missed this and thought it was to the search head. ... View more

Thanks Keith. I was referring to files in HDFS. Since there are so many files, using up many inodes, I was more aware of the files staying around. Thanks for this information. ... View more

Abhishek did you ever solve the performance issue? I have the same issue. ... View more

Hi. We have DMC running on our the Splunk Deployer for our Search Head Cluster. We are at Splunk 6.3.3.2 right now. I followed these instructions: http://docs.splunk.com/Documentation/Splunk/6.3.2/DMC/Deploymentsetupsteps and I successfully see our Splunk Deployment servers in the DMC Console. So what I believe I did was to add the deployment servers as search peers (along with our indexers and heads), turn on distributed mode in the DMC and then I see all these Splunk instances with their correct roles in the DMC. I do see on http://docs.splunk.com/Documentation/Splunk/6.3.2/DMC/Configureindistributedmode Note: If your deployment has nodes running Splunk Enterprise 6.1.x (instead of 6.2.0+), their instance (host) and machine values will not be populated. There are more instructions on that page on how to add the hosts. ... View more

Keith, it seems this is a bug with Splunk 6.4. The etc/system/default/indexes.conf has the Hive 0.12 binaries. vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.10.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.5.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-exec-0.12.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-metastore-0.12.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-serde-0.12.0.jar ... View more

And a third question.. we add this line to the _archive index stanza not the index stanza without the _archive. Correct? coldToFrozenScript = "$SPLUNK_HOME/etc/apps/splunk_archiver/bin/coldToFrozen.sh" ... View more

Keith: two questions from the documentation that you quote 1) "Note the following if you are using Hunk's coldToFrozenSh.script" That should probably be coldToFrozen.sh script 2) "All the search peers to the Hunk search head must have the script installed as well. You can do each peer manually or use the deployer for search head clusters. See Configure search head clustering." I see the binary on my indexers in /opt/splunk/etc/apps/splunk_archiver/bin so I don't need to do anything right? Doesn't that script come with Splunk 6.3 and greater? ... View more

I want infinite inodes! ... View more

We are running Splunk 6.3.1 on our deployment servers. I have a script that does reload /opt/splunk/bin/splunk reload deploy-server -auth myadminuser:password and that works for us. I just tested. If I leave the -auth off it prompts. If I include it I can see that server is reloaded with changes that I made. ... View more

Excellent info Keith! Can you get this documented? Thanks. ... View more

I guess I was confused by the DBConnect references to installation in a distributed environment (we use SHC.) http://docs.splunk.com/Documentation/DBX/2.1.3/DeployDBX/Distributeddeployment I do want to be able to periodically pull a few tables from a MySQL db to use with my SHC for querying. Yeah I don't want my Search Heads to to be heavy forwarders. So if I configure one dedicated heavy forwarder to pull the data from my DB, then no DB Connect needs to be installed on the search heads? ... View more

I am interested in examples. We have a number of alerts and where we alert on some threshold of events in the past 5/10/15 minutes. We noticed that we could transform our alerts into "isBatchMode" alerts to take advantage of batch mode search by adding "| stats count". But it is nice to see the actual events or else information about the events rather than just a count number. I welcome ideas on ways to convert searches into batch mode search alerts. ... View more

Dan: maybe a sample of the input would help us understand? ... View more

http://docs.splunk.com/Documentation/Hunk/6.3.3/Hunk/distributableandnondistributablesearchcommands which has a link that to the three modes http://docs.splunk.com/Documentation/Splunk/6.3.3/Search/Changethesearchmode but that information is more for general Splunk http://docs.splunk.com/Documentation/Hunk/6.3.3/Hunk/distributableandnondistributablesearchcommands Smart mode is the default and recommended setting for VIX searches. It maintains search behavior based on whether your search contains transforming commands. When searching virtual indexes we recommend that you search in smart mode, as it is more efficient. If you use verbose mode to search a VIX, note that Hunk does not start a MapReduce job for that search. This is because verbose mode searches search for all events as well as any reports that you might be running. The benefits of MapReduce jobs in that case are minimal and in some cases can have a negative impact on your searches. ... View more

This manual entry is a good start but I don't understand what truly makes a search qualify as a batchmode search. Could we have examples that show searches that ARE batchmode searches and ones that are not? ... View more

Our reason to upgrade to 6.3.2 is that we have a lot of alerts created by user Admin that were emailed out to Admin and Power users. Our Power users stopped being able to click on the alert results. 6.3.2 fixed this bug. "Power user having read and write permissions for a saved search owned by an admin user is unable to view results from scheduled email" ... View more

Update: I tried on Splunk 6.3.2 and the problem is fixed. ... View more

I can totally reproduce this bug. 1) using Splunk 6.3.1 have an Admin user create a savedsearch/alert. 2) Change the permissions on the saved search so that all can read and you can put the app in "All Apps". 3) Modify search to send mail to a Power user and an Admin user 4) The Admin user can see the content. The Power user will get "The view you requested could not be found." I opened a case yesterday with Splunk about this. I believe also that it used to work and is likely a 6.2->6.3 bug. ... View more

Thanks. Is there a good way to test what the setting is? i.e. to make sure the EXPORT is working? ... View more

Thanks Keith. So to increase heap size to 2048 would that be export HADOOP_CLIENT_OPTS="-Xmx2048m" ... View more

I am in a situation where I cannot easily comment out the line in hadoop-env.sh (it gets overwritten when I update any packages.) Is it possible to set these Hadoop variables with an export of HADOOP_CLIENT_OPTS instead of setting them in indexes.conf? Does that work? ... View more