Hey community, I have the situation that I need to set a bindDN password with special characters for the LDAP authentication without using the WebUI. I'm aware on the fact, that I can set the password in cleartext via authentication.conf - unfortunately it seems the special charaters within the password confuse Splunk. This results in the fact that each hash for the several LDAP strategies look different - even if its the same password. Also the authentication does not work at all. Unfortunately I cannot use the WebUI to set the password. I currently could image two solutions: 1. Is there a way to escape the special characters within the bindDN password within authentication.conf? 2. Is there a war so set the password via a CLI command, I wasn't able to find so far? Any ideas? Thank you   Regards, Christian ... View more

Hi all, is there a chance to track usage of "special licenses" (like the one for PCI App or ITSI) in a distributed landscaped, with a central License-Master which hosts those licenses? I'm thinking about a event beeing generated on the License-Master which indicates a systems has checked whether there is a valid license for Splunk Apps which require additional licenses. Greets Christian ... View more

Hi all, I got a running search head cluster on latest Splunk Version 6.4.1. I now need to add an additional member to the Cluster, and processed as described here: http://docs.splunk.com/Documentation/Splunk/6.4.1/DistSearch/Addaclustermember I'm not struggling with the process Add_the_instance. When using the command on the new member, I receive a login prompt asking over and over again for my username and password. I checked the password and made sure it is correct. I double proofed it by entering a wrong password which directly exits the "authentication loop". splunk4.my.net:/opt/splunk # /opt/splunk/bin/splunk add shcluster-member -current_member_uri https://splunk3.my.net:8089 Your session is invalid. Please login. Splunk username: admin Password: Your session is invalid. Please login. Splunk username: admin Password: Your session is invalid. Please login. Splunk username: admin Password: Your session is invalid. Please login. Splunk username: admin Password: [and so on and on ....] Thankfully there is a second way, which allows adding a new member to the cluster. So I ran the following command on one of the members which are already within the cluster, which also ends with an error: splunk3.my.net::/opt/splunk # /opt/splunk/bin/splunk add shcluster-member -new_member_uri https://splunk4.my.net:8089 In handler 'shclustermemberconsensus': Failed to Set Configuration. One potential reason is captain could not hear back from all the nodes in a timeout period. Ensure all to be added nodes are up, and increase the raft timeout. If all nodes are up and running, look at splunkd.log for appendEntries errors due to mgmt_uri mismatch I saw the above error mentioned in other answers which point to a password mismatch. So I added the password in cleartext to the server.conf and also gave the already hashed pass4SymKey from another instance a chance. Situation doesn't change. I also checked the network connectivity by telnet to port 8089 from the new system to a cluster member and vice versa - both directions are working. How can I add the new node to the search head cluster? Any ideas or recommendations? Btw: I already restarted setting up the new node by removing and installing Splunk again for several times - every time the same result. Greets Christian ... View more

Hi all, I'm often facing the question in which way Splunk Instances with different Roles can be combined - especially in huge and complex landscapes like with Index- and Searchhead-Clusters. Means for example can a deployer be the same Splunk-Instance as the Cluster-Master? Can the Deployment-Server also be the License Server? and so on ... I tried to put this in a simple scheme, allowing to give a quick answer. My approach can be seen within the attached picture: Sources for these results are some simple tests, experience and the following sources: https://answers.splunk.com/answers/150606/should-we-have-splunk-deployment-server-and-cluster-master-on-the-same-instance-recommendations-please.html https://answers.splunk.com/answers/96197/any-know-issues-with-deployment-server-and-master-on-same-machine.html https://answers.splunk.com/answers/302606/what-is-the-best-way-to-combine-a-license-master-d.html http://docs.splunk.com/Documentation/Splunk/6.2.8/Admin/ConfiguretheMonitoringConsole The yellow field is based on the fact the sources 1. and 2. conflict with source 3. How are your experiences regarding these combinations? Could you false or proof my experience? To be honest when started developing the above Matrix I realized it wasn't that trivial as I initially thought, but maybe we can build a reliable version of this together. I assume this could be helpful for some of the "Splunkers in the wild" 🙂 Regards, Christian ... View more

Hi all, are there any experiences out there regarding performance-comparison of macros, eventtypes and data-models? We're currently planing a rebuild of a huge Splunk app which should get more flexible to be run within other landscapes. Therefore we'd like to build some kind of sublayer between the individual Splunk landscapes and the upper Splunk App-Part. In Enterprise Security App this has been done using eventtypes. I'm currently unsure if eventtypes are the best solution. From my understanding eventtypes are some kind of "subsearches" - based on the behaviour of Splunk, this means a second search-thread is being initialized for each search eventtype-definition. Is this assumption correct? How do macros behave in this situation? Only replacing parts of the search string, and no additional search-thread? At least how do data-models fit in these thoughts? How is the experience in performance and maintaining a huge set of macros, eventtypes or data-models? Would love to hear some real-life experience. Thank you all Regards, Christian ... View more

Hi there, I'd like to modify the default search form of Splunk (flashtimeline view) for a new app. Therefore I'd like to add some search-strings to the search-field. These search-strings shouldn't be seen by the user. I think this would be a task for HiddenIntention module but read this module is deprecated, so I don't want to use it for a new app. How can the task be done without the HiddenIntention module? Regards, Christian ... View more

Hi community, let's say we have a online shop which is selling products which could appear in different variations - colors for example. So we have the fields productID and variation. Now I'd like to have a table showing the top10 sold productID, with all variations in the same row and their count. productID count variation ----------------------------------------------------- shirt1 532 blue (232) red (300) ----------------------------------------------------- jeans1 47 blue (35) black (12) ----------------------------------------------------- .... ... and so on. The number behind the variation value is the count for this variation. The complete sorting of the list should be based on the count field so I get the top10 independent from the variation. I already tried to add the variation field to the top command which results in a different sorting-order when comparing with the top command for productID. Is there a solution for the problem I'm facing? Regards, Christian ... View more

Hi there, I'd like to build individual Dashboards per Splunk-User (LDAP mapped). As there is a huge number of employes I'd like to build an dynamic dashboard which allows a user to see reports regarding his username. Is there a way to extract the current username out of the user-session, so he can see only the data regarding his username? I'd like to put this user-name variable into a search which filters then for the current username. Is this possible? Thank you. Christian ... View more

Hi Splunkers, one great way to bring additional information into your data is the use of lookups. One problem I found regarding this was the data for the lookups was changing dynamically. I had the requirement to build lookups for dynamic data several times and decided to build some type of automation. The result was a script which I'd like to introduce here in case some else could find this useful. It could be used as some kind of template. The functions itself have to be written individual. I commented the script itself so there shouldn't be a need to explain it. #!/bin/bash ######################################################################## # # This script automatically fills lookupfiles for your Splunk app. # Call the script with the filename of the lookupfile you'd like to fill. # Lookupfiles must be located in the lookups-folder of your app. # Exampl0e: fill_lookups.sh hostnames.csv # # author at splunkbase: christian_l # ######################################################################## # location of the splunk app folder APP_PATH=/opt/splunk/etc/apps/Your-AppName LOOKUP_FILE=$APP_PATH/lookups/$1 # count the lines before the update LINES_BEFORE=`wc -l $LOOKUP_FILE | cut -d' ' -f1` # create a backup of the lookupfile cp $LOOKUP_FILE $LOOKUP_FILE.bak # the following functions are doing the real work. They have to be developed for your individual needs. # I'll give two examples, one for a SQL-DB lookup to fill the file, and a second which fills the file # from content and filename of multiple files. # SQL-query example function hostname_lookup() { # SQL-host variables HOST=192.168.1.1 USER=splunk PASSWORD=pwd4splunk # write the header line which describes the column content echo "hostname,hostip,location,device" > $LOOKUP_FILE # build the query and send it to SQL-Server mysql -u $USER -h $HOST -p$PASSWORD --execute "SELECT a.host, b.hostip, b.location, c.device AS delete_row FROM host-db a, ip-list b, host-models c WHERE a.host = b.hostname AND a.host NOT LIKE 'staging';" | grep -v "delete_row" >> $LOOKUP_FILE # manual corrections by sed # delete the splunkhost from the file sed -i '/splunkhost/d' $LOOKUP_FILE # replace a device-name sed -i 's/WRT54G$/OpenWRT-device/g' $LOOKUP_FILE # replace the tabs from the SQL-query with spaces and optimize comma-usage sed -i 's/\t/,/g' $LOOKUP_FILE sed -i 's/,\s/,/g' $LOOKUP_FILE } # gather the information from several .txt files in the sources folder # the information is located in the filenames itself and in the content of the txt-files function hostname2_lookup { # write the header line which describes the column content echo "hostname,hostip,location,device" > $LOOKUP_FILE for FILENAME_COMPLETE in $APP_PATH/lookups/sources/*.txt do FILENAME=`basename $FILENAME_COMPLETE` LOCATION=`echo $FILENAME | cut -d'-' -f3-9 | cut -d'.' -f1` while read LINE; do HOSTNAME=`echo $LINE | cut -d ' ' -f1` HOSTIP=`echo $LINE | cut -d' ' -f2` DEVICE=`echo $LINE | cut -d' ' -f3-4` echo $HOSTNAME,$HOSTIP,$LOCATION,$DEVICE >> $LOOKUP_FILE done < $FILENAME_COMPLETE done # manual corrections sed -i 's/Long\,Island\,/Long-Island\,Long-Island\,/g' $LOOKUP_FILE } case "$1" in hostnames.csv) hostname_lookup ;; hostnames2.csv) hostname2_lookup ;; esac # convert lookup file splunk-readable iconv -f iso-8859-1 -t utf-8 $LOOKUP_FILE > $LOOKUP_FILE.tmp # replace old lookup-file with the new, currently generated version mv $LOOKUP_FILE.tmp $LOOKUP_FILE # count lines after update LINES_AFTER=`wc -l $LOOKUP_FILE | cut -d' ' -f1` # calculate difference DIFFERENCE=`echo $(($LINES_AFTER - $LINES_BEFORE))` # write information about changes into the splunk index if [ $DIFFERENCE -ge 0 ] ; then echo "Lookup-File $1 was updated. $DIFFERENCE lines were added." else DIFFERENCE=`echo $(($DIFFERENCE * -1))` echo "Lookup-File $1 was updated. $DIFFERENCE lines were removed." fi exit 0 Next step is to run the script automatically. It would be possible to schedule it per crontab. But I'd prefer a way which doesn't need a part outside Splunk so I can deploy the app without doing additional changes on the operating-system-level. I decided to run the script via a scripted input. One additional feature this method brings is to write the update-information (at the bottom of the script) into a index. This information can then be used for a Splunk alert - for example if more than 20 lines were removed from the lookupfile. The content of the inputs.conf for this could look like the following: # fill_lookups.sh isn't a real input script - just scheduled to renew the lookup data [script://./bin/fill_lookups.sh hostnames.csv] disabled = 0 interval = 1 20 * * * host = splunk source = lookup-filler sourcetype = lookup index = MyAppIndex That's all! Feel free to give feedback and improvement-suggestions. Thank you. ... View more