How are you updating the geoip files currently? ... View more

That's a lot of errors... Start with the Cloud Appinspect Criteria. Hopefully something obvious jumps out. If that doesn't help then it's going to be very hard to help debug this without some more details about what your app contains. Feel free to share the (obfuscated) config within the app and the high-level structure too. If you (understandably) don't want to share this on here then I would raise a ticket for Splunk support. ... View more

Your deployment and subsequent troubleshooting steps all look absolutely spot-on so far! You're right in thinking that you don't need the local inputs.conf on your Splunk Enterprise instance. I suspect the issue here could be down to a slight misconfiguration or typo. Is the index definitely enabled? Could the UF be forwarding to a different Splunk Instance? i.e. Have you got any other outputs.conf servers on your UF? Have you got any routing/index renaming props/transforms applied on Splunk Enterprise? Could this be a typo, on your Splunk Enterprise messages can you see this warning? "Received event for unconfigured/disabled/deleted index" If so make sure your index name matches your inputs.conf exactly. ... View more

This error has came up before a few times on the Splunk community. How to get WMI data collection by providing to spl... - Splunk Community Solved: Getting Error When trying to read windows event l... - Splunk Community The TL;DR is that the service account running the splunkd service on the forwarder host needs to have specific permissions set (on both the local and domain level). This Splunk Documentation does a good job of taking you through the required permissions and also the appropriate security considerations. ... View more

Hi @sudderthc, First and foremost, this add-on is supported by Tenable so I would raise a case with them if you can't get this going. Looking at tenable_account_validation.py that error is unfortunately just as vague as it looks. Funnily enough, the ConnectionError exception that's raised to trigger that exception is decently verbose but for some reason the TA Devs have decided to mask it with that generic error you're seeing. If you're comfortable with it, you could temporarily modify the TA to include the verbose exception details. I've also included some more generic troubleshooting steps to help if you'd rather not touch the code. Making the TA Validation More Verbose: Modify tenable_account_validation.py lines 221-223 From:   except Exception: msg = "Please enter valid Address, Username and Password or configure valid proxy settings or verify SSL certificate." raise Exception(msg)   To:   except Exception as e: msg = "Please enter valid Address, Username and Password or configure valid proxy settings or verify SSL certificate. Verbose Details: " + repr(e) raise Exception(msg)   Then try to add your account again. It should now include the verbose details about what went wrong. High-Level Troubleshooting: If I were troubleshooting this on the Splunk host, I would step-through the following: 1. Establish basic connectivity:   curl -k https://tenablesc_host:tenablesc_port/rest/system   If you're behind a proxy you can use the --proxy curl switch. Do you get a response? If not then you'll need to facilitate this connectivity first. 2. Validate SSL Settings: (This is only applicable if you're using verify_ssl=true, if not then skip to 3.)   curl https://tenablesc_host:tenablesc_port/rest/system   Does the same command work with SSL Verification (no -k)? If not, then you're probably using a self-signed certificate for your Tenable SC instance. Tenable does not support self-signed certificates on their SDK, but you could modify the TA slightly to support it. A quick look through the code shows that it leverages the restfly library so you could update the APISessions base class to support your self-signed certificate. Do this as your own risk, it will be a trade-off between SSL verification and upstream support. 3. Validate Credentials If you've managed to establish connectivity then I would test if your credentials are working properly.   curl https://tenablesc_host:tenablesc_port/rest/system -H "X-APIKey: accessKey=123; secretKey=123;"   (Replacing 123 with the appropriate keys) does this return the same content as the previous commands? If not and you receive a login denied message instead, validate that your credentials are correct and API key login is enabled on the SC instance. Good Luck, let me know how you get on! ... View more

Hi Jason, Unfortunately, add-ons are typically not built with this application-layer redundancy in mind and the Splunk Add-On for Cisco UCS is no exception. Your best bet would either be to manually build this resiliency into the TA itself, or sync the configuration between your forwarders using some kind of automation tool. ... View more

Hi @redhonda03_2  Just to add to this, the reason it's a struggle to get the regex going, is probably the backslashes giving you grief. The backslashes within search regex need to be escaped at the search layer and at the regex layer too. You need to triple escape the backslashes. This solution will work with both conventional lettered-drives and also UNC paths:   | rex field=FilePath "(?i)(?<parent>(?:[A-Z]\:|\\\\{2}[^\\\\]+)\\\\[^\\\\]+\\\\)"   Using an | eval like in @yeahnah's solution is definitely more readable and probably more practical too, just be mindful for any UNC paths. ... View more

Hi @adsquaired, I understand what you're saying. I'm not necessarily suggesting that you change the span value in the mstats command. The | mstats is calculating the average rate on a 5-minutely basis. To aggregate the averages how you suggest, you'll need to use an additional command. | timechart is going to be the one for this. For example, this will give you the maximum average rate across 5 minute windows for each week.   | mstats rate_avg("octets.*") WHERE index="network" chart=true host="device-*" span=5m by host | fields - _span* | rename "rate_avg(octets.rx): *" AS "in * bit/s" | rename "rate_avg(octets.tx): *" AS "out * bit/s" | foreach * [eval <<FIELD>>='<<FIELD>>' * 8 ] | timechart span=1week max(*) as *   When layering statistics on top of each other like this, it's important to bare in mind what they're doing. This is not the maximum rate at any given moment, it's the maximum average. ... View more

Hi @klay824, Your example looks good but going back to my previous post, the message.payload's won't be overriding each other because they are in totally independent searches. With this search, if message1 and message2 occurred at the exact same _time (down to the millisecond) then they will be grouped together. Try this search, which uses a where at the end to filter down to where message1 and message2 are populated at the same _time: index="example" TERM(STOP) | rename message.payload as message1 | table _time, message1 | append [| search index="example" TERM(index) | rename message.payload as message2 | table _time, message2 ] | stats values(message1) as message1, values(message2) as message2, count by _time | where isnotnull(message1) AND isnotnull(message2)   ... View more

Can you share the output of:   ls -l /bin/sudo   Sudo needs to have the suid permission set to run as root. Could this have been unset? Whilst you're there (as the Splunk user), does your sudo command execute properly when ran directly on the CLI (instead of via the scripted input). Also can you share some more details about the OS please? ... View more

Yes, This can be achieved in both Dashboard Studio and Simple XML Dashboards via a drill-down. Select the Visualisation > Scroll-Down the Configuration Menu > Add Drill-Down > On-Click >Link to Dashboard. You can read more here. ... View more

Yes, you've hit the nail on the head when you talk about summarising the data. You'll need to decide how to aggregate or roll-up your search into larger spans. For example do you want to display the average for that range? Or the max? Either way, this can all be decided dynamically using a | timechart and subsearch in tandem: In this example, the avg() will be taken per week if the search is a year long, or per day if the search is a month long (otherwise the default is 5min). | timechart [| makeresults | addinfo | eval search_range = (info_max_time - info_min_time)/(24*3600) | eval search = "avg(*) as * " + case(search_range>=365, "span=1week", search_range>=30, "span=1day", 1=1, span="5min") | table search]   ... View more

I've done some more testing, this time on CentOS and I'm experiencing similar symptoms to you again. $SPLUNK_HOME/bin/splunk start --accept-license --answer-yes is exiting with a 141 (just like tar was on Ubuntu) so it's failing to start. There must be a niche difference between the way Splunk is starting on the two OSs that explains why it started without a problem on Ubuntu. That being said, the change to wrapper.sh from my original answer (redirecting the STDOUT and STDERR away from Splunk) solved this issue again. Based on your original Scripts, here is what is working for me: wrapper.sh #!/bin/bash ( /opt/splunkforwarder/etc/apps/splunk_upgrade_lin_v8/bin/upgrade.sh & ) > /dev/null 2>&1   upgrade.sh #!/bin/bash # set splunk path SPLUNK_HOME=/opt/splunkforwarder # set desired version NVER=8.2.2 # determine current version CVER=`cat $SPLUNK_HOME/etc/splunk.version | grep VERSION | cut -d= -f2` if [ "$NVER" != "$CVER" ] then echo "Upgrading Splunk to $NVER." $SPLUNK_HOME/bin/splunk stop tar -xvf $SPLUNK_HOME/etc/apps/splunk_upgrade_lin_v8/static/splunkforwarder-8.2.2-87344edfcdb4-Linux-x86_64.tgz -C /opt $SPLUNK_HOME/bin/splunk start --accept-license --answer-yes fi   If you still can't get it to work, let's add some debugging to a /tmp/ file to check what's going on. Here is a modified upgrade.sh that will output some more verbose logs. upgrade.sh (with logging) #!/bin/bash LOG=/tmp/splunk_upgrade.log echo "Starting Splunk Upgrade." >> $LOG # set splunk path SPLUNK_HOME=/opt/splunkforwarder # set desired version NVER=8.2.2 # determine current version CVER=`cat $SPLUNK_HOME/etc/splunk.version | grep VERSION | cut -d= -f2` echo "Current Version: $CVER, Target Version: $NVER" >> $LOG if [ "$NVER" != "$CVER" ] then echo "Proceeding with upgrade." >> $LOG echo "Upgrading Splunk to $NVER." echo "Stopping Splunk." >> $LOG $SPLUNK_HOME/bin/splunk stop 2>>$LOG echo "Stopping Splunk returned exit code: $?." >> $LOG echo "Unpacking Splunk." >> $LOG tar -xvf $SPLUNK_HOME/etc/apps/splunk_upgrade_lin_v8/static/splunkforwarder-8.2.2-87344edfcdb4-Linux-x86_64.tgz -C /opt 2>>$LOG echo "Unpacking Splunk returned exit code: $?." >> $LOG echo "Starting Splunk." >> $LOG $SPLUNK_HOME/bin/splunk start --accept-license --answer-yes 2>>$LOG echo "Stopping Splunk returned exit code: $?." >> $LOG fi echo "Complete." >> $LOG   Please share a copy of these logs and we'll see if there's anything different happening on your system. ... View more

Your best bet is going to be the splunkd_access sourcetype. index="_internal" sourcetype="splunkd_access" host="<<SPLUNKHOST>>" | stats values(user) as user | mvexpand user That being said, if you're auditing a SH, you're going to see lots of traffic from splunkweb. To address this you could filter out the Splunk user agent (the risk with this is that user-agents can be modified):   index="_internal" sourcetype="splunkd_access" host="<<SPLUNKHOST>>" | regex useragent!="Splunkd?\/[\d\.]+ \(" | stats values(user) as user | mvexpand user    Or filter out any localhost connections: index="_internal" sourcetype="splunkd_access" host="<<SPLUNKHOST>>" clientip!="127.0.0.1" | stats values(user) as user | mvexpand user​ ... View more

Ah-ha, I understand. Depending on exactly what you're trying to acheive. If you truly are looking to use TERM in the conventional sense to group the message.payload properties, then you can let Splunk natively take care of the different TERMs via a subsearch:   index="example" TERM(STOP) | eval message1 = src_ip | table _time, message1 | append [| search index="example" TERM(index) | eval message2 = src_ip | table _time, message2 ] | stats values(message1) as message1, values(message2) as message2, count by _time If you know that your TERMs will always have a consistent prefix/suffix, e.g. always surrounded by a space, then you could use the LIKE command as follows:   index="example" (TERM(STOP) OR TERM(index)) | eval message1=if(LIKE(_raw,% STOP %), message.payload, null()) | eval message1=if(LIKE(_raw,% index %), message.payload, null()) | stats values(message1) as message1, values(message2) as message2, count by _time     ... View more

In your example, the tokens referenced in the depends= attributes are changing regardless of the submit action. Without using any custom JS; the only way (that I know of) to trigger a token change upon submit, is to use a search. Here is an example based on your original question that makes use of a | makeresults command which is executed upon submit.   <form> <label>My Dashboard</label> <search> <query>| makeresults | eval panel="$panelToggle$" | fields panel</query> <done> <unset token="panelA"></unset> <unset token="panelB"></unset> <unset token="panelC"></unset> </done> <done> <condition match="$result.panel$==&quot;A&quot;"> <set token="panelA"></set> </condition> <condition match="$result.panel$==&quot;B&quot;"> <set token="panelB"></set> </condition> <condition match="$result.panel$==&quot;C&quot;"> <set token="panelC"></set> </condition> </done> </search> <fieldset submitButton="true" autoRun="false"> <input type="dropdown" token="panelToggle" searchWhenChanged="false"> <label>dropdown_token</label> <choice value="A">A</choice> <choice value="B">B</choice> <choice value="C">C</choice> </input> </fieldset> <search> <query>| makeresults | eval panel="$panelToggle$"</query> <progress> <unset token="panelA"></unset> <unset token="panelB"></unset> <unset token="panelC"></unset> </progress> <done> <condition match="$results.panel$==&quot;A&quot;"> <set token="panelA"></set> </condition> <condition match="$results.panel$==&quot;B&quot;"> <set token="panelB"></set> </condition> <condition match="$results.panel$==&quot;C&quot;"> <set token="panelC"></set> </condition> </done> </search> <row> <panel depends="$panelA$"> <table> <title>Panel A</title> <search depends="$panelA$"> <query>index=a |table a b c</query> </search> </table> </panel> <panel depends="$panelB$"> <table> <title>Panel B</title> <search depends="$panelB$"> <query>index=a |table a b c</query> </search> </table> </panel> <panel depends="$panelC$"> <table> <title>Panel C</title> <search depends="$panelC$"> <query>index=a |table a b c</query> </search> </table> </panel> </row> </form>  (Note the use of depends= for the searches to, to stop the searches from running unnecessarily).    ... View more

Does this do what you need it to? (index="example" TERM(STOP)) OR index="example2" | eval message1=if(index=example, message.payload, null()) | eval message2=if(index=example2, message.payload, null()) | stats values(message1) as message1, values(message2) as message2, count by _time   ... View more

The only two things that I modified to get the installer to work were: a) remove the -v switch from tar b) pipe the forked-shell's output to /dev/null as per my answer Either of these should work. Nothing else needed to be changed to get this to work on Splunk UF 7.2.6 (Ubuntu 22.04), including the location of the if then statement. Entire wrapper.sh: #!/bin/bash ( /opt/splunkforwarder/etc/apps/splunk_upgrade_lin_v8/bin/upgrade.sh & ) > /dev/null 2>&1 I've asked for some more details, can you please share them when you get a chance? ... View more

Can you share the output of the following commands so that I can recreate your environment and help you to debug: grep ^VERSION= /etc/os-release getenforce /opt/splunkforwarder/bin/splunk display boot-start ... View more

Interesting let me try and recreate the issue as this was working fine in an Ubuntu VM. What OS are you testing on? Are you using SELinux? Are you using systemd or initd?   ... View more