The submit button doesn't generate an event you can subscribe to. Each element has its own change monitoring via the <change> tag. The Submit button just triggers change actions for any inputs that have changed and are not already set to "search on change" You could create search that runs whenever ANY of the form tokens change that sets a new token with the last change time.    <search> <query> | makeresults | eval last_update_ts=now() ```$TOKEN1$ $TOKEN2$ $TOKEN3$ $TOKEN4$``` </query> <earliest>-1m</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> <done> <set token="last_update_ts">$result.last_update_ts$</set> </done> </search>    That same search COULD actually output to your lookup table using "last_update_ts" as the time field ... View more

I know this is old, but for the benefits of future generations: This is caused by the "*_last_timestamp_*.duosecurity" files being blank. This can happen if you run out of disk space, for example. Duo is trying to read a timestamp form these files, getting a null instead, and attempting to treat it like a number. The fix is to simply delete all of the "*.duosecurity" files form "$SPLUNK_HOME/etc/apps/duo_splunkapp/bin" and restart Splunk. The app will recreate these files with proper timestamps. ... View more

if you go down this path, make your life easier with a scripted alert action on your deployment server and an app for the transforms. It will automate deployment to the parsing nodes, and gives you some flexibility about where this filtering is applied. Basic algorithm: 1. Search for high volume 2. Trigger update alert script 3. Update app in deployment-apps: append to transform REGEX with offending data: just keep adding |NEW PATTERN on the end of the REGEX. If you only care about hosts then use [HOST::*] in props and SOURCE= MetaData:Host in your transform 4. Restart Splunk on deployment server. This is generally easier than trying to login from scripts. You can also trying doing this via REST API: https://docs.splunk.com/Documentation/Splunk/8.0.5/RESTREF/RESTsystem#server.2Fcontrol.2Frestart ... View more

Information related to searches, such as lookup tables, are replicated to the indexers directly via Knowledge Bundle Replication. Basically, every time the the local contents of an app on the search head changes, those changes are replicated to the search heads. This is why you only need apps on your indexers that perform index time operations. The system even keeps track of which search head the bundles are coming form so that search heads can have different versions of the same assets and the indexers use the right one for the right search. Be careful of large lookups, however. Because the bundles need to be sent every time lookups change, large bundles can create a significant search latency and can delay when the contents of the lookup are actually used in searches . For frequently changing lookups, consider preventing the the lookup from replicating by using local=true in your lookup search command. This will prevent the lookup from replicating and wait until data reaches the search head before performing the lookup. The advantage is instant access to new lookup values and no search latency due to frequent bundle updates. The downsides are that you can't leverage multiple indexers so filtering on lookup values may be slow (tip: do the lookup as late in the chain as possible), and you can't use the lookup in Data Models. If you have to use your frequently changing lookup on indexer, consider truncating the lookup frequently to ensure it is small. I do this for an ip-to-username lookup used in a an Accelerated data model. The source query only grabs the latest login per machine/user and only runs every hour so that the bundles are small and replicate less often   Back to the larger question of automatic master node pushes, you can probably do this, but you shouldn't. If you set up your deployment server and master node correctly you can push packages to the Master node's master-apps directory instead of the app directory. And here is a presentation with the details of the command line to validate and push a cluster bundle. Combine that with cron and you have something like automated app pushes to the indexers. The biggest problems are: what happens if the bundle fails validation? What happens if there is a problem during deployment and rolling restart? I strongly recommend deploying cluster bundles during a maintenance window and manually monitoring the process. A safe partial automation is to set up deployment to the to the master node like above, so all you have to do from the GUI is validate and deploy. Another tip: consider creating Intermediate heavy Forwarders (IF) to capture your indexing traffic before it hits your index cluster, i.e. set forwarders to use the IF layer as their "indexers". This effectively removes parsing from your index layer AND means almost every app you would have deployed to your indexers now goes on the the intermediate forwarders, which will happily work with your deployment server. Search time configuration still gets replicated to the index cluster via Knowledge Bundle Replication.  ... View more