I have a scripted input in Splunk that sends it's data to Splunk via STDOUT. Is there any way to run the script on-demand and have the results sent to Splunk without restarting Splunk? Something like a "oneshot" cli method, but for scripted inputs, not for files. ... View more

I have a pretty complex search where I'm trying to get the DHCP and ACS authentication logs correlated by MAC address for all workstations where a particular user logged into the wireless network. index=main (host=dhcpserver) | extract mac | search [ search host=csacs* index=main CSCOacs_Passed_Authentications [ search host=csacs* index=main CSCOacs_Passed_Authentications user=*username* | fields trans_id ] | transaction maxpause=5s trans_id | lookup normalizemac input AS Calling_Station_ID OUTPUTNEW mac | dedup mac | fields mac ] | rex field=_raw "DHCPACK on (?<ip_assigned>[0-9\.]+) to [^\(]+\((?<hostname>[^\)]+)\)" | fields _time host hostname ip_assigned mac | append [ search host=csacs* index=main CSCOacs_Passed_Authentications [ search host=csacs* index=main CSCOacs_Passed_Authentications user=*username* | fields trans_id ] | transaction maxpause=5s trans_id | lookup normalizemac input AS Calling_Station_ID OUTPUTNEW mac | dedup mac | fields _time host mac user ] | transaction maxspan=20s mac Everything is working okay except for the final transaction to join the transaction between the two systems. I verified the relevant events have the same MAC address and format (lowercase aa:aa:aa:aa:aa:aa) and are well within the maxspan time. Does transaction not work across appended searches? ... View more

I am trying to produce a table for a dashboard that will summarize the top results of the search, but summarize the remaining data in an "other" row. I have the following search which is working, but thinking there should be a cleaner / more efficient way to do it: my_search_criteria index=summary_data search_name="per_source_summary" | stats count as hosts sum(score) avg(score) sum(one) sum(two) sum(three) sum(four) by group | sort limit=10 sum(score) desc | append [ search my_search_criteria index=summary_data search_name="per_source_summary" NOT [ search my_search_criteria index=summary_data search_name="per_source_summary" | stats count as hosts sum(score) avg(score) sum(one) sum(two) sum(three) sum(four) by group | sort limit=10 sum(score) desc |fields group ] | stats count as hosts sum(score) avg(score) sum(one) sum(two) sum(three) sum(four) | eval group="Other" ] | addtotals col=t row=f labelfield=group label="Overall" hosts sum(score) sum(one) sum(two) sum(three) sum(four) This simplified example looks a lot cleaner than my actual search of course. It's using summary data so the search itself isn't taking terribly long or too resource intensive, but is there a "better" way of doing this I'm missing? ... View more

I have an Windows 2008R2 AD Domain Controller with the Splunk Universal Forwarder (v4.3.1) installed. Over the past few weeks the log level on this system increased 2-3 times (another mystery for now) and the events were getting further and further delayed. Searching the logs, I could see blocking happening on this system in teh "wel_queueu" 06-07-2012 09:53:40.275 -0400 INFO Metrics - group=queue, name=wel_queue, blocked=true, max_size_kb=500, current_size_kb=499, current_size=363, largest_size=391, smallest_size=162 I can't find reference to the wel_queue anywhere, but this is obviously an issue. I created a local limits.conf file on the DC under the SplunkUniversalForwarder app to change the maxKBps value from the default of 256 to 8192 and validated the settings took. splunk cmd btool limits list --debug ... SplunkUniv [thruput] SplunkUniv maxKBps = 8192 ... Since making the change I haven't seen any more blocking errors, my latency is still several hours behind and is only gaining about 8 minutes per hour. The network view on the Windows Resource Monitor shows that the forwarder is sending about 120k-140k B/sec which isn't even hitting the 256KBps default... is there anything else I can do to speed up processing on this guy? ... View more