As of the time of writing this it is only available on single value, single value icon, and single radial visualizations. I very much would like to see them add it for line graph visualizations. ... View more

Couldn't you theoretically deploy an app with a scripted input that would make changes to files in etc/system/local? Not saying this is the best method because if it fails you could brick the forwarder, but I would think it is theoretically possible if there are no other means possible. ... View more

This is the new link to the documentation. https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Configuredistributedsearch#Use_the_CLI ... View more

punct can be super helpful for trying to diagnose issues by looking at the kinds of events in your logs. I use this pretty regularly on Windows machines when there is an issue going on. index=wineventlog | stats latest(_raw) as _raw count by source, punct | sort - count   ... View more

Yes, if it can be done with stats that would be best. It is the Splunky way to do it. ... View more

Have you tried using a left (also called outer) join vs and inner join. An inner join will only give you data where the node_id appears in both sets of data. A left join will give you all the results from the base search joined where it matches in the sub search. ... View more

Congrats all. ... View more

You can search the internal Splunk logs to see if there are any errors logged. If you run this what do you get? index=_internal sendemail sourcetype!=splunkd_ui_access sourcetype!=splunkd_remote_searches ERROR   ... View more

This is the beauty of using DNS CNames to reference all your Splunk servers in configuration. Ideally you don't put references to any physical names in your configs. That way when you switch servers you can build your new server along side your old server and then when you want to switch to a new servers you just flip the CName over to the new server. ... View more

I think this should work. You shouldn't need to include the host=* or sourcetype=* as every event has a host and sourcetype. |tstats count from datamodel={your data model} where index="es_web" AND {your data model}.action="blocked" by {your data model}.category |sort 5 -count ... View more

The event code description trimming is not turned on by default. You need to specifically turn it on in a local props.conf. ... View more

Your second stats will not work because after the first stats you only have the User ID and count fields. The info_max_time and info_min_time fields no longer exist. ... View more

There are a number of event codes that have static descriptions of the event in each iteration of the event. This page shows how to trim off the event descriptions on ingest. This can save a lot of data. https://docs.splunk.com/Documentation/WindowsAddOn/latest/User/Configuration ... View more

Something like this should work. I called the lookup db_names.csv ... change that to whatever your actual lookup is named. Everything above the comment just emulates the data you gave. | makeresults count=1 | eval _raw="DatabaseName,Instance,CPUUtilization A,A1,10 A,A2,20 C,C1,40 C,C2,50 D,D,60" | multikv forceheader=1 | fields - _time, _raw, linecount ```^^^^ This emulates the data you gave ^^^^``` | eval inst_cpu=Instance+"#"+CPUUtilization | fields - Instance CPUUtilization | inputlookup db_names.csv append=true ```<-- change the lookup name here``` | stats list(inst_cpu) as inst_cpu by DatabaseName | mvexpand inst_cpu | eval Instance=mvindex(split(inst_cpu,"#"), 0) | eval CPUUtilization=mvindex(split(inst_cpu,"#"), 1) | fillnull value="NULL" Instance CPUUtilization | fields - inst_cpu     ... View more

You could do something like this. Imagine a lookup that looks like this. ip_lookup.csv ip 10.10.53.22 127.0.0.1 192.168.0.54     index=myindex ([| inputlookup ip_lookup.csv | stats values(eval("ip=\""+src_ip+"\"")) as search | eval search=mvjoin(search, " OR ")])   This produces ...   index=myindex (src_ip="10.10.53.22" OR src_ip="127.0.0.1" OR src_ip="192.168.0.54")   ... or ...   index=myindex ([| inputlookup ip_lookup.csv | stats values(eval("src_ip!=\""+ip+"\"")) as search | eval search=mvjoin(search, " AND ")])   This produces ...   index=myindex (src_ip!="10.10.53.22" AND src_ip!="127.0.0.1" AND src_ip!="192.168.0.54")     You could even just put the sub-search into a macro that references the lookup to make using it easier for reuse. An example would be like this. Macro name: my_ip_macro Macro definition:   [| inputlookup ip_lookup.csv | stats values(eval("src_ip=\""+ip+"\"")) as search | eval search=mvjoin(search, " OR ")]     Search using the macro:   index=myindex `my_ip_macro`     ... View more

What do you mean by pulling the _raw? Do you mean "pulling" as in removing _raw from the fields list? Are you using the collect command to add the events into another index? If you do and don't explicitly set a sourcetype then you will not incur a licensing hit for the data copied to the other index. ... View more

I don't think there is a way to do that in a single search. After all you are looking for specific events (search one) and then trying to expand around each of those events (searches 2+). There is the map command which technically isn't a single search as it runs once for each event up to the maxsearches values. That's really not a great solution as it could easily end up running hundreds of searches to actually be useful with terrible performance. You could reduce the maxsearches but then you would not get data for each event in the base search. The best way I can think to do it would be a dashboard with a base search and drilldowns that pass values to a second search to get more detailed information. ... View more

Another option would be to create a dashboard with the base search to pull up the errors and then use a drill down to get the rest of the detail. Here is some example simple xml for a dashboard. <form theme="dark"> <label>Error Dashboard</label> <fieldset submitButton="false"> <input type="time" token="timePicker"> <label></label> <default> <earliest>-24h@h</earliest> <latest>now</latest> </default> </input> </fieldset> <row> <panel> <table> <search> <query>index=_internal sourcetype IN (splunkd) AND ERROR | table _time index host source sourcetype _raw</query> <earliest>$timePicker.earliest$</earliest> <latest>$timePicker.latest$</latest> </search> <option name="count">5</option> <option name="drilldown">row</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">true</option> <drilldown> <eval token="tok_time_earliest">$click.value$ - 10</eval> <eval token="tok_time_latest">$click.value$ + 10</eval> <set token="tok_index">$row.index$</set> <set token="tok_host">$row.host$</set> <set token="tok_source">$row.source$</set> <set token="tok_sourcetype">$row.sourcetype$</set> </drilldown> </table> </panel> </row> <row> <panel depends="$tok_index$"> <title></title> <event> <search> <query>index=$tok_index|s$ host=$tok_host|s$ source=$tok_source|s$ sourcetype=$tok_sourcetype|s$ earliest=$tok_time_earliest|s$ latest=$tok_time_latest|s$ | highlight error</query> <earliest>-24h@h</earliest> <latest>now</latest> </search> <option name="type">raw</option> <option name="list.drilldown">none</option> <option name="refresh.display">progressbar</option> </event> </panel> </row> </form>   ... View more