This doesn't necessarily answer your question but I'm having trouble understanding your need to have the subsearch in the first place. The subsearch is getting the messsage_ids for systems with the status fields of clean. As written I don't see why you couldn't just do host=*myhosts KLMS av_status=clean as_status=clean | ... Of course there could be elements of the search you are leaving out which is fine. ... View more

oh but make sure Classification is part of your stats command as well. ... View more

Try adding by Classification to your eventstats. Eventstats works the same as stats in terms of what functions you can do. The main difference is the only fields available after the stats commands are the ones created as part of the functions or included in the 'by'. For example your Date field isn't available after the stats command. ... View more

This isn't a great answer but you could do something like this in the immediate term host=host* sourcetype=foo | rex field=host "\w+(?<host_num>\d+)" | search host_num > 877 .... You could also bake the field extraction into your configs like this so the field is there for searching prior to the first pipe. One of the issues with the above is that it will return all of the events and then strip out those you don't want which isn't efficient. Props [foo] EXTRACT-foo_host_num = \w+(?<host_num>\d+) in host Beyond that I'm curious if anyone else has a different solution to this as I'd be interested as well! ... View more

First thing is remove | format . That command can be used to show you what the output of a search would be if it were a subsearch but you don't want to bake it into an actual subsearch. It sounds as though if you had 10.10.10.0/24 in your lookup you are wanting to search across all of your data for each of the 256 addresses - correct? I don't think there is a command that will allow you to do that. I suppose you could come up with a case statement that translates 10.10.10.0/24 into 10.10.10.* and then maybe use a regex command after the initial results to make sure your data matches an IPv4 address (ie \d{1,3}.\d{1,3}.\d{1,3}.\d{1,3} ) but that seems like it would be high overhead and becomes a pain if you aren't using A, B, or C blocks . The most ideal scenario is if address fields have been defined in your data and search on explicit fields. The state of the data might not support that currently though. ... View more

Also I wouldn't use your DS to push apps with UI elements unless you want them to have kiosk like limitations. Otherwise when users make changes they will be blown away the next time the SH checks in with the DS. ... View more

I'm not sure there is a good way through the GUI (might be wrong). Otherwise though look for inputs.conf in the app default directory as you were thinking. The other thing you might want to explore at some point is having a 'job' server where you try to move your scheduled searches to. That is really just a SH with a more dedicated role not something Splunk specific like a DS, LM, etc. ... View more

There is a dashboard that will show the new and old search side by side but won't highlight the specific changes. I should note though that it will only be able to show changes going forward from when you installed it. Haven't tested it with 6.3 as I'm not using that version. In theory it should work. ... View more

I'd try adding recursive = false to your second stanza. ... View more

Is part of the issue you have agents on several DCs but need to scale back? I'd shut off the agents on the DCs you don't want to have data sent from. If there is a Splunk Deployment Server setup as part of your PoC then there are some options (ie push an 'app' that disables getting the Security logs but allows you to still get the System/Application logs) ... View more

Is the data coming in and the props/transforms on the same Splunk instance or is there some separation? ... View more

I'd invite you to check out an app I made for that sort of thing - though you aren't able to see who made the change*. If you aren't able to use something like subversion an option is to use REST calls but that will only return the current configs. The main thought behind my app was to run the REST searches once a day which allows you at least to see what has changed over a period of time - new things, deleted things, changed things. if it was a critical change you might be able to go back through the splunkd_ui_access logs (_internal index) and see who hit the dashboard and went to the edit screen. https://splunkbase.splunk.com/app/2627/ ... View more

Just to be sure - have you gone under the covers to the transforms and added the CIDR match_type? Unfortunately unless something has changed in like 6.2+ you can't do this via the GUI. As an example I have a lookup named network-allocations and want to do CIDR lookups against the field network [network-allocations] filename = network-allocations.csv match_type = CIDR(network) max_matches = 1 min_matches = 1 ... View more

This might hold some answers; it seems like you want to know the time difference between two events not the time difference between the start and end of an event/action. If you just want to get something like the start and end of something you could use | stats range(_time) by connectionID or the transaction command creates a new field call duration (more expensive search). Otherwise I think you are likely going to need to use streamstats https://answers.splunk.com/answers/129084/calculate-the-time-between-a-transaction-events-starttime-and-the-endtime-of-the-previous-event.html ... View more

How many indexes do you have? I'd start with putting index=(index where syslog data is). Then I'd take out the pipe and search command and simply put "query:" in befor your first pipe. Leave the quotes in assuming there is a space before the q and after the colon. If there IS a space you could put that in the query like index=foo sourcetype=syslog TERM(query:) To learn more about TERM you'll likely have to search on this board. The main point is though the way your search is constructed is it is first pulling back ALL the syslog data and then looking for just the logs that contain query: which is inefficient. Edit: In thinking more about your question - when you say 4 cores is that all that is available on the system and is that system also processing logs? The general rule of thumb is Splunk uses 4 cores to ingest data and 1 core per active search (pre 6.3 or 6.3 w/o turning some knobs and flipping some switches). The next bit is what sort of IOPS does your system get? The reference architecture is something like 12 cores, 12 GB of RAM, and 800 IOPS (or is that 1200 now? maybe just 'enterprise level'). At any rate there are things that can be done to speed up your search but if the system is over taxed then there is only so much that can be done. ... View more

Move your eventstats after your stats command and change it to | eventstats sum(todayValue) as todayTotal ... View more

Depending on the volume of data and other factors (ie lazy quotient) I might look at a join but only really if you are looking to get the avg duration per group and not per group and status. My guess is though what you might really want is the avg duration per status code though right? It might look something similar to this which is the avg and max memory usage by system over a series of weeks - assuming the table thing works ok host | week of 10/04/15 | week of 10/18/15 | etc system1 avg: 14.5 % avg: 21.0 % max: 17.3 % max: 22.5 % system2 avg: 64.6% avg: 67.1% max: 65.9% max: 71.0% If that is the case the query looks like this - at least for my example. You'd need to translate sourcetype="Perfmon:Memory" counter="% Committed Bytes In Use" earliest=-4w@w latest=@w | bin span=1w _time | eval time = "Week of " .strftime(_time, "%m/%d/%y") | eval host=upper(host) | stats avg(Value) as avg max(Value) as max by host time | foreach avg max [eval <<FIELD>> = "<<FIELD>>: " .round(<<FIELD>>,1). " %"] | eval val = avg . "|" .max | makemv val delim="|" | xyseries host time val The part to pick up on is at the stats command where I'm first getting a line per host and week with the avg and max values. The foreach bit adds the % sign instead of using 2 evals. Then I'm creating a new field to merge the avg and max values BUT with a delimiter to use to turn around and make it a multivalue field (so that the results show up on 2 lines). Then I'm using the xyseries command that I referenced above. ... View more

Yeah look at jkat54's answer. I don't play with the GUI field extractor /shrug. If you don't have CLI access I'd bake that into a macro or something. ... View more

Remove the \ in front of your "." as that is telling the engine to specifically look for a dot character. You also don't have a closing paren though that might be a copy/paste issue. The following methodology should work (in theory) ... | rex "^(?<field1>.{8})(?<field2>.{6})" ... View more

Here is a link to all search commands which I have bookmarked. Once there you can navigate to each command easily enough. http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/ListOfSearchCommands I have often wished foreach would work with the fieldformat command which would allow you to add something like your % sign but keep the results an int under the covers but it "only" works with eval. Obviously eval is very powerful. The only other real issue I've run into is if the fields have a special character like "." or ":" which is why I included the bit about adding the single quotes in my answer (which solves for that issue). Being able to list fields makes it run faster than a wildcard pulling in all fields but sometimes I have to use that as there isn't a way to have it run over all fields EXCEPT this field, that field, etc. At any rate as with all commands you can over use foreach but it is incredibly handy. ... View more

Bleh I'm tried of monkeying with the formatting to have the bullet list items show up correctly 😃 ... View more