I'm actually a little confused why this happened, given that your only "filtering" command is stats , I tested this code below to see if either fields or sort would filter out if the field was null, and it wasn't: | windbag | table host | streamstats count | eval potato=if(count<3, "baked", null()) | sort -potato # I would have expected this line | fields host potato # or this line to filter out the lines to reproduce your issue That being said, I think I can offer a few tips on how to debug and how to move forward. First, I'd suggest going line by line on an event you know should be included in your search. You can create a slightly more custom initial search (i.e. "index=syslog source="Inbound.log" externalId={IdYouKnowShouldExist}) Make sure it passes the stats command, the evals (evals don't filter so this is very unlikely) and finally the fields and sort. Going line by line should help you in this and going forward debug which command is filtering something out, since Splunk is a pipe based language (the output of each line directly goes into the next one, making it very easy to track down what is going wrong, if not why). If you figure this out, let us know and we can more accurately diagnose, or hopefully this helps you identify your search issue or an issue outside your search (the log you thought should've existed doesn't, etc) Second, there is a now() function of the eval command which will allow you to fill whichever time field you're looking to fill. You can include that as part of the variable initialization or as a separate line, example: | eval oc_time=round((coalesce(sent, now()) - received)/3600,2) You can also separately coalesce sent and now() in an eval command. You can also nullfill with the fill null command, like so: | fillnull value=0 OR | fillnull oc_time value=0 You cannot fillnull with the now() parameter unfortunately. One other thing you can try is removing the "sent" value if it is the same as the "received" value, in the case that you only have one log. You can either do this by filling it with null() or now(): | eval sent=if(sent=received, null(), sent) Hope this helps! ... View more

So eval will only work if you have both fields in the same row, which won't work until you've aggregated your two rows into one. That being said, you have to rename the fields so as not to lose them when you convert them (or you can make it a multivalue field and deal with some splitting and other stuff). I'd suggest something like this: baseSearch | eval durationEventA = if(type="eventA", duration, null()) | eval durationEventB = if(type="eventB", duration, null()) | stats values(durationEventA) as durationEventA, values(durationEventB) as durationEventB by id | where isnotnull(durationEventA) isnotnull(durationEventB) | eval delta_duration= durationEventB - durationEventA What I'm doing here: creating two fields on each log that will either be null or have a duration, then aggregating them by id, filtering out those that don't have both, and taking the delta. Hope this helps! ... View more

This took me a long time to figure out what you were trying to get haha. This should work: | eval goodMachines= if(Percent>90 AND Failures=0, 1, 0) | stats sum(goodMachines) as sumOfGoodMachines, count(goodMachines) as countOfMachines by Group | eval percentGoodMachines = sumOfGoodMachines / countOfMachines # optional wrap it in parenthesis and add ."%" for formatting You essentially want to "pass through" the count of what machines are good, but also keep track of how many machines there were total. This can be achieved by using different functions in the stats command (distinct_count vs count vs sum, etc). In this case I'm passing through the sum (since it's booleanized 1's/0's) as the numerator for my percentage and the count (how many times does group appear at all) as the denominator. Does this make sense? Hope this helps! ... View more

Are you familiar with the lookup command, and is there a reason that doesn't work for you? If you check out the docs here https://docs.splunk.com/Documentation/Splunk/8.0.0/SearchReference/Lookup, it does essentially what you want (join/lookup on a value, and optionally OUTPUT any fields you want). Example that should work for you: index="ironport" | lookup exfil_filenames file_name OUTPUT matching_criteria |table file_name matching_criteria Alternatively and perhaps more performantly, index="ironport" [ inputlookup exfil_filenames | fields file_name ] | lookup exfil_filenames file_name OUTPUT matching_criteria | table file_name matching_criteria You also don't need the wildcards in the csv, there is an option in the lookup configuration that allows you do wildcard a field when doing lookup matches: Settings -> Lookups -> Lookup definitions -> filter to yours -> click it -> advanced options -> Match type -> WILDCARD(file_name). Hope this helps! ... View more

I answered a similar question generally here - https://answers.splunk.com/answers/786499/the-percentage-of-non-high-priority-searches-lagge.html#answer-787630. The gist is that you can use the Monitoring Console (and it's inherent queries) to better diagnose specifically what your issues are. Here's the path (assuming you're a Splunk admin on your instance): Settings (Top right) -> Monitoring Console -> Search -> Scheduler Activity: Instance, and inputting the timeframe when this occurred. Hopefully the information under "historical charts" can point you in the direction of what caused this to occur (perhaps the machine blipped, you have a misconfigured search etc), or at least narrow down the timeframe/options so you can continue debugging. Hope this helps! ... View more

After line 6, add a | stats sum(Linux) as Linux, sum(Windows) as Windows by host . This should take all of the numeric values for all indexes post-characterization of Linux or Windows and sum them by host. The alternative is to put your two eval statements first, and then using the same command I showed above in place of your stats command, but that option might be slightly more expensive. You can try them both and assuming they return the same values, pick whichever one returns faster! Hope this helps ... View more