There is no hardcoded pause in the new tailing processor for 4.1. The only limit we have is that any given file or directory should only be checked for changes every 1s at most. It may be worthwhile to correlate the log messages in splunkd with output from the strace command to see exactly what splunkd is doing during those 250-300ms, it could be checking files within a component that doesn't log every system call. Another possibility is that data is being read and put on a queue for processing. ... View more

The query looks fine, except that it wouldn't work if the data in the file is really in the order presented above. The startswith condition requires that the event have an earlier timestamp OR arrive first if they have the same timestamp. It cannot have a later timestamp. You cannot really tell Splunk to use the sequence of events in the file. The timestamp is most precedent and for ties, the arrival order is used. ... View more

This limit is purely based on the events retrieved based on the search keywords, so scoping the search down to retrieve fewer than 100-200k events per second per distributed indexer will guarantee that you don't hit this. However, it seems odd that you'd run into this by expanding your time range. This warning comes up for a particular second, and expanding the time range will only make a difference if you capture another particular second that has more events than this (again, per server). The limitation is in splunkd itself, but we could explore making the parameter tunable based on memory available. The reason for the limit is that the index (and its notion of a cursor) doesn't support subsecond precision, and to guarantee inverse-time-order retrieval, we must pull all events for a given second and then sort them with respect to subsecond and arrival. ... View more

Reposting as an answer: Yes, this is an idiosyncrasy in the implementation of the transaction command in the search language. Although you're thinking of the the transaction as being aggregated as time moves forward, the command experiences time in the other direction, we start from the more recent events and move backwards. Once we hit the "startswith" condition, we emit the transaction. The OR will naturally work, but not when we have both conditions, as the first to reach the command wins. Do you really need the "startswith" condition? I think that removing it would solve the problem. ... View more

Hi Nik, I don't think that this is because of the output of the initial search, whether you use tag= or the inputlookup. Splunk will not emit the same event twice to the rest of the search string. The issue is probably that stats count will emit two rows if the tag::eventtype is a multivalued field (that is, there are two tags for the eventtype in the same event). The easiest thing to do here is to use the mvcombine command to flatten the tag::eventtype into a single valued field. You may want to filter out the tags to be those that match your "whats." For example, you may want to have the search be: tag::sourcetype=WINDOWS tag::eventtype=an_action startdaysago=7 tag=groupfoo | eval what=mvfilter(tag::eventtype LIKE "what-%") | mvcombine what | stats count by _time,User_Name,what ... ... View more

The subsearch here never returns results or events to the outer search, it just adds search terms. Putting the fields command in the subsearch makes sure that the boolean expression that the subsearch emits looks like: name=name_1 OR name=name_2 OR name=_name3 ... As opposed to: (name=name_1 AND version_test=version_test1) OR (name=name_2 AND version_test=version_test2) ... However, either way wouldn't cause the outer search to give "name"s not in the subsearch. If you want to find the packages that are inconsistent and the hosts they are on, you probably don't even need the subsearch: sourcetype=package_formatted | stats dc(version) as version_count values(version) as versions values(host) as hosts by name | search version_count>1 | fields - version_count What would you like your final table of results to look like? ... View more