I have the following search string (which I've obfuscated slightly):
sourcetype=NetworkImpression | fields User_ID IP Advertiser_ID Buy_ID Ad_ID Creative_ID Site_ID Page_ID | eval key = User_ID + IP + Advertiser_ID + Buy_ID + Ad_ID + Creative_ID + Site_ID + Page_ID | eventstats count by key
If I run it over about fifteen minutes of my data (~45,000 events), it behaves as expected. I get a
count field in 100% of them. See here:
However, anything much more and I start to see this:
That was over half an hour - ~90,000 events - I see this:
It continues to go down over a longer time frame.
Can anyone give me an indication of what's going on here? There are no errors that I can see in the internal index, and the search inspector doesn't show anything that stands out to me.
Now that I've done some more googling it looks like you might be right - there's a [stats] stanza documented for limits.conf which has a
maxresultrows option. It's still specified as only applying to searches that create additional rows, but if it's applying to my eventstats command that'd make sense why I stop seeing values after ~50,000 rows, as that's the universal default for maxresultrows.
There is a "configurable" limit of 50,000 search results returned when you Submit a search:
[searchresults] maxresultrows = 50000
Although the README says that you should not increase this figure. Can you sample smaller time-ranges?
I can see all the events missing the
count field, corresponding to how many are missing in the search in my question. I don't understand though - I can't find any limits options for
eventstats, and 100% of them have