Splunk Search

Problem using eventstats - not populating all fields

Path Finder

I have the following search string (which I've obfuscated slightly):

sourcetype=NetworkImpression
| fields User_ID IP Advertiser_ID Buy_ID Ad_ID Creative_ID Site_ID Page_ID
| eval key = User_ID + IP + Advertiser_ID + Buy_ID + Ad_ID + Creative_ID + Site_ID + Page_ID
| eventstats count by key

If I run it over about fifteen minutes of my data (~45,000 events), it behaves as expected. I get a count field in 100% of them. See here:

alt text

However, anything much more and I start to see this:

alt text

That was over half an hour - ~90,000 events - I see this:

alt text

It continues to go down over a longer time frame.

Can anyone give me an indication of what's going on here? There are no errors that I can see in the internal index, and the search inspector doesn't show anything that stands out to me.

Tags (2)
0 Karma

Path Finder

What happens if you add the following to the end of your search: | search NOT count="*"

0 Karma

Path Finder

Now that I've done some more googling it looks like you might be right - there's a [stats] stanza documented for limits.conf which has a maxresultrows option. It's still specified as only applying to searches that create additional rows, but if it's applying to my eventstats command that'd make sense why I stop seeing values after ~50,000 rows, as that's the universal default for maxresultrows.

0 Karma

Path Finder

I just checked the docs, and it looks like this option only affects search commands that grow the result set, so hopefully this isn't it.

0 Karma

Path Finder

The thing is that the search still returns all my results - just many of them are not decorated with the data from the eventstats command.

0 Karma

Path Finder

I would run the search more frequently (so less than 50K results) and save the results to a summary index if this is something you need to report often.

0 Karma

Path Finder

There is a "configurable" limit of 50,000 search results returned when you Submit a search:

[searchresults]
maxresultrows = 50000

Although the README says that you should not increase this figure. Can you sample smaller time-ranges?

0 Karma

Path Finder

I can see all the events missing the count field, corresponding to how many are missing in the search in my question. I don't understand though - I can't find any limits options for eventstats, and 100% of them have count.

0 Karma

Path Finder

Yes, I meant to mention- it's always on 100% of events.

0 Karma

Legend

Does the "key" field exist in all the events?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!