We are joining a large set of information using the join command and are only getting 50000 results. The indexes look like this:
"index=first" has 350k records each with an ID field
"index=second" has 125k records and a similar ID field
We are doing a search like this:
index=first | join ID [search index=second]
and we only get 50000 records. we have already copied default/limits.conf to local/limits.conf and changed subsearch_maxout to 500000, and then restarted the server. The inner join still only returns 50k results.
Note: We are already investigating using 'stats' to join the data instead of 'join' but some of the data ends up as multi-value. so, ultimately, we want to use a join.
Thanks to a previous post you need to change two stanzas in the limits.conf.
[join] & [searchresults].
So for example I now have in $SPLUNK_HOME/etc/system/local/limits.conf:
[join] subsearch_maxout = 500000 [searchresults] maxresultrows = 500000
Note the above value is 500,000. I have added an extra 0 to the defaul
I believe this is because when you run the "join" in your query you are also using the "search" command so both parts are limiting you to the default of 50,000. i.e. joint FIELDNAME [search index=.....
I have the same issue. A join query returns only 50000 out of the 87000 that I expect. So using the answer above I set system/local/limits.conf and increased the maxout to 5M. Restarted splunk and still having the same issue.
Output of "splunk cmd btool limits list join" gives:
subsearchmaxout = 500000
subsearchmaxtime = 60
subsearch_timeout = 120
Any other ideas? Or is this a bug? (FYI. I am using Splunk Enterprise v6.3.0)