Splunk Search

How do I get around the issue of the Segmentation and Subsearch limit if I have 30000 results?

DanielFordWA
Contributor

Hi,

I have a index of raw usage data (iis) and a separate index of entitlement data (rest_ent_prod), both indexes have a unique identifier for each user "GUID".

I would like to be able to ad hoc search the raw usage index for user behavior of users with certain entitlements and also create summary indexes.

The issue I have is the number of unique users with certain entitlements is around 30k and subsearches max out at 10.5k.

Can anyone advise how I can get around this issue?

Thanks,

Dan

0 Karma
1 Solution

jeffland
SplunkTrust
SplunkTrust

Joining may be more comfortable, but you can always get the same mechanics going with a simple stats on a search comprising both sources, split by the field you would usually join on. I.e., instead of

index=iis | join GUID [search index=rest_ent_prod]

you would do

index=iis OR index=rest_ent_prod | stats values(something) by GUID

Check this cool post for more detail!

View solution in original post

woodcock
Esteemed Legend

The answer by @jeffland is absolutely the correct way but if you cannot make that work, and you can deal with using a 2-stage process to pump some of your data out to file or KV-store (using outputlookup), then you can use this trick to escape append/subsearch limits:

https://answers.splunk.com/answers/318428/how-can-i-escape-the-50k-subsearch-limit-while-lin.html

0 Karma

jeffland
SplunkTrust
SplunkTrust

Joining may be more comfortable, but you can always get the same mechanics going with a simple stats on a search comprising both sources, split by the field you would usually join on. I.e., instead of

index=iis | join GUID [search index=rest_ent_prod]

you would do

index=iis OR index=rest_ent_prod | stats values(something) by GUID

Check this cool post for more detail!

Get Updates on the Splunk Community!

Almost Too Eventful Assurance: Part 1

Modern IT and Network teams still struggle with too many alerts and isolating issues before they are notified. ...

Demo Day: Strengthen Your SOC with Splunk Enterprise Security 8.1

Today’s threat landscape is more complex than ever. Security operation centers (SOCs) are overwhelmed with ...

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...