Hi,
I have a index of raw usage data (iis) and a separate index of entitlement data (rest_ent_prod), both indexes have a unique identifier for each user "GUID".
I would like to be able to ad hoc search the raw usage index for user behavior of users with certain entitlements and also create summary indexes.
The issue I have is the number of unique users with certain entitlements is around 30k and subsearches max out at 10.5k.
Can anyone advise how I can get around this issue?
Thanks,
Dan
Joining may be more comfortable, but you can always get the same mechanics going with a simple stats
on a search comprising both sources, split by the field you would usually join
on. I.e., instead of
index=iis | join GUID [search index=rest_ent_prod]
you would do
index=iis OR index=rest_ent_prod | stats values(something) by GUID
Check this cool post for more detail!
The answer by @jeffland is absolutely the correct way but if you cannot make that work, and you can deal with using a 2-stage process to pump some of your data out to file or KV-store (using outputlookup
), then you can use this trick to escape append/subsearch limits:
https://answers.splunk.com/answers/318428/how-can-i-escape-the-50k-subsearch-limit-while-lin.html
Joining may be more comfortable, but you can always get the same mechanics going with a simple stats
on a search comprising both sources, split by the field you would usually join
on. I.e., instead of
index=iis | join GUID [search index=rest_ent_prod]
you would do
index=iis OR index=rest_ent_prod | stats values(something) by GUID
Check this cool post for more detail!