Okta Data - Appending to lookup table question

[jpsheridan]

I have 5 separate endpoints for our Okta environment that I'm pulling into Splunk. The data is all event driven so if I'm trying to map user, group and application data together and the groups or applications were created over a year ago, it won't find the data unless I move the search window back, causing long searches.

What I would like to do is  create lookup tables for each of those endpoints so I only have to run one long query, one time for those endpoints, and then append any group, application and user that is create each data on a saved search.

Is this the right strategy and could someone help me with how you would do that? I did see a few articles on appending data to table but it didn't seem to meet my needs for this scenario.

Thanks,

Joel

[bowesmana]

You can build a lookup table in time window phases and the window can be any length you need, so you can build your 1 year initial build in 12 goes each doing 1 month and your subsequent regular searches will simply use a 1 day window (or whatever you choose) - the method is the same. For example

<base search to collect data>
| stats <aggregations as needed> by endpoint group application user
| inputlookup append=t your_lookup_file
| stats values(*) as * by  endpoint group application user
| outputlookup your_lookup_file

If your lookup has rows with data associated with endpoint group application and user fields, then the above will do that for any time period.

This may need to be tweaked if you want any other specific behaviour, e.g. if you want to record the latest time of a user for an application/endpoint, then you would add max(lastUse) as lastUse after the stats values.