Find correlation between nearly static data and a data feed.

[tboyden]

I'm looking to find a way to match up info from one data source that only changes once per day, and another data source that changes frequently. Each night we map user_id to computer_id and that file gets ingested into Splunk. During the day I have a constant stream of data coming in with mappings of action_taken and computer_id.

My challenge is that I need to be able to look up the mapping of user_id to action_taken historically, to within the minute, and through the API.

What is the best way to search/lookup/report that mapping?

Thanks!

[sundareshr]

I think you should use an lookup table and maybe even setup automatic lookup. In a nutshell, the process you have running once a day, generates a .csv file. Setup this .csv as a lookup and have the daily process update the same file. You can also do an automatic lookup which will automatically, include the user_id on every search. Here's what your search will look like. Assume your lookup file is called usermapping.csv

your base search on streaming data | lookup usermapping.csv computer_id OUTPUT user_id | ....

http://docs.splunk.com/Documentation/Splunk/6.0.5/Knowledge/Usefieldlookupstoaddinformationtoyoureve...

[woodcock]

The other search should that should be setup to run every 24 hours just after the other file is written is this:

your base search on daily data | dedup user_id computer_id | table user_id computer_id | outputcsv usermapping.csv