Archive
Highlighted

Find correlation between nearly static data and a data feed.

New Member

I'm looking to find a way to match up info from one data source that only changes once per day, and another data source that changes frequently. Each night we map user_id to computer_id and that file gets ingested into Splunk. During the day I have a constant stream of data coming in with mappings of action_taken and computer_id.

My challenge is that I need to be able to look up the mapping of user_id to action_taken historically, to within the minute, and through the API.

What is the best way to search/lookup/report that mapping?

Thanks!

0 Karma
Highlighted

Re: Find correlation between nearly static data and a data feed.

Legend

I think you should use an lookup table and maybe even setup automatic lookup. In a nutshell, the process you have running once a day, generates a .csv file. Setup this .csv as a lookup and have the daily process update the same file. You can also do an automatic lookup which will automatically, include the user_id on every search. Here's what your search will look like. Assume your lookup file is called usermapping.csv

your base search on streaming data | lookup usermapping.csv computer_id OUTPUT user_id | ....

http://docs.splunk.com/Documentation/Splunk/6.0.5/Knowledge/Usefieldlookupstoaddinformationtoyoureve...

0 Karma
Highlighted

Re: Find correlation between nearly static data and a data feed.

Esteemed Legend

The other search should that should be setup to run every 24 hours just after the other file is written is this:

your base search on daily data | dedup user_id computer_id | table user_id computer_id | outputcsv usermapping.csv
0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.