SPlunk DB Lookup

[chitra]

Hi ALL,

I have usecase where i need to display the mis match values between different data source in dashboard.

Use Case:
Consider i have 3 database data sources DS1, DS2 and DS3.
1. DS1 data will index daily at 1 AM. Take the unique Id's say Ds1_ID from this index
2. In DS2 database, I have to ingest only those events whose Ds2_Id matches with Ds1_ID.
3. In Ds3 database, i have to ingest only those events whose DS3_ID matches with Ds1_ID.
4. Using Step 2 and step 3 indexed data, i have to create the mismatch report.

Please suggest the methods to solve this use case.

[woodcock]

There is no "natural" way to create index-time filters based on data that is not inside each event.

[niketn]

In essence you want to compare DS2_ID with DS3_ID, which contain the same value of DS1_ID.

Step 1) You can create aliases for DS2_ID and DS3_ID as DS_ID based on their index/sourcetypes. (Alternatively you will have to use rename or eval during search time to match the field names for correlation.
Step 2) Run the following stats command which will combine results from DS2 (index/source) and DS3 (index/source) and select only rows with eventcount<=1.

<Your base search with multiple index/sourcetype for DS2 and DS3 respectively> DS_ID=* | stats count(DS_ID) as eventcount values(sourcetype) as SourceTypes matched by DS_ID | search eventcount<=1

You can also use transaction or append or join but they will be expensive queries compared to stats.

[chitra]

Only Ds1 has events in splunk and other data source are databases.

i need to index only those events whose DS1 id's are matched with database Ds2 Id. i need to correlate splunk field with database column.