https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148250#M9019 <P>I'm creating a dashboard based on the data contained in 2 files: one with alarms fired from some equipment and one with all available equipment. </P> <P>My 1st attempt was to use lookups to get the data of the equipment file and correlate that with the alarms, but I noticed that since the equipment can change over time, I can no longer relate when looking at historical data. </P> <P>So for my 2nd attempt I'm trying to merge the data (alarms + equipment) before indexing it. I believe that it's possible to achieve this running a python script before indexing the data, but from my searches, I'm not able to find much about it. </P> <P>This is achievable with summary indexing, but I was thinking about using it as a last resort since I had a problem before because the scheduled search didn't run for some reason and I ended up with a hole in the final data.</P> Mon, 23 Feb 2015 00:34:26 GMT diogofgm 2015-02-23T00:34:26Z https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148250#M9019 <P>I'm creating a dashboard based on the data contained in 2 files: one with alarms fired from some equipment and one with all available equipment. </P> <P>My 1st attempt was to use lookups to get the data of the equipment file and correlate that with the alarms, but I noticed that since the equipment can change over time, I can no longer relate when looking at historical data. </P> <P>So for my 2nd attempt I'm trying to merge the data (alarms + equipment) before indexing it. I believe that it's possible to achieve this running a python script before indexing the data, but from my searches, I'm not able to find much about it. </P> <P>This is achievable with summary indexing, but I was thinking about using it as a last resort since I had a problem before because the scheduled search didn't run for some reason and I ended up with a hole in the final data.</P> Mon, 23 Feb 2015 00:34:26 GMT https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148250#M9019 diogofgm 2015-02-23T00:34:26Z https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148251#M9020 <P>Any idea about this?</P> Fri, 27 Mar 2015 11:24:20 GMT https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148251#M9020 diogofgm 2015-03-27T11:24:20Z https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148252#M9021 <P>There are a couple of options for correlating your alarm data but it all really depends on where your equipment data is coming from. One way would be to create a lookup from the external file and then compare it against your alarm data in Splunk. What you're looking for is this: <A href="http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Addfieldsfromexternaldatasources">http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Addfieldsfromexternaldatasources</A></P> Fri, 27 Mar 2015 15:42:51 GMT https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148252#M9021 rsennett_splunk 2015-03-27T15:42:51Z https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148253#M9022 <P>How is these two type of data correlated? Do you have some equipment_id type of primary key? If there is a primary key available, my suggestion would be to use a Time based lookup which will store the available equipment_id based on time and you should be able to do a lookup to correlate historical data as well.</P> <P>See this</P> <P><A href="http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Addfieldsfromexternaldatasources#Set_up_a_time-bounded_lookup" target="_blank">http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Addfieldsfromexternaldatasources#Set_up_a_time-bounded_lookup</A></P> Mon, 28 Sep 2020 19:18:17 GMT https://community.splunk.com/t5/Dashboards-Visualizations/How-to-complete-correlation-of-data-contained-in-2-files-before/m-p/148253#M9022 somesoni2 2020-09-28T19:18:17Z