Hybrid Splunk and ELK correlation?

[splunkcol]

Hi, I hope that asking this question will not cause controversy.

I currently manage a hybrid between Splunk and ELK, some of the sources come directly to Splunk where we pay for the licensing but as there are sources that send very large volumes of information (As we know Splunk is very good but very expensive), we send them to ELK and what we do is that from Splunk we use this type of queries to display data from ELK | ess eaddr="http://localhost:9200" index=paloalto* tsfield="@timestamp" query="src_ip:198.7.62.204" fields="*"

now, making clear that these logs are not arriving directly to Splunk, and what Splunk is doing is an external query to ELK I would like to know if it is possible to correlate two sources, in this case I need to correlate the palalto logs type THREAT with the type TRAFFIC

This is what I have tried but it does not work

| ess eaddr="http://localhost:9200" index=paloalto* tsfield="@timestamp" query="Type:TRAFFIC AND Threat_ContentType:end AND Action:allow AND NOT SourceLocation:172.16.0.0-172.31.255.255 AND NOT SourceLocation:192.168.0.0-192.168.255.255 AND NOT SourceLocation:Colombia" fields="GeneratedTime,Threat_ContentType,Action,SourceIP,DestinationIP,DestinationPort,NATDestinationIP,SourceLocation,DestinationLocation,SourceZone,DestinationZone"
|table GeneratedTime Threat_ContentType Action SourceIP DestinationIP DestinationPort NATDestinationIP SourceLocation DestinationLocation SourceZone DestinationZone *
 | append [ ess eaddr="http://localhost:9200" index=paloalto* tsfield="@timestamp" query="type:THREAT" fields="threat" |table threat ]
|table threat Action *

[PickleRick]

Technically, you can use all the splunk commands with data queried from external sources (be it ES, be it any RDBMS via DBX, be it anything else).

It's just that with data stored within splunk you can leverage all the map/reduce functionality and push much of the work to indexers and perform it in a distributed fashion. Generally, the third-party event generating commands (like ES or DBX pulling ones) are mostly meant as a way for enriching data you have in splunk, not necessarily as a way to use splunk's search head as a frontend for a completely different solution.

So can you craft a search which will analyze/correlate/whatever two or more different data sets pulled from ES? Most probably yes (with all caveats and limitations regarding multiple searches, subsearches and so on). Will it be effective? Most probably not.