topic Re: splunk match between different sourcetypes in Splunk Search

splunk match between different sourcetypes

kirkj — Fri, 21 Jun 2024 15:37:04 GMT

I'm trying to create a search where I take a small list of IPs from sourcetype A and compare them against a larger set of IPs in sourcetype B. I will then make a table using fields from sourcetype B that do not exist in sourcetype A to create a more detailed look of the events involving the IP.

Is there a way to do this without using a lookup table?

index=paloalto (sourcetype=sourcetype_B OR sourcetype=sourcetype_A) | eval small_tmp=case(log_type="CORRELATION", src_ip) | eval large_tmp=case(log_type!="CORRELATION", src_ip) | where match(small_tmp, large_tmp) | table field A, field B, field C

Re: splunk match between different sourcetypes

yuanliu — Fri, 21 Jun 2024 18:10:07 GMT

Yes, such use cases are quite common, simple, and it is not always appropriate to use lookup table. In fact, correlation search is the most fundamental strength of Splunk. Meanwhile, you do want to consider whether it is appropriate to compare the two sourcetypes in the same time search period.

This said, your final table is not very illustrative for the statement "make a table using fields from sourcetype B that do not exist in sourcetype A" because IP is nowhere in that table. Mind-reading 1: I will insert src_ip into the table. More critically, you did not illustrate what you mean exactly by "compare (IPs from sourcetype A) against a larger set of IPs". In the end result, do you want to list IPs in sourcetype B that do not exist in sourcetype A? Mind-reading 2: I will assume no on this.

index=paloalto (sourcetype=sourcetype_B OR sourcetype=sourcetype_A) | stats values(field_A) as field_A values(field_B) as field_B values(field_C) as field_C values(sourcetype) as sourcetype by src_ip | where sourcetype == sourcetype_A | fields - sourcetype

Here, the filter uses a side effect of Splunk's equality comparator on multivalue fields. (There are more semantically expressive alternatives but most people just use this shortcut.)

Re: splunk match between different sourcetypes

kirkj — Fri, 21 Jun 2024 18:36:45 GMT

Okay let me back up. One sourcetype contains the correlation logs with src_ip as it's primary identifier. the other sourcetype is our threat logs where we see far more data about destination, url, app, etc. I want to create a search that takes the IPs from the correlation logs and looks for the same src_ip in the threat logs within a range of 1-2 hours and returns a detailed table describing what could have caused the correlation event to be created.

Is this possible to do without using an outputlookup?

Also this index has a datamodel that I could leverage where nodenames are log.threat and log.correlation

Re: splunk match between different sourcetypes

yuanliu — Sat, 22 Jun 2024 04:33:54 GMT

Have you tried the search I suggested? That does exactly what you are saying here, and doesn't use lookup. (I understand field_A, field_B, etc., are standins for real field names.)