find only values in 2 index files

katmagee — Wed, 30 Sep 2020 05:03:22 GMT

I have a index of my gcp firewalls (all of them) and I need to take that and match it against another dataset (firewalls allowed - a CSV) and then return the query for just the information that matches the values in the firewalls allowed with data from gcp_firewall index

Thoughts on what I need to add in the index to achieve that? example syntax below:

index=gcp_firewall "data.jsonPayload.connection.src_ip"="*"
| rename data.jsonPayload.connection.src_ip as Source
| rename data.jsonPayload.connection.dest_ip as Destination
| rename data.jsonPayload.connection.dest_port as Port
| rename data.jsonPayload.instance.vm_name as Name
| rename data.jsonPayload.rule_details.reference as firewall
| dedup Source
| table Source Name Destination Port firewall
| stats count by firewall

This returns EVERY firewall in GCP, when i really just want it to return ones that match the allowed firewall csv.

Re: find only values in 2 index files

manjunathmeti — Mon, 20 Apr 2020 21:44:23 GMT

Assuming firewall field exists in the csv file. Here sub search with inputlookup command filters index gcp_firewall with values in firewall.csv file.

index=gcp_firewall "data.jsonPayload.connection.src_ip"="*" 
    [| inputlookup firewall.csv 
    | rename firewall as "data.jsonPayload.rule_details.reference" 
    | fields "data.jsonPayload.rule_details.reference"] 
| rename data.jsonPayload.connection.src_ip as Source 
| rename data.jsonPayload.connection.dest_ip as Destination 
| rename data.jsonPayload.connection.dest_port as Port 
| rename data.jsonPayload.instance.vm_name as Name 
| rename data.jsonPayload.rule_details.reference as firewall 
| stats latest(*) as * by Source 
| table Source Name Destination Port firewall 
| stats count by firewall

topic Re: find only values in 2 index files in Getting Data In

find only values in 2 index files

Re: find only values in 2 index files