topic Re: Merging events from two indexes in Splunk Search

Merging events from two indexes

corti77 — Tue, 01 Jun 2021 12:37:39 GMT

I would like to create a dashboard to query the logs of our two firewall devices (paloalto and sns). Both has their own index created.

Firstly I tried to simply query both indexes

index=pan_logs OR index=sns | table _time, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes

with the issue that in the stormshield index some fields are named slightly different. I then tried to rename the values in order to merge all events from both indexes... but I didnt succeed.

index=pan_logs [ search index=sns | rename src as src_ip, dst as dest_ip | eval bytes=bytes_in+bytes_out ] | table _time, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes

could someone point me to the right direction?

thanks a lot.

Re: Merging events from two indexes

gcusello — Tue, 01 Jun 2021 12:44:55 GMT

Hi @corti77,

your first approach is better: you have to use eval and rename to merge data, something like this:

index=pan_logs OR index=sns | eval src_ip=if(index="sns",src,src_ip), dest_ip=if(index="sns",dst,dest_ip), bytes=bytes_in+bytes_out | table _time action src_ip dest_ip app user src_port dest_port bytes

Ciao.

Giuseppe

Re: Merging events from two indexes

richgalloway — Tue, 01 Jun 2021 12:51:15 GMT

The second search was very close. Using a subsearch creates a very different type of query, however. Just add the rename command - there' s no need to associated it with a specific index because it will only apply to events with the stated fields.

index=pan_logs OR index=sns | rename src as src_ip, dst as dest_ip | eval bytes=bytes_in+bytes_out | table _time, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes

Re: Merging events from two indexes

corti77 — Tue, 01 Jun 2021 15:25:22 GMT

the field bytes already existed in paloalto logs, so with your query it might get overwrote .

below my draft query so far, using few variables

index=pan_logs OR index=sns | eval src_ip=if(index="sns",src,src_ip), dest_ip=if(index="sns",dst,dest_ip), bytes_new=if(index="sns",bytes_in+bytes_out, bytes) | search src_ip="$source_ip$" AND dest_ip="$destination_ip$" AND action="$fw_action$" | table _time, index, host, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes_new, vendor_action