Merging events from two indexes

[corti77]

I would like to create a dashboard to query the logs of our two firewall devices (paloalto and sns). Both has their own index created. 

Firstly I tried to simply query both indexes

index=pan_logs OR index=sns
| table _time, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes

with the issue that in the stormshield index some fields are named slightly different. I then tried to rename the values in order to merge all events from both indexes... but I didnt succeed.

index=pan_logs
[ search
index=sns
| rename src as src_ip, dst as dest_ip
| eval bytes=bytes_in+bytes_out ]
| table _time, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes

could someone point me to the right direction?

thanks a lot.

[richgalloway]

The second search was very close.  Using a subsearch creates a very different type of query, however.  Just add the rename command - there' s no need to associated it with a specific index because it will only apply to events with the stated fields.

index=pan_logs OR index=sns
| rename src as src_ip, dst as dest_ip
| eval bytes=bytes_in+bytes_out
| table _time, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes

[corti77]

the field bytes already existed in paloalto logs, so with your query it might get overwrote .

below my draft query so far, using few variables

index=pan_logs OR index=sns 
| eval src_ip=if(index="sns",src,src_ip), dest_ip=if(index="sns",dst,dest_ip), bytes_new=if(index="sns",bytes_in+bytes_out, bytes)
| search src_ip="$source_ip$" AND dest_ip="$destination_ip$" AND action="$fw_action$"
| table _time, index, host, action, src_ip, dest_ip, app, user, src_port, dest_port, bytes_new, vendor_action

[gcusello · Accepted solution]

Hi @corti77,

your first approach is better: you have to use eval and rename to merge data, something like this:

index=pan_logs OR index=sns
| eval src_ip=if(index="sns",src,src_ip), dest_ip=if(index="sns",dst,dest_ip),  bytes=bytes_in+bytes_out 
| table _time action src_ip dest_ip app user src_port dest_port bytes

Ciao.

Giuseppe