I have a log line for when the ip is added to the blacklist and another log line with ips that were removed from the blacklist.
What I need to do is search for this ip that was added in the removed line, can you help me?
Here is an example of my log and my research:
--------------------------------------------------------------------------------------------------------------------------------------------
index="my search"
| rex field=_raw "Message: Host (?<ip>.*?) w"
| rex field=_raw "Message: Hosts (?<ips>.*?) w"
| eval ips=mvjoin(ips,", ") | rex mode=sed field=ips "s/, /\n/g"
| table ip ips
I need to loop through the list of ips in column <ips> to find the ip of column <ip>
I need to loop through the list of ips in column <ips> to find the ip of column <ip>
It is unclear what the desired outcome is when the value of <ip> is found in <ips>. (Do you only want to know if <ips> contain <ip>? Or do you need to know when <ip> appears in <ips>? etc.) But I get the gist of the challenge you feel. If your next steps do not require raw data, the easiest way out is to use stats instead of table after your search, i.e.,
index="my search"
| rex field=_raw "Message: Host (?<ip>.*?) w"
| rex field=_raw "Message: Hosts (?<ips>.*?) w"
| eval ips=mvjoin(ips,", ") | rex mode=sed field=ips "s/, /\n/g"
| stats values(ip) as ip list(ips) as ips
If raw data is still important, use eventstats instead of stats. By using list() on <ips>, I am assuming that you want to preserve the order from raw events. But list() is memory intensive and bears more limitations. If raw order is unimportant, use values().
(I highly recommend using text table to illustrate data. Image should be the last resort.)