Search for events from ip addresses in the file

[isitnikov]

Hello,

This seems to be like a very easy thing to do which I can't figure out. I have a csv file with ip addresses. I would like to use that file to see if there is any events that match ip addresses in it. I been looking for hours, and inputcvs, lookup and other commands don't quite work for me. The most i was able to accomplish was display all ip addresses in that file in search results (| inputlookup ip_lookup), but again it is showing me 0 events.

[DalJeanis]

First, try this -

| inputlookup ip_lookup | table IP | format | rex field=search mode=sed "s/IP=//g"

That should create a string called search that looks like

("1.1.1.1" OR "1.1.2.2" OR....)

After you've verified that looks like a decent list of IPs, all in quotes and surrounded by parenthesis and ORs, put that portion inside square brackets and you're good to go...

index=foo  any other search terms 
[| inputlookup ip_lookup | table IP | format | rex field=search mode=sed "s/IP=//g"]
| the rest of your query

[cpetterborg]

Let's suppose that your .csv file has only one column that looks like:

IP
10.0.0.1
10.0.0.2
10.0.0.3

Now if you want to see if there is an IP address that matches this set and you don't have a particular field to look at you will have to collect all the IP addresses in the events. I would use a rex command to do this, something like:

... | rex max_match=0 "(?P<IP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"

Which will make a field called IP for each event and be able to have multiple values from each event. Using the lookup definition name ips, the search that would give me the events that contain one of those IPs would be something like:

... | rex max_match=0 "(?P<IP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | foreach IP [lookup ips IP OUTPUT IP as other] | search other="*"

Now, this is a very general case and your needs may be more specific, but I do know that this works to find any instance of an IP from a csv from all events that make it through your base search, and will result in only those events that contain one of those IPs.

[niketn]

@isitnikov, It would be easier for us to assist you with exact query if you can provide sample data from CSV and your events along with field names.

Best case scenario, If your sourcetype with events has IP Address field extracted (for example my_event_ip) and your csv file has IP addresses stored as my_lookup_ip, you can use a search similar to the following:

<YourBaseSearch> [|inputlookup ip_lookup.csv | rename my_lookup_ip as my_event_ip | table my_event_ip]

In case your events do not have IP Addresses extracted as fields, you can try the following:

<YourBaseSearch> [|inputlookup ip_lookup.csv | fields my_lookup_ip | rename my_lookup_ip  as search | format ]

PS: assuming your lookup file is ip_lookup.csv and lookup IP field name my_lookup_ip. Please change as per your actual file and field names.

[isitnikov]

Sample IP address from CSV is 50.x.x.x
Sample Event: 2017-07-25 15:22:45 Local4.Error 192.x.x.x :Jul 25 15:22:45 EDT: %ASA-session-3-710003: TCP access denied by ACL from 12.x.x.x/61819 to outside:50.x.x.x/80

The IP address is not limited to that one event though, but they all Cisco ASA events if that helps. I hope I am actually providing you the information you requested...I am pretty new to this...

[niketn]

@isitnikov, Based on the data provided does the lookup file contains IPs that are outside (just to understand the purpose of lookup file)? You have still not provided IP field names from CSV or Event/s. For example in the events are you planning to map csv ip to outside:50.x.x.x/80? If you have extracted outside:50.x.x.x/80 as a field what is the field name.

I do understand that there will be several events. I was trying to get sample (only one event for the time being, which you have already provided) and the metadata information i.e. in the index (with sourcetype) what are the actual field names for IP Address. If default search time field extractions is taking place based on Cisco ASA logs or you have created field extractions already it would appear on the left in the Splunk Search UI as Interesting fields on performing a search of the events. Please provide the extracted field name for IP address in raw events, if you see them. If not you might have to created field extraction (http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/ExtractfieldsinteractivelywithIFX).

[niketn]

Also event if you have not created field extraction you can try with the second search (change field name my_lookup_ip with one of your own in the lookup file)

[bheemireddi]

|inputlookup ip_lookup | search (whatever you want to search)

[isitnikov]

When I do |inputlookup ip_lookup | search * I only get the list of IPs in that file and 0 events. I am trying to find every event in the database that has the IPs in the file.

[bheemireddi]

isitnikov,

|inputlookup ip_lookup - is to just display what's in that lookup

If you need to further search for the events that match any of these IPs - then you would need to further add the search

|inputlookup ip_lookup | search

[isitnikov]

I mentioned this below. When I do |inputlookup ip_lookup | search * I only get the list of IPs in that file and 0 events. I am trying to find every event in the database that has the IPs in the file.