Splunk Search

Search for events from ip addresses in the file

isitnikov
Engager

Hello,

This seems to be like a very easy thing to do which I can't figure out. I have a csv file with ip addresses. I would like to use that file to see if there is any events that match ip addresses in it. I been looking for hours, and inputcvs, lookup and other commands don't quite work for me. The most i was able to accomplish was display all ip addresses in that file in search results (| inputlookup ip_lookup), but again it is showing me 0 events.

Tags (1)
0 Karma

DalJeanis
Legend

First, try this -

| inputlookup ip_lookup | table IP | format | rex field=search mode=sed "s/IP=//g"

That should create a string called search that looks like

("1.1.1.1" OR "1.1.2.2" OR....)

After you've verified that looks like a decent list of IPs, all in quotes and surrounded by parenthesis and ORs, put that portion inside square brackets and you're good to go...

index=foo  any other search terms 
[| inputlookup ip_lookup | table IP | format | rex field=search mode=sed "s/IP=//g"]
| the rest of your query
0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Let's suppose that your .csv file has only one column that looks like:

IP
10.0.0.1
10.0.0.2
10.0.0.3

Now if you want to see if there is an IP address that matches this set and you don't have a particular field to look at you will have to collect all the IP addresses in the events. I would use a rex command to do this, something like:

... | rex max_match=0 "(?P<IP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"

Which will make a field called IP for each event and be able to have multiple values from each event. Using the lookup definition name ips, the search that would give me the events that contain one of those IPs would be something like:

... | rex max_match=0 "(?P<IP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | foreach IP [lookup ips IP OUTPUT IP as other] | search other="*"

Now, this is a very general case and your needs may be more specific, but I do know that this works to find any instance of an IP from a csv from all events that make it through your base search, and will result in only those events that contain one of those IPs.

0 Karma

niketn
Legend

@isitnikov, It would be easier for us to assist you with exact query if you can provide sample data from CSV and your events along with field names.

Best case scenario, If your sourcetype with events has IP Address field extracted (for example my_event_ip) and your csv file has IP addresses stored as my_lookup_ip, you can use a search similar to the following:

<YourBaseSearch> [|inputlookup ip_lookup.csv | rename my_lookup_ip as my_event_ip | table my_event_ip]

In case your events do not have IP Addresses extracted as fields, you can try the following:

<YourBaseSearch> [|inputlookup ip_lookup.csv | fields my_lookup_ip | rename my_lookup_ip  as search | format ]

PS: assuming your lookup file is ip_lookup.csv and lookup IP field name my_lookup_ip. Please change as per your actual file and field names.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

isitnikov
Engager

Sample IP address from CSV is 50.x.x.x
Sample Event: 2017-07-25 15:22:45 Local4.Error 192.x.x.x :Jul 25 15:22:45 EDT: %ASA-session-3-710003: TCP access denied by ACL from 12.x.x.x/61819 to outside:50.x.x.x/80

The IP address is not limited to that one event though, but they all Cisco ASA events if that helps. I hope I am actually providing you the information you requested...I am pretty new to this...

0 Karma

niketn
Legend

@isitnikov, Based on the data provided does the lookup file contains IPs that are outside (just to understand the purpose of lookup file)? You have still not provided IP field names from CSV or Event/s. For example in the events are you planning to map csv ip to outside:50.x.x.x/80? If you have extracted outside:50.x.x.x/80 as a field what is the field name.

I do understand that there will be several events. I was trying to get sample (only one event for the time being, which you have already provided) and the metadata information i.e. in the index (with sourcetype) what are the actual field names for IP Address. If default search time field extractions is taking place based on Cisco ASA logs or you have created field extractions already it would appear on the left in the Splunk Search UI as Interesting fields on performing a search of the events. Please provide the extracted field name for IP address in raw events, if you see them. If not you might have to created field extraction (http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/ExtractfieldsinteractivelywithIFX).

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

Also event if you have not created field extraction you can try with the second search (change field name my_lookup_ip with one of your own in the lookup file)

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

bheemireddi
Communicator

|inputlookup ip_lookup | search (whatever you want to search)

isitnikov
Engager

When I do |inputlookup ip_lookup | search * I only get the list of IPs in that file and 0 events. I am trying to find every event in the database that has the IPs in the file.

0 Karma

bheemireddi
Communicator

isitnikov,

|inputlookup ip_lookup - is to just display what's in that lookup

If you need to further search for the events that match any of these IPs - then you would need to further add the search

|inputlookup ip_lookup | search

0 Karma

isitnikov
Engager

I mentioned this below. When I do |inputlookup ip_lookup | search * I only get the list of IPs in that file and 0 events. I am trying to find every event in the database that has the IPs in the file.

0 Karma
Get Updates on the Splunk Community!

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...