Getting Data In

How do I get my CSV inputlookup search to return a certain field?

tschack_welltok
New Member

I have searched through the knowledge base and have tried a number of things to fix my issue. I have not found my answer....so I am asking for help:

I created a lookup table file using a CSV called "shunlist.csv".

shunlist.csv format (first 5 lines):

srcip,timestamp,info
1.34.83.14,2015-09-11 06:02:53,SSH Brute Force
1.93.11.145,2015-09-03 15:50:58,RA SCAN Unusually fast Terminal Server Traffic Inbound
1.93.20.160,2015-09-15 19:30:40,RA SCAN Unusually fast Terminal Server Traffic Inbound
1.233.92.197,2015-09-06 19:52:15,SSH Brute Force

The following commands work fine:

| inputlookup shunlist.csv| table *
| inputlookup shunlist.csv | format

When I search using the following command, I get results, but I do not see the info field (from the CSV file) in the list of fields:

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr]

I try this command and I only see the srcaddr and dstaddr fields. The info field does not show up.

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr] | fields srcaddr dstaddr info

Any assistance will be appreciated. Thanks!

0 Karma

somesoni2
Revered Legend

I guess you're doing two things here-
1) Filter the flow logs to show only from dstaddr present in the lookup (in field srcip) [Done using subsearch below]
2) Enrich the filter data by adding info field from the lookup. [Done using looku command below]

So, try something like this

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr] |table _time protocol srcaddr srcport dstaddr dstport 
 | lookup shunlist.csv srcip as dstadrr OUTPUT info
0 Karma

HeinzWaescher
Motivator

Does the "info" field exist in your base search or do you want to add it here by a lookup?
Your inputlookup results in:

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT AND (dstaddr=A OR dstaddr=D OR...)

It adds a filter using dstaddr values that exist in your lookup file.

To add the info field you should use something like this:

 index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT
| lookup shunlist.csv srcip as dstadrr OUTPUT info
0 Karma

tschack_welltok
New Member

Thanks for the answer HeinzWaescher.

Here is what I am trying to accomplish:

Search through flow logs and match "dstaddr" to the "srcip" field in shunlist.csv. If a match is found, display the src and destination information from the flow in addition to the the "info" field from the match line in the shunlist.csv file. Here's the table statement I want to end up with:

|table _time protocol srcaddr srcport dstaddr dstport info

I have done this before using known bad ssl certificates lookups but having issues on another splunk setup. (http://vitalisec.blogspot.com/2014/07/ssl-blacklist-bro-and-splunk.html)

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...