Lookup- subnet matching in csv to autolookup in multiple indexes

[_ashwin]

Hey Guys,

I am still figuring out the lookup feature. I have checked the previous question but couldnt get a way out.
I wanted to do something similar as described here:
http://splunk-base.splunk.com/answers/57094/join-ip-with-a-subnet

Note: I have configured lookup with GUI way, no editing of any .conf file done yet.
No Advanced options set in definitions.

Scenario: I have a CSV file (subnet-lookup.csv) with list of subnets and appropriate identifier name.
e.g.
Subnet,Name
10.1.1.0/24,ABC
10.1.2.0/24,PQR
10.1.3.0/24,XYZ
Uploaded to Splunk , configured definitions and autolookup with lookuptable name subnet-tagged

Query: |inputlookup subnet-tagged is showing the csv results successfully in splunk.

so below are my queries:
1. Will the subnet field in csv can be matched with any IP field of the logs by default or i have to configure something additional so as to match the CIDR subnet in csv file?
2. What do i need to specify in autolookup ,if i want to use same lookup across all my indexes and why index is not available in dropdown ?
3. When i am trying to run query like
sourcetype=foo ip=* | lookup subnet-tagged ip OUTPUT Name | table ip Subnet Name

It is loading all the results with Subnet Name as blank whereas expected are only the IP matching the subnet range in csv file with respective Name.

let me know if i am not clear at any steps and how shall i move forward to get this working.

Thanks in advance.

[dwaddle]

1. Will the subnet field in csv can be matched with any IP field of the logs by default or i have to configure something additional so as to match the CIDR subnet in csv file?

You have to enable the CIDR match_type option on the lookup file definition. This is (best I can tell) not available in the Manager GUI, and must be done by editing the transforms.conf configuraton file directly. The previous example you pointed to shows how to do that.

1. What do i need to specify in autolookup ,if i want to use same lookup across all my indexes and why index is not available in dropdown ?

Automatic lookups can be enabled for a sourcetype, source, or host, but not for an index. You may want to clarify what you mean here. As a shortcut, to apply this lookup to EVERYTHING, you could apply it to a source of .../*.

Under the covers, automatic lookups are defined as rules in props.conf config files. The props.conf specification supports three different types of stanzas that you can apply a rule to - sourcetype, source, and host. Of the three, sourcetype allows no wildcard, but source and host both do.

Borrowing from the docs @ http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met.
*   matches anything but / 0 or more times.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So a source of .../* means roughly "any filename under any depth of recursion". This covers "almost everything", with the exception of some values of source that aren't based on a file name at all - like scripted inputs. A similar construction of host::* would make the lookup fire for any and all values of host.

1. When i am trying to run query like sourcetype=foo ip=* | lookup subnet-tagged ip OUTPUT Name | table ip Subnet Name

It is loading all the results with
Subnet Name as blank whereas expected
are only the IP matching the subnet
range in csv file with respective
Name.

This is because of the missing match type I mentioned above.

I will update the examples in the original post about this in order to show how to make this lookup fire automatically.

[dwaddle]

See update

[_ashwin]

Thanks dwaddle.

I will give it a try on the things mentioned here.
Also I didnt get the part where you mentioned " to apply this lookup to EVERYTHING, apply it to a source of .../.
So does this source ( .../ ) will cover all data sources ? what does it indicates ?