Splunk Search
Highlighted

Wildcards in lookup file

Path Finder

I'm trying to use wildcards in a lookup file and am not able to get them working. I have referenced other posted answers but am not having success. Steps I have taken:

  • Created a lookup file called 'dt_s.csv' using the web interface by uploading the following content:

    cshost, issuspicious
    www.google.com, yes
    www.*, yes

  • Created a lookup definition called 'dt' using the web interface, based off dt_s.csv

  • Our administrator added the below to transforms.conf on the indexers

    [dt]
    filename = dts.csv
    match
    type = WILDCARD(cs_host)

When I run a search such as '-search- | lookup dt cshost | head 50 | fields cshost, is_suspicious' I only get results for www.google.com and nothing for any other www.* entries.

What are we doing wrong? Is there any other step-by-step official documentation on how to set this up? Thank you.

Answer in comments below: In a pre v6.6 deployment with indexers separate from search head, the [dt] section above has to be added to the local transforms.conf on indexers AND the search head.

0 Karma
Highlighted

Re: Wildcards in lookup file

Contributor
0 Karma
Highlighted

Re: Wildcards in lookup file

Path Finder

Including the props.conf changes? I was under the impression props.conf should only be necessary if we want the lookup to be automatic, which I definitely do NOT in this case.

0 Karma
Highlighted

Re: Wildcards in lookup file

Contributor

We used it only with props.conf

0 Karma
Highlighted

Re: Wildcards in lookup file

Path Finder

I don't understand. You only made the props.conf changes and not the transforms.conf changes?

Can you share your lookup name and the what you put in props.conf as an example?

0 Karma
Highlighted

Re: Wildcards in lookup file

Motivator

transforms.conf seems correct. Problem with your search query

0 Karma
Highlighted

Re: Wildcards in lookup file

Contributor

to clarify we done it with transforms.conf and props.conf as we used automatic lookups and did not tested it with transforms.conf only. So our configs was same as in example:

props.conf
[yoursourcetype]
LOOKUP-user = userlookup user OUTPUT username

transforms.conf
[userlookup]
filename = userlookup.csv
match_type = WILDCARD(user)

0 Karma
Highlighted

Re: Wildcards in lookup file

Path Finder

Thanks. Did you deploy the transforms.conf changes on the search head or on the indexers?

0 Karma
Highlighted

Re: Wildcards in lookup file

Communicator

I tested this and got successful results with

index=*
| head 1
| eval cshost="www.foo.com"
| lookup dt cs
host | fields cshost, issuspicious

I get the same results with www.google.com and www.foo.com (i.e. is_suspicious=yes).

2 things to consider:

1) Look at app context and permissions on the lookup knowledge objects. I doubt this is your issue since it works for the google domain.

2) I was able to do this all from the GUI without having to edit my transforms manually by configuring "advanced" options under the lookup definition, and using WILDCARD(cs_host) as my match type.

View solution in original post

Highlighted

Re: Wildcards in lookup file

Motivator

Adauria,

I made the change in the advanced options, adding WILDCARD(process), and then ran the following search where I'm trying to match on executables other than the two paths, but I'm seeing all process file names and not just defrag.exe that I copied to the desktop and executed.

sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational EventCode=1 Image!=Windows\\System32 Image!=Windows\\SysWOW64 | eval process=lower(process) | lookup isWindowsSystemFile_lookup process | search systemFile=true | table _time dest host user process Image
0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.