About dewoodruff

dewoodruff · ‎11-06-2019

Perfect, that's exactly what I'm looking for! kv was not in my regular arsenal of commands. Simple and dynamic, thank you.

dewoodruff · ‎10-29-2019

Thanks. Almost all of the lookups come from TAs like the TA for Windows or Identity Management. They're doing things like pulling out any 'user' field regardless of source type and enriching with LDAP information. Other examples are the built in date_ fields. I don't think those are lookups, are they? Maybe I'm asking with the wrong term. What I'm really looking for is a search command that will output only the fields that were extracted from the log message itself - no enrichments, no lookups, no built-ins. Does such a thing exist?

dewoodruff · ‎10-29-2019

That's an idea. I don't think our admins would go for that though. Many of the lookups come from TAs.

dewoodruff · ‎10-25-2019

I am using a summary index where the events being added to it contain different types of data, and therefore have different kinds and numbers of fields. I want to run a generic search against this summary index for when type=alert and have an email generated containing all the fields for that event. The problem is others using this search head have defined many different automatic lookups that sometimes match my fields. I do not want the alert email to contain any of the automatic lookup output. How can I disable all automatic lookups for just this search so my email isn't filled with junk? I found this previous question and understand the on up-voted response, but the question is not actually answered - https://answers.splunk.com/answers/113653/ignore-automatic-lookup-just-for-a-search.html I contemplated doing "|fields -" but there are over 150 auto lookup fields in some cases... Is there a way to disable lookups for a specific eventtype, in which case I can define one for this summary index? Thanks in advance.

dewoodruff · ‎11-17-2017

This answer pointed me in the right direction but it was missing a piece of the puzzle - changes to props.conf. Here are all the configuration changes that had to be made to turn clientip into a multivalued field and correctly parse the X_Forwarded_For IPs: transforms.conf - modified. The new regex is highlighted [access-extractions] REGEX = ^**(?<all_xff_ip>(([.\d]+|[a-fA-f0-9\:\.]+|-|localhost)(?:,\s)?)+)**\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]](?:\s++"(?<referer>[[bc_domain:referer_]]?+[^"]*+)"(?:\s++[[qstring:useragent]](?:\s++[[qstring:cookie]])?+)?+)?[[all:other]] # new section [clientip] SOURCE_KEY=all_xff_ip REGEX = (?P<clientip>[.:\d]+|[a-fA-f0-9\:.]+|-|localhost) MV_ADD = true props.conf # new section [access_combined] REPORT-access_combined_clientip = clientip

dewoodruff · ‎07-13-2017

Andrew, that did it. We added transforms.conf section to the search head as well and now it's working. So in short, in a pre v6.6 deployment with indexers separate from search head, the [dt] section above has to be added to the local transforms.conf on indexers AND the search head.

dewoodruff · ‎07-12-2017

Thanks Andrew. We're still on 6.5.4 which does not have that option in the web interface, and we're some months away for a 6.6.x upgrade. The administrator put the configuration block in /opt/splunk/etc/system/local/transforms.conf on the indexers only, not on the search head. Do you think it needs to be added to the search head as well? Does splunk need to be reloaded for the changes to take affect?

dewoodruff · ‎06-30-2017

Thanks. Did you deploy the transforms.conf changes on the search head or on the indexers?

dewoodruff · ‎06-30-2017

I don't understand. You only made the props.conf changes and not the transforms.conf changes? Can you share your lookup name and the what you put in props.conf as an example?

dewoodruff · ‎06-30-2017

Including the props.conf changes? I was under the impression props.conf should only be necessary if we want the lookup to be automatic, which I definitely do NOT in this case.

dewoodruff · ‎06-30-2017

I'm trying to use wildcards in a lookup file and am not able to get them working. I have referenced other posted answers but am not having success. Steps I have taken: Created a lookup file called 'dt_s.csv' using the web interface by uploading the following content: cs_host, is_suspicious www.google.com, yes www.*, yes Created a lookup definition called 'dt' using the web interface, based off dt_s.csv Our administrator added the below to transforms.conf on the indexers [dt] filename = dt_s.csv match_type = WILDCARD(cs_host) When I run a search such as '-search- | lookup dt cs_host | head 50 | fields cs_host, is_suspicious' I only get results for www.google.com and nothing for any other www.* entries. What are we doing wrong? Is there any other step-by-step official documentation on how to set this up? Thank you. Answer in comments below: In a pre v6.6 deployment with indexers separate from search head, the [dt] section above has to be added to the local transforms.conf on indexers AND the search head.

dewoodruff · ‎06-22-2017

The root cause ended up being a custom app with very long file paths caused the knowledge bundle to fail to replicate. Once the app was removed, the knowledge bundle began replicating again and lookups started working. The is apparently a known issue with 6.5.1 and has been resolved in some later version.

dewoodruff · ‎05-15-2017

6.5.1. Search head is standalone. There are multiple indexers.

dewoodruff · ‎05-12-2017

Here you are. Hostname and index name removed for privacy. inputlookup: <hostname>/en-US/app/search/search?q=%7Cinputlookup%20dt1&display.page.search.mode=verbose&dispatch.sample_ratio=1&earliest=-24h%40h&latest=now&display.general.type=statistics&sid=1494609014.208099&display.page.search.tab=statistics index= |lookup dt1 cs_host <hostname>/en-US/app/search/search?q=search%20index%3D<indexname>%20%7Clookup%20dt1%20cs_host&display.page.search.mode=verbose&dispatch.sample_ratio=1&earliest=-24h%40h&latest=now&display.general.type=events&display.page.search.tab=events&sid=1494609057.208159 index= |lookup local=true dt1 cs_host <hostname>/en-US/app/search/search?q=search%20index%3D<indexname>%20%7Clookup%20local%3Dtrue%20dt1%20cs_host&display.page.search.mode=verbose&dispatch.sample_ratio=1&earliest=-24h%40h&latest=now&display.general.type=events&display.page.search.tab=events&sid=1494609075.208172

dewoodruff · ‎05-12-2017

We're not able to find a distsearch.conf file in the search app directory. The query does succeed using local=true.

dewoodruff · ‎05-10-2017

Searches with lookups are failing in our environment. I have created a lookup file called dt1.csv and a lookup definition called dt1. Both the file and the definition have read and write permissions for all users in the search app. This works successfully and shows the contents of the lookup: |inputlookup dt1 However, using the lookup as part of a query such as: <search> |lookup dt1 cs_host Fails with these errors: [indexer1] Streamed search execute failed because: Error in 'lookup' command: The lookup table 'dt1' does not exist or is not available. [indexer2] Streamed search execute failed because: Error in 'lookup' command: The lookup table 'dt1' does not exist or is not available. Every indexer errors out with that message. This happens regardless of the app the lookup is created in, and regardless of the user that creates the lookup. Any ideas on what would cause lookups to fail with these errors? We're on Splunk 6.5.1.

Posts	16
Solutions	1
Karma Given	4
Karma Received	1
Member Since	‎05-10-2017

Online Status	Offline
Date Last Visited	‎06-05-2020 02:03 AM

Disable automatic lookups for a search

Wildcards in lookup file

Why do lookup searches fail with "The lookup table...

Re: Disable automatic lookups for a search

Re: Disable automatic lookups for a search

Re: Disable automatic lookups for a search

Disable automatic lookups for a search

Re: Apache access_combined with X-Forwarded-For in...

Re: Wildcards in lookup file

Re: Wildcards in lookup file

Re: Wildcards in lookup file

Re: Wildcards in lookup file

Re: Wildcards in lookup file

Wildcards in lookup file

Re: Why do lookup searches fail with "The lookup t...

Re: Why do lookup searches fail with "The lookup t...

Re: Why do lookup searches fail with "The lookup t...

Re: Why do lookup searches fail with "The lookup t...

Why do lookup searches fail with "The lookup table...