topic How to add a static field using a lookup file for a partial match in the Universal forwarder? in Getting Data In

How to add a static field using a lookup file for a partial match in the Universal forwarder?

varunanand — Tue, 16 Dec 2014 22:57:21 GMT

I am new to splunk and trying to add a static field (action) using a lookup file. It needs to be a partial match with the log entry.
I would prefer doing it in the forwarder because the indexer is common many projects.

lookups/lookup-file.csv

raw,action
*BoExceptions*,exclude
*No existing PackageTrade is found*,include
*deadLetter | 145 | ExchangeExchange[ExchangePattern:InOnly, BodyType:String]*,exclude

transforms.conf

[default]
max_matches=1
min_matches=1
default_match=exclude
case_sensitive_match=false
match_type=WILDCARD(raw)
[lookup-app-log]
filename=lookup-file.csv

I tried the following two approaches.
props.conf

[default]
[source::.../server-1-*.log]
sourcetype=luxor-server
LOOKUP-action=lookup-app-log OUTPUT action
[source::.../server-2-*.log]
sourcetype=luxor-gemfire-server
REPORT-action=lookup-app-log

Re: How to add a static field using a lookup file for a partial match in the Universal forwarder?

lguinn2 — Tue, 16 Dec 2014 23:34:17 GMT

You can't do a lookup on a forwarder. Lookups happen only at search time - forwarders work at input time only.

Here are some references that may help:
Splunk docs: Index time vs. Search time

Splunk docs: Configuration parameters and the data pipeline

Splunk wiki: Where do I configure my Splunk settings?

Re: How to add a static field using a lookup file for a partial match in the Universal forwarder?

varunanand — Wed, 17 Dec 2014 01:17:27 GMT

I configured it on the indexer and it still doesn't work. Is the conf right? Also added:

fields.conf

[action]
INDEXED_VALUE=false

Re: How to add a static field using a lookup file for a partial match in the Universal forwarder?

lguinn2 — Mon, 28 Sep 2020 18:28:17 GMT

Here is what you need:

props.conf
[source::.../server-1-*.log]
sourcetype=luxor-server
LOOKUP-action=lookup-app-log raw as _raw OUTPUT action

transforms.conf
[lookup-app-log]
filename=lookup-file.csv
max_matches=1
min_matches=1
default_match=exclude
case_sensitive_match=false
match_type=WILDCARD(raw)

Note that the name of the field in the events is _raw not raw. That is why you have to specify them both in LOOKUP setting. Also, this lookup may be pretty inefficient.

If you do not have a large number of entries in the lookup table, you might consider using eventtypes instead. Or a combination of eventtypes and tags.

Re: How to add a static field using a lookup file for a partial match in the Universal forwarder?

varunanand — Thu, 18 Dec 2014 05:54:56 GMT

Thanks Iguinn. Solved my issue.