How to add a static field using a lookup file for a partial match in the Universal forwarder?

[varunanand]

I am new to splunk and trying to add a static field (action) using a lookup file. It needs to be a partial match with the log entry.
I would prefer doing it in the forwarder because the indexer is common many projects.

lookups/lookup-file.csv

raw,action
*BoExceptions*,exclude
*No existing PackageTrade is found*,include
*deadLetter | 145 | ExchangeExchange[ExchangePattern:InOnly, BodyType:String]*,exclude

transforms.conf

[default]
max_matches=1
min_matches=1
default_match=exclude
case_sensitive_match=false
match_type=WILDCARD(raw)
[lookup-app-log]
filename=lookup-file.csv

I tried the following two approaches.
props.conf

[default]
[source::.../server-1-*.log]
sourcetype=luxor-server
LOOKUP-action=lookup-app-log OUTPUT action
[source::.../server-2-*.log]
sourcetype=luxor-gemfire-server
REPORT-action=lookup-app-log

[lguinn2 · Accepted solution]

Here is what you need:

props.conf
[source::.../server-1-*.log]
sourcetype=luxor-server
LOOKUP-action=lookup-app-log raw as _raw OUTPUT action

transforms.conf
[lookup-app-log]
filename=lookup-file.csv
max_matches=1
min_matches=1
default_match=exclude
case_sensitive_match=false
match_type=WILDCARD(raw)

Note that the name of the field in the events is _raw not raw. That is why you have to specify them both in LOOKUP setting. Also, this lookup may be pretty inefficient.

If you do not have a large number of entries in the lookup table, you might consider using eventtypes instead. Or a combination of eventtypes and tags.

[varunanand]

Thanks Iguinn. Solved my issue.

[lguinn2]

You can't do a lookup on a forwarder. Lookups happen only at search time - forwarders work at input time only.

Here are some references that may help:
Splunk docs: Index time vs. Search time

Splunk docs: Configuration parameters and the data pipeline

Splunk wiki: Where do I configure my Splunk settings?

[varunanand]

I configured it on the indexer and it still doesn't work. Is the conf right? Also added:

fields.conf

[action]
INDEXED_VALUE=false