Hey,
I am looking to add a static field "instance=testdrive" to all results from a source input with td-idp-manager in the path.
I was able to extract a value from the source like so, but would prefer to not have to extract the "td-idp-manager" path in source and just use the word "testdrive" instead. Not sure the syntax to use...
What i am currently using:
props.conf
[source::.../td-idp-manager/*]
REPORT-instance = testdrive_transform
transforms.conf
[testdrive_transform]
SOURCE_KEY = source
REGEX = (?i)[\/A-Za-z]+\/(?<instance>td-idp-manager+)/.*
fields.conf
[instance]
INDEXED_VALUE = false
There are several possibilities that I can think of - choose your favorite:
1 - Add a field to each event
In props.conf
[source::.../td-idp-manager/*]
TRANSFORM = testdrive_transform
EXTRACT-tim1 = \sinstance:(?<instance>testdrive)$
In transforms.conf
[testdrive_transform]
REGEX=(.*)
FORMAT=$1 instance:testdrive
DEST_KEY=raw
This will add the string " instance:testdrive" to each event, which is then extracted into the field. Now you can use instance=testdrive
in your searches.
2 - Use Tags
Delete the transform that you have now. Set up a tag named testdrive
that corresponds to
source=*td-idp-manager*
Then you can search by entering tag=testdrive
3 - Use Lookups
Keep your current transform, but change the name of the field extracted - make it instance_raw
instead. Then set up a lookup table with two columns:
instance_raw,instance
td-idp-manager,testdrive
Set this as an automatic lookup, and then you will have a field named instance with the value testdrive. So you will be able to search instance=testdrive
. If you have multiple values that you want to map, just add additional entries to the lookup table.
Summary
Option 1 adds a small amount of overhead at indexing time, and a few bytes to each event.
Option 2 is all done at search time; this is the most simple option.
Option 3 is the only one that uses your existing transform, but it then uses a lookup to map the values.
For option 3, keep your transform. Just change <instance>
to <instance_raw>
in your REGEX.
You can create the lookup very easily from the Splunk Manager UI:
Once the lookup is created, you can see that it adds additional entries to props.conf and transforms.conf. If you want to edit them directly, take a look at Setup a fields lookup based on a static file
There are several possibilities that I can think of - choose your favorite:
1 - Add a field to each event
In props.conf
[source::.../td-idp-manager/*]
TRANSFORM = testdrive_transform
EXTRACT-tim1 = \sinstance:(?<instance>testdrive)$
In transforms.conf
[testdrive_transform]
REGEX=(.*)
FORMAT=$1 instance:testdrive
DEST_KEY=raw
This will add the string " instance:testdrive" to each event, which is then extracted into the field. Now you can use instance=testdrive
in your searches.
2 - Use Tags
Delete the transform that you have now. Set up a tag named testdrive
that corresponds to
source=*td-idp-manager*
Then you can search by entering tag=testdrive
3 - Use Lookups
Keep your current transform, but change the name of the field extracted - make it instance_raw
instead. Then set up a lookup table with two columns:
instance_raw,instance
td-idp-manager,testdrive
Set this as an automatic lookup, and then you will have a field named instance with the value testdrive. So you will be able to search instance=testdrive
. If you have multiple values that you want to map, just add additional entries to the lookup table.
Summary
Option 1 adds a small amount of overhead at indexing time, and a few bytes to each event.
Option 2 is all done at search time; this is the most simple option.
Option 3 is the only one that uses your existing transform, but it then uses a lookup to map the values.
Thanks lguinn, i like option #3 the best, will go with lookups. Do i need to do any changes in props.conf such as
EXTRACT-instance = testdrive_transform or call the lookup in props.conf?