Extract field in indexing time

[splunklearner]

I want to extract value from the following field while indexing the data to use it to map it with index.

vs_name=v-jupiter-prd-cbc-us.sony-443-ipv6

I want to extract every field after v- and till sony. I.e., jupiter-prd-cbc-us.sony as fqdn so that this fqdn will check in lookup to map it to correct index.

Please help me with props and transforms to extract fqdn correctly.

[livehybrid]

Hi @splunklearner 

This is quite complex to achieve in props/transforms but shouldnt be impossible - lets have a go..

This is what it would look like as SPL - use this to tweak your eval to match your field names and config, then apply to the transforms as below. 

| makeresults 
| eval _raw="something=v-jupiter-prd-cbc-us.sony-443-ipv6"
| eval hostType=replace(_raw, ".*v\-(?<hostType>[^\.]+)\.sony.*", "\1")
| eval yourIndex=json_extract(lookup("testlookup.csv",json_object("hostType",hostType), json_array(index)),"index")
``` as one line ```
| eval yourIndexNew=json_extract(lookup("testlookup.csv",json_object("hostType",replace(_raw, ".*v\-(?<hostType>[^\.]+)\.sony.*", "\1")), json_array(index)),"index")

You will also need a lookup in $SPLUNK_HOME/system/lookups - in this example its testlookup.csv. For the purposes of testing in SPL you can create a temporary lookup with this:

| makeresults 
| eval hostType="jupiter-prd-cbc-us", index="index1" 
| outputlookup testlookup.csv

Props/transforms.conf

== props.conf ==
[yourSourcetype]
TRANSFORMS-defineIndex = defineIndex

== transforms.conf ==
[defineIndex]
INGEST_EVAL = index=json_extract(lookup("testlookup.csv",json_object("hostType",replace(_raw, ".*v\-(?<hostType>[^\.]+)\.sony.*", "\1")), json_array(index)),"index")

For more info on how the lookup command works, have a look at https://docs.splunk.com/Documentation/Splunk/9.4.0/SearchReference/ConditionalFunctions#lookup.28.26...

Please let me know how you get on and consider accepting this answer or adding karma this answer if it has helped.
Regards

Will

[splunklearner]

I already have ingest eval in place. I only need to extract fqdn from vs_name in order to match there. 

[livehybrid]

You would therefore use "vs_name" in-place of "_raw" in the replace command.

You can use multiple transforms on a single sourcetype - even if you're already using an INGEST_EVAL.

For example

== props.conf ==
[yourSourcetype]
TRANSFORMS-defineIndex =.defineIndex
TRANSFORMS-extractServerId =.extractServerId
... etc ...

Please let me know how you get on and consider accepting this answer or adding karma this answer if it has helped.
Regards

Will