Getting Data In

Help with custom field extraction during indexing

damucka
Builder

Hello,

I monitor several VMs and the corresponding database logs with the following stanza:[monitor:///usr/sap/.../HDB02/vhsfh*/trace/DB_*/*

The example of the path would be:

/usr/sap/NDJ/HDB02/vhsfhndjdb01/trace/DB_VU7

Now, I would like to extract during indexing from the above path, which is the field "source" additional default fields that are always there, which would be:

SYSTEMDB  (here: NDJ)

vhost (here: vhsfhndjdb01)

DBSID (here: VU7)

The corresponding regex expression, if I would do this in the search is:

| rex field=source "\/usr\/sap\/(?P<SYSTEMDB>.+)\/HDB02\/(?P<vhost>.+)\/trace\/DB_(?P<DBSID>.+)$"

 

But how would I do this in the props.conf and transform.conf to make it permanent and always available as the default fields?

I was thinking of the following:

props.conf

[default]

TRUNCATE=0
TRANSFORMS-ExtractKeyFields = fieldextraction
...
 
transforms.conf
[fieldextraction]
REGEX field=source "\/usr\/sap\/(?P<SYSTEMDB>.+)\/HDB02\/(?P<vhost>.+)\/trace\/DB_(?P<DBSID>.+)$"
 
but I am not sure about the syntax in the transforms.conf.
Could anyone advice?
Also, what I am obviously trying to do is to set additional default fields based on other default field which is "source". Is it possible at all? Would the "source" already exist and be available at that point to create additional fields out of it?
 
Last, do I need additionally put the fields.conf on my SH?
 
Kind Regards,
Kamil
 
 
Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

As you suspected, the transform is incorrect.  Try this

[fieldextraction]
REGEX = "\/usr\/sap\/(?P<SYSTEMDB>.+)\/HDB02\/(?P<vhost>.+)\/trace\/DB_(?P<DBSID>.+)$"
SOURCE_KEY = MetaData:Source
FORMAT = SYSTEMDB::$1 vhost::$2 DBSID::$3
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

As you suspected, the transform is incorrect.  Try this

[fieldextraction]
REGEX = "\/usr\/sap\/(?P<SYSTEMDB>.+)\/HDB02\/(?P<vhost>.+)\/trace\/DB_(?P<DBSID>.+)$"
SOURCE_KEY = MetaData:Source
FORMAT = SYSTEMDB::$1 vhost::$2 DBSID::$3
---
If this reply helps you, Karma would be appreciated.
0 Karma

damucka
Builder

Hello,

Thank you. It did not work straight away, but after a bit googling and try-errorying the following worked (interestingly, the REGEX itself had to be changed, I would never come up with this myself, it is also not really documented or at least not so easily understandable for me):

[fieldextraction]
REGEX=^source::\/usr\/sap\/(?P<SYSTEMDB>.+)\/SplunkTest\/(?P<vhost>.+)\/trace\/DB_(?P<DBSID>.+)\/.+$
SOURCE_KEY=MetaData:Source
FORMAT=SYSTEMDB::$1 vhost::$2 DBSID::$3
WRITE_META=true

Kind Regards,

Kamil

Tags (1)
0 Karma

isoutamo
SplunkTrust
SplunkTrust
Yes you should put fields.conf to sh so it knows that those fields are indexed. Otherwise it don’t, unless you are using list on your with <field>::<value> notation.
R. Ismo
0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...