Splunk Search

Transforms are working on a local Splunk instance, but why are fields not extracted correctly when deployed to my search head cluster?

paimonsoror
Builder

Having a strange issue. I am trying to set up a transform to automatically extract key/value pairs from a non standard XML file. I have a local Splunk instance where I did this development. Here is what I did:

  1. Did a Add New of a data source
  2. Selected a local copy of the XML file
  3. The data was read once (not set to monitor the file)
  4. Data was indexed into an index called 'app_test'
  5. Data is in the Search app

I then went ahead and updated my props.conf and transforms.conf in my $SPLUNK_ROOT/etc/apps/search/local

props.conf

[dmwt:xml]
REPORT-myXmlClassName = xmlTransform

transforms.conf

[xmlTransform]
REGEX = \<(\w+[^\n\/\>]+)\/?\>([^\<\n][^\<]*)\<
FORMAT = $1::$2

The extraction worked beautifully. So next I went to go add it to my clustered environment. I have the XML data being indexed to an index called app_dmwt. I also created an app called 'DMWT'. I created similar additions to the props.conf and transforms.conf, but this time put it in $SPLUNK_ROOT/etc/shcluster/app/DMWT/default . Note that the difference here is that instead of doing an add datasource from the UI, I have a forwarder monitoring 3 files each as their own sourcetype.

props.conf

[ dmwt:delete ]
REPORT-xmlkv = xmlkv-alternative

[ dmwt:insert ]
REPORT-xmlkv = xmlkv-alternative

[ dmwt:update ]
REPORT-xmlkv = xmlkv-alternative

transforms.conf

[xmlkv-alternative]
REGEX = \<(\w+[^\n\/\>]+)\/?\>([^\<\n][^\<]*)\<
FORMAT = $1::$2

I then did a bundle push from the deployer, but for some reason, the transform isn't taking effect.

Any thoughts to what might be going on?

0 Karma

skalliger
SplunkTrust
SplunkTrust

Yea, the transforming is done on the indexers, not the search heads.
But I am also wondering why you tried to deploy the changes in the "default" directory. This is only done when you deploy the first time. When you update the configuration, you usually store it under \local\ and don't change \default\ files..

Skalli

0 Karma

paimonsoror
Builder

Interestingly enough, it didn't even work on the indexers when i pushed the bundle. I was messing around and came back to pushing them to the forwarders. Once I did an

index=whatever | extract reload=t 

it updated everything....

Also, the reason i am working in default is because i am developing this app in a lower environment for my users to test.

0 Karma

paimonsoror
Builder

I may have found my problem. I was applying my props on the search head cluster, not on the indexer. Correcting that now to see if it resolves the problem!

0 Karma

Afef
Communicator

did you solve the problem ?

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...