Splunk Search

Transforms are working on a local Splunk instance, but why are fields not extracted correctly when deployed to my search head cluster?

paimonsoror
Builder

Having a strange issue. I am trying to set up a transform to automatically extract key/value pairs from a non standard XML file. I have a local Splunk instance where I did this development. Here is what I did:

  1. Did a Add New of a data source
  2. Selected a local copy of the XML file
  3. The data was read once (not set to monitor the file)
  4. Data was indexed into an index called 'app_test'
  5. Data is in the Search app

I then went ahead and updated my props.conf and transforms.conf in my $SPLUNK_ROOT/etc/apps/search/local

props.conf

[dmwt:xml]
REPORT-myXmlClassName = xmlTransform

transforms.conf

[xmlTransform]
REGEX = \<(\w+[^\n\/\>]+)\/?\>([^\<\n][^\<]*)\<
FORMAT = $1::$2

The extraction worked beautifully. So next I went to go add it to my clustered environment. I have the XML data being indexed to an index called app_dmwt. I also created an app called 'DMWT'. I created similar additions to the props.conf and transforms.conf, but this time put it in $SPLUNK_ROOT/etc/shcluster/app/DMWT/default . Note that the difference here is that instead of doing an add datasource from the UI, I have a forwarder monitoring 3 files each as their own sourcetype.

props.conf

[ dmwt:delete ]
REPORT-xmlkv = xmlkv-alternative

[ dmwt:insert ]
REPORT-xmlkv = xmlkv-alternative

[ dmwt:update ]
REPORT-xmlkv = xmlkv-alternative

transforms.conf

[xmlkv-alternative]
REGEX = \<(\w+[^\n\/\>]+)\/?\>([^\<\n][^\<]*)\<
FORMAT = $1::$2

I then did a bundle push from the deployer, but for some reason, the transform isn't taking effect.

Any thoughts to what might be going on?

0 Karma

skalliger
Motivator

Yea, the transforming is done on the indexers, not the search heads.
But I am also wondering why you tried to deploy the changes in the "default" directory. This is only done when you deploy the first time. When you update the configuration, you usually store it under \local\ and don't change \default\ files..

Skalli

0 Karma

paimonsoror
Builder

Interestingly enough, it didn't even work on the indexers when i pushed the bundle. I was messing around and came back to pushing them to the forwarders. Once I did an

index=whatever | extract reload=t 

it updated everything....

Also, the reason i am working in default is because i am developing this app in a lower environment for my users to test.

0 Karma

paimonsoror
Builder

I may have found my problem. I was applying my props on the search head cluster, not on the indexer. Correcting that now to see if it resolves the problem!

0 Karma

Afef
Communicator

did you solve the problem ?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...