Getting Data In

Universal Forwarder props.conf and transforms.conf settings

kniloo
Explorer

I am trying to get the output from a python script to indexer. So i added transforms.conf and props.conf under C:\Program Files\SplunkUniversalForwarder\etc\system\local

transforms.conf
[myexternaltable]
REGEX = (.)
external_cmd = addnum.py $1
DEST_KEY = queue
FORMAT = indexQueue

props.conf
[sitescope_daily2_log]
TRANSFORMS-runscript=myexternaltable

But its not working, can anyone please help me with correct settings needs to be done on UF.

Thanks,
Niloo

0 Karma

didatams
New Member

Just an idea.. but if you want to input data from a script.

You can put the script in the bin directory of an app, refer it in the inputs.conf.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The props.conf and transforms.conf files should be installed on the indexer(s), not the UF.

---
If this reply helps you, Karma would be appreciated.
0 Karma

kniloo
Explorer

I have moved props.conf and transforms.conf to indexer ,but still its not working.
transforms.conf
[myexternaltable]
REGEX = (.)
external_cmd = testscript.py $1
fields_list = log
DEST_KEY = queue
FORMAT = indexQueue
WRITE_META = true

props.conf
[sitescope_daily2_log]
TRANSFORMS-runscript=myexternaltable

0 Karma

mikev
Path Finder

I know this is an older post but I believe that you should be using DEST_KEY per the documentation:

DEST_KEY = <KEY>
* NOTE: This setting is only valid for index-time field extractions.
* Specifies where Splunk software stores the expanded FORMAT results in
  accordance with the REGEX match.
* Required for index-time field extractions where WRITE_META = false or is
  not set.
* For index-time extractions, DEST_KEY can be set to a number of values
  mentioned in the KEYS section at the bottom of this file.
  * If DEST_KEY = _meta (not recommended) you should also add $0 to the
    start of your FORMAT setting.  $0 represents the DEST_KEY value before
    Splunk software performs the REGEX (in other words, _meta).
    * The $0 value is in no way derived *from* the REGEX match. (It
      does not represent a captured group.)
* KEY names are case-sensitive, and should be used exactly as they appear in
  the KEYs list at the bottom of this file. (For example, you would say
  DEST_KEY = MetaData:Host, *not* DEST_KEY = metadata:host .)

0 Karma

kniloo
Explorer

Thanks for the response.
But if we required to parse some data at UF (before sending to indexer) can't we use transforms.conf and props.conf on UF ?

if yes ,can you share the steps as well.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is the source of this requirement? Just because it is required does not make it possible (or correct).
The filtering you are trying to do is performed by indexers or heavy forwarders, not universal forwarders. Consider replacing the UF with a HF.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...