Field transformation based on source

[stefan_radovano]

Hi All,

We log data from devices belonging to different customers, they are written to our syslog server in files named /data/log/CUSTOMER/site/router1.log, for example. I wanted to have a search-time field called customer with the value CUSTOMER, taken from the source filename.

I did this via the web GUI, under Settings/Fields/Field Transformation, this is what was written in $SPLUNK_HOME/etc/apps/search/local/transforms.conf:

[get_customer]
FORMAT = customer::$1
REGEX = \/data\/log\/(.*)\/site.*
SOURCE_KEY = MetaData:Source

Unfortunately nothing happens, I get no field named customer when I search. From what I can tell, the regex is correct. I also tried just "source" as SOURCE_KEY but nothing changed. Is anything wrong with my transform ?

I am also not sure how this transform is applied, is it run against log messages arriving via all indexes ?

As additional info, we are running splunk on a separate server (so basically the indexer) and we use a light forwarder on our syslog server. The transform above is done on the indexer.

Thanks,
Stefan

[stefan_radovano]

Yes, I do have that too.

[emechler_splunk · Accepted solution]

In props.conf:

[your_sourcetype]
EXTRACT = \/data\/log\/(?<customer>\w+)\/site.* in source

No need to touch transforms.conf for this.

[martin_mueller]

You could also just add in source after the end of the field extraction regex when editing it through the UI.

[emechler_splunk]

You can do this in SPL itself:

| extract reload=t

[stefan_radovano]

I actually tried to do this via GUI (Fields/Field Extraction) but when I chose "source" for "Apply To", it also wanted me to specify which source. I obviously didn't want to restrict it to one particular source, didn't know what to put in there. Was I doing something wrong ?

In any case, this worked. I just had to restart splunk. Is there no way around restarting ?

[martin_mueller]

Did you specify a REPORT-foo = get_customer entry in props.conf for that sourcetype?