We log data from devices belonging to different customers, they are written to our syslog server in files named /data/log/CUSTOMER/site/router1.log, for example. I wanted to have a search-time field called customer with the value CUSTOMER, taken from the source filename.
I did this via the web GUI, under Settings/Fields/Field Transformation, this is what was written in $SPLUNK_HOME/etc/apps/search/local/transforms.conf:
[get_customer] FORMAT = customer::$1 REGEX = \/data\/log\/(.*)\/site.* SOURCE_KEY = MetaData:Source
Unfortunately nothing happens, I get no field named customer when I search. From what I can tell, the regex is correct. I also tried just "source" as SOURCE_KEY but nothing changed. Is anything wrong with my transform ?
I am also not sure how this transform is applied, is it run against log messages arriving via all indexes ?
As additional info, we are running splunk on a separate server (so basically the indexer) and we use a light forwarder on our syslog server. The transform above is done on the indexer.
[your_sourcetype] EXTRACT = \/data\/log\/(?<customer>\w+)\/site.* in source
No need to touch transforms.conf for this.
I actually tried to do this via GUI (Fields/Field Extraction) but when I chose "source" for "Apply To", it also wanted me to specify which source. I obviously didn't want to restrict it to one particular source, didn't know what to put in there. Was I doing something wrong ?
In any case, this worked. I just had to restart splunk. Is there no way around restarting ?
You could also just add
in source after the end of the field extraction regex when editing it through the UI.