Getting Data In
Highlighted

Field transformation based on source

Hi All,

We log data from devices belonging to different customers, they are written to our syslog server in files named /data/log/CUSTOMER/site/router1.log, for example. I wanted to have a search-time field called customer with the value CUSTOMER, taken from the source filename.

I did this via the web GUI, under Settings/Fields/Field Transformation, this is what was written in $SPLUNK_HOME/etc/apps/search/local/transforms.conf:

[get_customer]
FORMAT = customer::$1
REGEX = \/data\/log\/(.*)\/site.*
SOURCE_KEY = MetaData:Source

Unfortunately nothing happens, I get no field named customer when I search. From what I can tell, the regex is correct. I also tried just "source" as SOURCE_KEY but nothing changed. Is anything wrong with my transform ?

I am also not sure how this transform is applied, is it run against log messages arriving via all indexes ?

As additional info, we are running splunk on a separate server (so basically the indexer) and we use a light forwarder on our syslog server. The transform above is done on the indexer.

Thanks,
Stefan

Highlighted

Re: Field transformation based on source

SplunkTrust
SplunkTrust

Did you specify a REPORT-foo = get_customer entry in props.conf for that sourcetype?

0 Karma
Highlighted

Re: Field transformation based on source

Path Finder

In props.conf:

[your_sourcetype]
EXTRACT = \/data\/log\/(?<customer>\w+)\/site.* in source

No need to touch transforms.conf for this.

View solution in original post

Highlighted

Re: Field transformation based on source

I actually tried to do this via GUI (Fields/Field Extraction) but when I chose "source" for "Apply To", it also wanted me to specify which source. I obviously didn't want to restrict it to one particular source, didn't know what to put in there. Was I doing something wrong ?

In any case, this worked. I just had to restart splunk. Is there no way around restarting ?

0 Karma
Highlighted

Re: Field transformation based on source

Path Finder

You can do this in SPL itself:

| extract reload=t

0 Karma
Highlighted

Re: Field transformation based on source

SplunkTrust
SplunkTrust

You could also just add in source after the end of the field extraction regex when editing it through the UI.

Highlighted

Re: Field transformation based on source

Yes, I do have that too.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.