topic Re: How to set Host from an extracted field? in Getting Data In

How to set Host from an extracted field?

_smp_ — Tue, 29 Sep 2020 10:10:00 GMT

I have some BlueCoat proxy log files being indexed by Splunk. The indexer and Search Head both have the BlueCoat add-on which extracts a bunch of different fields. When the events are indexed, the Host is set to the name of the Universal Forwarder. I would like to change the Host value in these events to the value of an extracted field named x_bluecoat_proxy_primary_address. After reading the doc on props and transforms, here's what I've come up with on my indexer:

props.conf

[bluecoat:proxysg:access:file]
TRANSFORMS-proxyhost = x_bluecoat_proxy_primary_address

transforms.conf:

[x_bluecoat_proxy_primary_address]
SOURCE_KEY = field:x_bluecoat_proxy_primary_address
REGEX = (.*)
DEST_KEY = MetaData:Host

This is not working, obviously. Can someone clue me into what I'm doing wrong here? Thanks.

Re: How to set Host from an extracted field?

somesoni2 — Tue, 29 Sep 2020 10:09:23 GMT

I believe you'd need to provide REGEX which can extract the value of x_bluecoat_proxy_primary_address from your raw events. Could you post some sample raw data?

Re: How to set Host from an extracted field?

woodcock — Thu, 07 Jul 2016 19:41:10 GMT

Your SOURCE_KEY must be wrong; surely you do not have a field named field:x_bluecoat_proxy_primary_address. What it the field that you need to use and, more importantly, how is this field created?

Re: How to set Host from an extracted field?

_smp_ — Tue, 29 Sep 2020 10:10:03 GMT

Thanks for your response. Here is a raw event:
2016-07-07 19:39:53 31 172.20.176.110 200 TCP_NC_MISS 202 3848 GET http sync.tidaltv.com 80 /genericusersync.ashx ?dpid=1205/ user521 - - sync.tidaltv.com - "Mozilla/5.0 (Windows NT 6.1; rv:47.0) Gecko/20100101 Firefox/47.0" - OBSERVED "Web Ads/Analytics" - 192.168.15.21 SG-HTTP-Service

The BlueCoat TA is extracting 192.168.15.21 into a field named x_bluecoat_proxy_primary_address. This is the value I would like to use for Host. And when I read the doc for transforms, I saw this about SOURCE_KEY:

If starts with "field:" or "fields:" the meaning is changed.
Instead of looking up a KEY, it instead looks up an already indexed field.
For example, if a CSV field name "price" was indexed then
"SOURCE_KEY = field:price" causes the REGEX to match against the contents
of that field. It's also possible to list multiple fields here with

That's why I tried what I did. But I am really, really confused and frustrated by these config files, so I really have no idea if what I want to do is even possible.

Re: How to set Host from an extracted field?

_smp_ — Tue, 29 Sep 2020 10:10:06 GMT

Thank you for your response. I really don't understand Splunk well enough to create fields, and am having a very difficult time understanding these config files. All the fields I see were created by the BlueCoat TA that I have installed. The field that has value I want use for Host is named x_bluecoat_proxy_primary_address. But I'm not sure how to describe to you how it is created - I wish I could.

Re: How to set Host from an extracted field?

somesoni2 — Tue, 29 Sep 2020 10:09:28 GMT

Well, it does say that it works for already indexed field. The x_bluecoat_proxy_primary_address could very well be a search time extracted field, thus it didn't work. There should be a props.conf entry which is doing this search time field extraction. I would find that out and use the regular expression from that props.conf and use it in your Host Override transform.conf.

Re: How to set Host from an extracted field?

woodcock — Thu, 07 Jul 2016 20:06:49 GMT

Change this:

 SOURCE_KEY = field:x_bluecoat_proxy_primary_address

To this:

 SOURCE_KEY = x_bluecoat_proxy_primary_address

Then deploy to the indexers, restart all Splunk instances there, and check only newly arriving data (old events will stay broken).

Re: How to set Host from an extracted field?

_smp_ — Tue, 29 Sep 2020 10:09:31 GMT

I got this error starting Splunk:

Undocumented key used in transforms.conf; stanza='x_bluecoat_proxy_primary_address' setting='SOURCE_KEY' key='x_bluecoat_proxy_primary_address'

Re: How to set Host from an extracted field?

woodcock — Thu, 07 Jul 2016 20:28:36 GMT

I have never seen that so I cannot help you there.