Getting Data In

How to set Host from an extracted field?

_smp_
Builder

I have some BlueCoat proxy log files being indexed by Splunk. The indexer and Search Head both have the BlueCoat add-on which extracts a bunch of different fields. When the events are indexed, the Host is set to the name of the Universal Forwarder. I would like to change the Host value in these events to the value of an extracted field named x_bluecoat_proxy_primary_address. After reading the doc on props and transforms, here's what I've come up with on my indexer:

props.conf

[bluecoat:proxysg:access:file]
TRANSFORMS-proxyhost = x_bluecoat_proxy_primary_address

transforms.conf:

[x_bluecoat_proxy_primary_address]
SOURCE_KEY = field:x_bluecoat_proxy_primary_address
REGEX = (.*)
DEST_KEY = MetaData:Host

This is not working, obviously. Can someone clue me into what I'm doing wrong here? Thanks.

0 Karma

woodcock
Esteemed Legend

Change this:

 SOURCE_KEY = field:x_bluecoat_proxy_primary_address

To this:

 SOURCE_KEY = x_bluecoat_proxy_primary_address

Then deploy to the indexers, restart all Splunk instances there, and check only newly arriving data (old events will stay broken).

0 Karma

_smp_
Builder

I got this error starting Splunk:

Undocumented key used in transforms.conf; stanza='x_bluecoat_proxy_primary_address' setting='SOURCE_KEY' key='x_bluecoat_proxy_primary_address'

0 Karma

woodcock
Esteemed Legend

I have never seen that so I cannot help you there.

0 Karma

woodcock
Esteemed Legend

Your SOURCE_KEY must be wrong; surely you do not have a field named field:x_bluecoat_proxy_primary_address. What it the field that you need to use and, more importantly, how is this field created?

0 Karma

_smp_
Builder

Thank you for your response. I really don't understand Splunk well enough to create fields, and am having a very difficult time understanding these config files. All the fields I see were created by the BlueCoat TA that I have installed. The field that has value I want use for Host is named x_bluecoat_proxy_primary_address. But I'm not sure how to describe to you how it is created - I wish I could.

0 Karma

somesoni2
Revered Legend

I believe you'd need to provide REGEX which can extract the value of x_bluecoat_proxy_primary_address from your raw events. Could you post some sample raw data?

0 Karma

_smp_
Builder

Thanks for your response. Here is a raw event:
2016-07-07 19:39:53 31 172.20.176.110 200 TCP_NC_MISS 202 3848 GET http sync.tidaltv.com 80 /genericusersync.ashx ?dpid=1205/ user521 - - sync.tidaltv.com - "Mozilla/5.0 (Windows NT 6.1; rv:47.0) Gecko/20100101 Firefox/47.0" - OBSERVED "Web Ads/Analytics" - 192.168.15.21 SG-HTTP-Service

The BlueCoat TA is extracting 192.168.15.21 into a field named x_bluecoat_proxy_primary_address. This is the value I would like to use for Host. And when I read the doc for transforms, I saw this about SOURCE_KEY:

If starts with "field:" or "fields:" the meaning is changed.
Instead of looking up a KEY, it instead looks up an already indexed field.
For example, if a CSV field name "price" was indexed then
"SOURCE_KEY = field:price" causes the REGEX to match against the contents
of that field. It's also possible to list multiple fields here with

That's why I tried what I did. But I am really, really confused and frustrated by these config files, so I really have no idea if what I want to do is even possible.

0 Karma

somesoni2
Revered Legend

Well, it does say that it works for already indexed field. The x_bluecoat_proxy_primary_address could very well be a search time extracted field, thus it didn't work. There should be a props.conf entry which is doing this search time field extraction. I would find that out and use the regular expression from that props.conf and use it in your Host Override transform.conf.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...