Getting Data In

How to set Host from an extracted field?

_smp_
Builder

I have some BlueCoat proxy log files being indexed by Splunk. The indexer and Search Head both have the BlueCoat add-on which extracts a bunch of different fields. When the events are indexed, the Host is set to the name of the Universal Forwarder. I would like to change the Host value in these events to the value of an extracted field named x_bluecoat_proxy_primary_address. After reading the doc on props and transforms, here's what I've come up with on my indexer:

props.conf

[bluecoat:proxysg:access:file]
TRANSFORMS-proxyhost = x_bluecoat_proxy_primary_address

transforms.conf:

[x_bluecoat_proxy_primary_address]
SOURCE_KEY = field:x_bluecoat_proxy_primary_address
REGEX = (.*)
DEST_KEY = MetaData:Host

This is not working, obviously. Can someone clue me into what I'm doing wrong here? Thanks.

0 Karma

woodcock
Esteemed Legend

Change this:

 SOURCE_KEY = field:x_bluecoat_proxy_primary_address

To this:

 SOURCE_KEY = x_bluecoat_proxy_primary_address

Then deploy to the indexers, restart all Splunk instances there, and check only newly arriving data (old events will stay broken).

0 Karma

_smp_
Builder

I got this error starting Splunk:

Undocumented key used in transforms.conf; stanza='x_bluecoat_proxy_primary_address' setting='SOURCE_KEY' key='x_bluecoat_proxy_primary_address'

0 Karma

woodcock
Esteemed Legend

I have never seen that so I cannot help you there.

0 Karma

woodcock
Esteemed Legend

Your SOURCE_KEY must be wrong; surely you do not have a field named field:x_bluecoat_proxy_primary_address. What it the field that you need to use and, more importantly, how is this field created?

0 Karma

_smp_
Builder

Thank you for your response. I really don't understand Splunk well enough to create fields, and am having a very difficult time understanding these config files. All the fields I see were created by the BlueCoat TA that I have installed. The field that has value I want use for Host is named x_bluecoat_proxy_primary_address. But I'm not sure how to describe to you how it is created - I wish I could.

0 Karma

somesoni2
Revered Legend

I believe you'd need to provide REGEX which can extract the value of x_bluecoat_proxy_primary_address from your raw events. Could you post some sample raw data?

0 Karma

_smp_
Builder

Thanks for your response. Here is a raw event:
2016-07-07 19:39:53 31 172.20.176.110 200 TCP_NC_MISS 202 3848 GET http sync.tidaltv.com 80 /genericusersync.ashx ?dpid=1205/ user521 - - sync.tidaltv.com - "Mozilla/5.0 (Windows NT 6.1; rv:47.0) Gecko/20100101 Firefox/47.0" - OBSERVED "Web Ads/Analytics" - 192.168.15.21 SG-HTTP-Service

The BlueCoat TA is extracting 192.168.15.21 into a field named x_bluecoat_proxy_primary_address. This is the value I would like to use for Host. And when I read the doc for transforms, I saw this about SOURCE_KEY:

If starts with "field:" or "fields:" the meaning is changed.
Instead of looking up a KEY, it instead looks up an already indexed field.
For example, if a CSV field name "price" was indexed then
"SOURCE_KEY = field:price" causes the REGEX to match against the contents
of that field. It's also possible to list multiple fields here with

That's why I tried what I did. But I am really, really confused and frustrated by these config files, so I really have no idea if what I want to do is even possible.

0 Karma

somesoni2
Revered Legend

Well, it does say that it works for already indexed field. The x_bluecoat_proxy_primary_address could very well be a search time extracted field, thus it didn't work. There should be a props.conf entry which is doing this search time field extraction. I would find that out and use the regular expression from that props.conf and use it in your Host Override transform.conf.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...