All Apps and Add-ons

how can we add new fields to proxy logs

CONSORP
Loves-to-Learn Lots

Hi,
Currently i'm using SplunkAppForBlueCoatProxySG app which is working as expected. My user wanted to add few more additional fields for proxy logs. We tested by adding only one new field at the END and its failing to extract existing fields in splunk. Anyone have any suggestions how to add new fields for proxy logs.

Thanks in advance.

@SplunkAppForBlueCoatProxySG

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @ CONSORP,
the problem is that SplunkAppForBlueCoatProxySG works using summary indexes (because usually there are too many events to directly display in dashboards and dashboards are too slow).
This means that you cannot limit to extract the new field but you have to do some other jobs:

  • extract the new field as usual from the logs (e.g. using regex);
  • modify the scheduled search that updates summary index adding the new field;
  • modify all searches in dashboards panels.

Ciao.
Giuseppe

0 Karma

FrankVl
Ultra Champion

If you are ingesting the bluecoat logs over syslog, you will need to customize the REPORT field extractions defined in props/transforms of the TA. Those use a regex that covers the entire event and map all bits and pieces of it to specific fields. If you add a field, the regex no longer matches, hence (as you discovered) the extractions break.

Customizing this can be done by creating a copy (in local/props.conf and local/transforms.conf) of the existing config, give it a new name (e.g. auto_kv_for_bluecoat_CONSORP) and then edit the REGEX and FORMAT to extend it with the additional field(s).

If you're ingesting the logs as W3C (ELLF) formatted files, you'd have to disable the INDEXED_EXTRACTIONS = w3c and write your own field extraction config for it.

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...