Getting Data In

Has anyone had props.conf and transforms.conf to work properly for Bluecoat 6.7.4.3 log formatting?

BenzSann
Splunk Employee
Splunk Employee

We use Splunk Bluecoat-TA but many fields are missing.    They have not changed log format.  But it seems they changed log format.   

The sample log is 

 

Sep 18 15:25:44 2020-09-18 07:25:41 4115 10.X.X.X 200 TCP_TUNNELED 6569 1787 CONNECT tcp sy.abc.net 443 / - abc2323 LOCAL1\ACC124 - 172.X.X.X - - “Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36” OBSERVED "MyShopWhitelist;LivePe;ProjectA_URL;docsApproveURLs;ABC_GBB_GGG;Business/Econony” - 172.X.X.X 0#015

Labels (3)
0 Karma

dan_mizuho
New Member

yea I found the TA and associated regex overly complicated. Plus BC added a few fields in main log in v6.7. I ended up re-writing the regex and it works fine for us. It is based on the default 'main' log. We did end up create a explicit log format on the proxy and just copied the main format to it. This ensures that the log format wont change after upgrades

[auto_kv_for_bluecoat_v6_7_x]
Regex = ^(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\".*?\"|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\".*?\"|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\".*?\"|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\".*?\"|\-)\s+(\".*?\"|\-)\s+(\".*?\"|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)\s+(\S+|\-)$
Format =
date::$1 time::$2 time_taken::$3 c_ip::$4 cs_username::$5 cs_auth_group::$6 s_supplier_name::$7 s_supplier_ip::$8 s_supplier_country::$9 s_supplier_failures::$10 x_exception_id::$11 sc_filter_result::$12 cs_categories::$13 cs_Referer::$14 sc_status::$15 s_action::$16 cs_method::$17 rs_Content_Type::$18 cs_uri_scheme::$19 cs_host::$20 cs_uri_port::$21 cs_uri_path::$22 cs_uri_query::$23 cs_uri_extension::$24 cs_User_Agent::$25 s_ip::$26 sc_bytes::$27 cs_bytes::$28 x_virus_id::$29 x_bluecoat_application_name::$30 x_bluecoat_application_operation::$31 x_bluecoat_application_groups::$32 cs_threat_risk::$33 x_bluecoat_transaction_uuid::$34 x_icap_reqmod_header::$35 x_icap_respmod_header::$36

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...