Getting Data In

Splunk Add-on for Blue Coat ProxySG: Has anyone have props.and transforms to work properly for Bluecoat 6.7.3.5 log formatting?

mshakeb
Loves-to-Learn Lots

Hi Experts
Splunk Add-on for Blue Coat ProxySG: Has anyone gotten the props and transforms to work properly for Bluecoat 6.7.3.5 formatting, I have applied the 6.6.x.x on props and transforms but could not see the field extraction Properly. Many fields are missing. please advice

Tags (1)
0 Karma

kiyohito
Engager

I had the same problem for bluecoat:proxysg:access:syslog. It's because Splunk Add-on for Blue Coat ProxySG Version 3.5.0 does not catch up with SGOS 6.7.

I'm not sure but found two issues for Add-on:

  1. Regular expression error for cs-categories: "Technology/Internet;Web Ads/Analytics" was splited into "Tech....Web" and "Ads/Analytics".
  2. Missing x-bluecoat-application-groups field

Solution is following. So far it works for me(Splunk 7.2.6, SGOS 6.7.2.1).

Add to transforms.conf


[auto_kv_for_bluecoat_v6_7_x]
REGEX = ^(?:"([^"]+)"|([^"]\S*))\s+(?:"(\d{1,2}:\d{1,2}:\d{1,2})"|(\d{1,2}:\d{1,2}:\d{1,2}))\s+(?:"(\d+)"|(\d+))\s+(?:"(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})"|(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*$
FORMAT = date::$1 date::$2 time::$3 time::$4 time_taken::$5 time_taken::$6 c_ip::$7 c_ip::$8 cs_username::$9 cs_username::$10 cs_auth_group::$11 cs_auth_group::$12 s_supplier_name::$13 s_supplier_name::$14 s_supplier_ip::$15 s_supplier_ip::$16 s_supplier_country::$17 s_supplier_country::$18 s_supplier_failures::$19 s_supplier_failures::$20 x_exception_id::$21 x_exception_id::$22 sc_filter_result::$23 sc_filter_result::$24 cs_categories::$25 cs_categories::$26 cs_Referer::$27 cs_Referer::$28 sc_status::$29 sc_status::$30 s_action::$31 s_action::$32 cs_method::$33 cs_method::$34 rs_Content_Type::$35 rs_Content_Type::$36 cs_uri_scheme::$37 cs_uri_scheme::$38 cs_host::$39 cs_host::$40 cs_uri_port::$41 cs_uri_port::$42 cs_uri_path::$43 cs_uri_path::$44 cs_uri_query::$45 cs_uri_query::$46 cs_uri_extension::$47 cs_uri_extension::$48 cs_User_Agent::$49 cs_User_Agent::$50 s_ip::$51 s_ip::$52 sc_bytes::$53 sc_bytes::$54 cs_bytes::$55 cs_bytes::$56 x_virus_id::$57 x_virus_id::$58 x_bluecoat_application_name::$59 x_bluecoat_application_name::$60 x_bluecoat_application_operation::$61 x_bluecoat_application_operation::$62 x-bluecoat-application-groups::$63 x-bluecoat-application-groups::$64 cs_threat_risk::$65 cs_threat_risk::$66 x_bluecoat_transaction_uuid::$67 x_bluecoat_transaction_uuid::$68 x_icap_reqmod_header::$69 x_icap_reqmod_header::$70 x_icap_respmod_header::$71 x_icap_respmod_header::$72

Add to props.conf

# Supports Bluecoat 6.7 field format
REPORT-auto_kv_for_bluecoat_v6_7_x = auto_kv_for_bluecoat_v6_7_x

I hope it help someone and Add-on be update.

joshuaonsecurit
New Member

Is it possible to share your Bluecoat log format? I'm struggling to get the extract working.

0 Karma

bsanta201
New Member

Hello, Does this regex work with IVP6? if not does anyone have an updated regex that will work? I am not having any luck with IVP regex

0 Karma

bsanta201
New Member

Here is what I have (?:"([^"]+)"|([^"]\S*))\s+(?:"(\d{1,2}:\d{1,2}:\d{1,2})"|(\d{1,2}:\d{1,2}:\d{1,2}))\s+(?:"(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})"|(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}))\s+(?:([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))|(?:[0-9]{1,3}.){3}[0-9]{1,3}\b))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*

0 Karma

kiyohito
Engager

Hi,

Log: main
Log format: bcreportermain_v1


date time time-taken c-ip cs-username cs-auth-group s-supplier-name s-supplier-ip s-supplier-country s-supplier-failures x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action cs-method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-bluecoat-application-name x-bluecoat-application-operation x-bluecoat-application-groups cs-threat-risk x-bluecoat-transaction-uuid x-icap-reqmod-header(X-ICAP-Metadata) x-icap-respmod-header(X-ICAP-Metadata)

Ref: onfigure logging in your Blue Coat ProxySG appliance for the Splunk Add-on for Symantec Blue Co...

0 Karma

sdchakraborty
Contributor

Hi,

Can you provide a sample log.

Sid

0 Karma

mshakeb
Loves-to-Learn Lots

please find the logs :

Apr 3 03:14:48 133.22.00.00 2019-04-03 07:14:48 74 10.130.122.151 Jastaniahfm aljfs\Group%20Internet%20Basic www.google.com 172.217.19.132 None - - OBSERVED "CRM-Access;Search Engines/Portals" https://www.google.com/ 204 TCP_NC_MISS POST text/html https www.google.com 443 /gen_204 ?atyp=csi&ei=-VekXNPyG66PlwT_hZDYDg&s=newtab&t=all&action=update&conn=onchange&ima=1&ime=1&imeb=0&imeo=0&wh=618&scp=0&net=dl.1650,ect.4g,rtt.150&mem=ujhs.10,tjhs.11,jhsl.2330,dm.8&sto=&sys=hc.4&rt=xhr.342,aft.4,cst.0,dnst.0,rqst.70,rspt.21,rqstt.21,unt.21,cstt.21,dit.159&zx=1554275688237 - "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" 133.22.00.001 306 1429 - "none" "none" "none" 1 7ee0b471858c027d-00000000228366b2-000000005ca45d68 - -

Apr 3 03:14:48 103.33.33.3 2019-04-03 07:14:48 68 10.0.51.51 Karimykd aljfs\Group%20Internet%20No%20Restriction ctldl.windowsupdate.com 10.111.000.241 None - - OBSERVED "White list website;Non-Viewable/Infrastructure" - 304 TCP_MISS GET application/vnd.ms-cab-compressed http ctldl.windowsupdate.com 80 /msdownload/update/v3/static/trustedr/en/disallowedcertstl.cab ?5fae3ccb4e56dcf8 cab "Microsoft-CryptoAPI/6.1" 10.22.22.22 386 322 - "Microsoft Update" "Update Software" "none" 1 7ee0b471858c027d-00000000228366af-000000005ca45d68 - "{ %22expect_sandbox%22: false }"

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...