Hi Experts
Splunk Add-on for Blue Coat ProxySG: Has anyone gotten the props and transforms to work properly for Bluecoat 6.7.3.5 formatting, I have applied the 6.6.x.x on props and transforms but could not see the field extraction Properly. Many fields are missing. please advice
I had the same problem for bluecoat:proxysg:access:syslog. It's because Splunk Add-on for Blue Coat ProxySG Version 3.5.0 does not catch up with SGOS 6.7.
I'm not sure but found two issues for Add-on:
Solution is following. So far it works for me(Splunk 7.2.6, SGOS 6.7.2.1).
Add to transforms.conf
[auto_kv_for_bluecoat_v6_7_x]
REGEX = ^(?:"([^"]+)"|([^"]\S*))\s+(?:"(\d{1,2}:\d{1,2}:\d{1,2})"|(\d{1,2}:\d{1,2}:\d{1,2}))\s+(?:"(\d+)"|(\d+))\s+(?:"(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})"|(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*$
FORMAT = date::$1 date::$2 time::$3 time::$4 time_taken::$5 time_taken::$6 c_ip::$7 c_ip::$8 cs_username::$9 cs_username::$10 cs_auth_group::$11 cs_auth_group::$12 s_supplier_name::$13 s_supplier_name::$14 s_supplier_ip::$15 s_supplier_ip::$16 s_supplier_country::$17 s_supplier_country::$18 s_supplier_failures::$19 s_supplier_failures::$20 x_exception_id::$21 x_exception_id::$22 sc_filter_result::$23 sc_filter_result::$24 cs_categories::$25 cs_categories::$26 cs_Referer::$27 cs_Referer::$28 sc_status::$29 sc_status::$30 s_action::$31 s_action::$32 cs_method::$33 cs_method::$34 rs_Content_Type::$35 rs_Content_Type::$36 cs_uri_scheme::$37 cs_uri_scheme::$38 cs_host::$39 cs_host::$40 cs_uri_port::$41 cs_uri_port::$42 cs_uri_path::$43 cs_uri_path::$44 cs_uri_query::$45 cs_uri_query::$46 cs_uri_extension::$47 cs_uri_extension::$48 cs_User_Agent::$49 cs_User_Agent::$50 s_ip::$51 s_ip::$52 sc_bytes::$53 sc_bytes::$54 cs_bytes::$55 cs_bytes::$56 x_virus_id::$57 x_virus_id::$58 x_bluecoat_application_name::$59 x_bluecoat_application_name::$60 x_bluecoat_application_operation::$61 x_bluecoat_application_operation::$62 x-bluecoat-application-groups::$63 x-bluecoat-application-groups::$64 cs_threat_risk::$65 cs_threat_risk::$66 x_bluecoat_transaction_uuid::$67 x_bluecoat_transaction_uuid::$68 x_icap_reqmod_header::$69 x_icap_reqmod_header::$70 x_icap_respmod_header::$71 x_icap_respmod_header::$72
Add to props.conf
# Supports Bluecoat 6.7 field format REPORT-auto_kv_for_bluecoat_v6_7_x = auto_kv_for_bluecoat_v6_7_x
I hope it help someone and Add-on be update.
Is it possible to share your Bluecoat log format? I'm struggling to get the extract working.
Hello, Does this regex work with IVP6? if not does anyone have an updated regex that will work? I am not having any luck with IVP regex
Here is what I have (?:"([^"]+)"|([^"]\S*))\s+(?:"(\d{1,2}:\d{1,2}:\d{1,2})"|(\d{1,2}:\d{1,2}:\d{1,2}))\s+(?:"(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})"|(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}))\s+(?:([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))|(?:[0-9]{1,3}.){3}[0-9]{1,3}\b))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s+(?:"([^"]+)"|([^"]\S*))\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*(?:"([^"]+)"|([^"]\S*))?\s*
Hi,
Log: main
Log format: bcreportermain_v1
date time time-taken c-ip cs-username cs-auth-group s-supplier-name s-supplier-ip s-supplier-country s-supplier-failures x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action cs-method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-bluecoat-application-name x-bluecoat-application-operation x-bluecoat-application-groups cs-threat-risk x-bluecoat-transaction-uuid x-icap-reqmod-header(X-ICAP-Metadata) x-icap-respmod-header(X-ICAP-Metadata)
Hi,
Can you provide a sample log.
Sid
please find the logs :
Apr 3 03:14:48 133.22.00.00 2019-04-03 07:14:48 74 10.130.122.151 Jastaniahfm aljfs\Group%20Internet%20Basic www.google.com 172.217.19.132 None - - OBSERVED "CRM-Access;Search Engines/Portals" https://www.google.com/ 204 TCP_NC_MISS POST text/html https www.google.com 443 /gen_204 ?atyp=csi&ei=-VekXNPyG66PlwT_hZDYDg&s=newtab&t=all&action=update&conn=onchange&ima=1&ime=1&imeb=0&imeo=0&wh=618&scp=0&net=dl.1650,ect.4g,rtt.150&mem=ujhs.10,tjhs.11,jhsl.2330,dm.8&sto=&sys=hc.4&rt=xhr.342,aft.4,cst.0,dnst.0,rqst.70,rspt.21,rqstt.21,unt.21,cstt.21,dit.159&zx=1554275688237 - "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" 133.22.00.001 306 1429 - "none" "none" "none" 1 7ee0b471858c027d-00000000228366b2-000000005ca45d68 - -
Apr 3 03:14:48 103.33.33.3 2019-04-03 07:14:48 68 10.0.51.51 Karimykd aljfs\Group%20Internet%20No%20Restriction ctldl.windowsupdate.com 10.111.000.241 None - - OBSERVED "White list website;Non-Viewable/Infrastructure" - 304 TCP_MISS GET application/vnd.ms-cab-compressed http ctldl.windowsupdate.com 80 /msdownload/update/v3/static/trustedr/en/disallowedcertstl.cab ?5fae3ccb4e56dcf8 cab "Microsoft-CryptoAPI/6.1" 10.22.22.22 386 322 - "Microsoft Update" "Update Software" "none" 1 7ee0b471858c027d-00000000228366af-000000005ca45d68 - "{ %22expect_sandbox%22: false }"