Hi experts
Bluecoat proxysg logs are not parsing properly, we are sending logs from Bluecoat proxy to syslog-ng server in W3C format. we notice that the bluecoat proxy logs itself does not sending the log headers. Does TA Require header in logs. please need help
Thanking in advance
@mshakeb ,
There is no standard TA for bluecoat proxy which works for all versions of bluecoat proxy.
log format is changed based on version and how you format the log in the proxy.
if you can share sample log or the variables used to format the log in bluecoat (preferred) is helpful to help you.
Hi,
we are forwarding W3C ELEF format from Bluecoat proxysg to syslog-ng. we have notice on syslog-ng server bluecoat logs are not having log headers. please see the log sample
Aug 9 12:00:00 10.0.xx.xx 2020-08-09 09:00:00 1 10.xx.0.10 aaxxxx yyyDomain\xcyv%20XXXXXXX%20Internet%2Group 10.0.xx.xxx 10.0.xx.xxx None - - OBSERVED "Youtube;Audio/Video Clips" https://www.youtube.com/ 200 TCP_HIT GET text/plain https r6---sn-4wgd.googlevideo.com 443 /videoplayback ?expire=15969881&ei=nbEvX9yLA4OWWMuDg5AO&ip=98.220.xx.xxx&id=oAJ4JFOEDAszlxnibA285uN6_lOZUonUhoyA44Rb2mE6&itag=396&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278%cccv%2C395%2C396%2C397%2C398%2C399&source=youtube95390&req_id=53b73fd85edb82c5&altitags=395%2C394&rn=541668&rbuf=0 - "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/10.0.0.xxx Safari/537.36" 10.x.x.000 2113 1530 - "YouTube" "Play Video" "Video Hosting" 2 a954beb9a-00000000b2ac6060-000000005f2fbb10 - -
download TA from https://splunkbase.splunk.com/app/2758/
use below props.conf and transforms.conf in local directory. make sure your sourctype of bluecoat logs is matching with "bluecoat:proxysg:access:syslog"
props.conf
[bluecoat:proxysg:access:syslog]
pulldown_type = true
category = Network & Security
description = Data from Blue Coat ProxySG in W3C ELFF format thru syslog
KV_MODE = none
SHOULD_LINEMERGE = false
EVENT_BREAKER_ENABLE=true
MAX_DAYS_AGO = 10951
REPORT-categories = bluecoat_categories
REPORT-bluecoat_custom = REPORT-bluecoat_custom
FIELDALIAS-cookie = cs_Cookie as cookie
FIELDALIAS-duration = time_taken as duration
FIELDALIAS-src=c_ip as src
FIELDALIAS-src_port = c_port as src_port
FIELDALIAS-user = cs_username as user
FIELDALIAS-http_referrer = cs_Referer as http_referrer
FIELDALIAS-status = sc_status as status
FIELDALIAS-action = s_action as vendor_action
FIELDALIAS-http_method = cs_method as http_method
FIELDALIAS-content_type = rs_Content_Type as http_content_type
FIELDALIAS-dest_host = cs_host as dest_host
FIELDALIAS-dest_port = s_port as dest_port
FIELDALIAS-user_agent = cs_User_Agent as http_user_agent
FIELDALIAS-dest_ip = cs_ip as dest_ip
FIELDALIAS-dvc = s_ip as dvc
FIELDALIAS-bytes_in = sc_bytes as bytes_in
FIELDALIAS-bytes_out = cs_bytes as bytes_out
FIELDALIAS-uri_path = cs_uri_path as uri_path
FIELDALIAS-uri_query = cs_uri_query as uri_query
FIELDALIAS-protocol = cs_protocol as protocol
FIELDALIAS-packets_in = c_pkts_received as packets_in
FIELDALIAS-session_id = s_session_id as session_id
EVAL-app = "Blue Coat ProxySG"
EVAL-dest = coalesce(dest_ip, dest_host)
EVAL-bytes = bytes_in + bytes_out
EVAL-url = coalesce(cs_uri, if(isnull(cs_uri_scheme) OR (cs_uri_scheme=="-"), "", cs_uri_scheme+"://") + cs_host + cs_uri_path + if(isnull(cs_uri_query) OR (cs_uri_query == "-"), "", cs_uri_query))
EVAL-product = "ProxySG"
EVAL-vendor = "Blue Coat"
EVAL-vendor_product = "Blue Coat ProxySG"
# Eval action to blocked if sc_filter_result is DENIED
EVAL-action = case(sc_filter_result=="DENIED","blocked")
LOOKUP-vendor_traffic_action = bluecoat_proxy_action_lookup vendor_action OUTPUTNEW action, transport
transforms.conf
Note: the fields after field4 may not match with your field format, identify and replace right field names in below FIELDS
[REPORT-bluecoat_custom]
DELIMS = " "
#Below was used before remove default string appended by syslog-ng
FIELDS = "field1","field2","field3","field4","date","time","time-taken","c-ip","cs-username","cs-auth-group","s-supplier-name","s-supplier-ip","s-supplier-country","s-supplier-failures","x-exception-id","sc-filter-result","cs-categories","cs(Referer)","sc-status","s-action","cs-method","rs(Content-Type)","cs-uri-scheme","cs-host","cs-uri-port","cs-uri-path","cs-uri-query","cs-uri-extension","cs(User-Agent)","s-ip","sc-bytes","cs-bytes","x-virus-id","x-bluecoat-application-name","x-bluecoat-application-operation","x-bluecoat-application-groups","cs-threat-risk","x-bluecoat-transaction-uuid","x-icap-reqmod-header(X-ICAP-Metadata)","x-icap-respmod-header(X-ICAP-Metadata)"
[bluecoat_proxy_action_lookup]
filename = bluecoat_proxy_actions.csv
case_sensitive_match = false
[bluecoat_categories]
SOURCE_KEY = cs_categories
REGEX = (?<category>[^;]+)
MV_ADD = true
The reason why default transforms doesn't work is because you are collecting events through syslog server, and syslog server is appending data , time, and host to actual message coming from proxy.
up vote if this works.
Hi
General rules is "Don't remove (or change) configurations on default folder! Just add needed configuration files to local folder and only needed parts which you must change"
r. Ismo
Bluecoat proxy version 6.7
format : "date","time","time-taken","c-ip","cs-username","cs-auth-group","s-supplier-name","s-supplier-ip","s-supplier-country","s-supplier-failures","x-exception-id","sc-filter-result","cs-categories","cs(Referer)","sc-status","s-action","cs-method","rs(Content-Type)","cs-uri-scheme","cs-host","cs-uri-port","cs-uri-path","cs-uri-query","cs-uri-extension","cs(User-Agent)","s-ip","sc-bytes","cs-bytes","x-virus-id","x-bluecoat-application-name","x-bluecoat-application-operation","x-bluecoat-application-groups","cs-threat-risk","x-bluecoat-transaction-uuid","x-icap-reqmod-header(X-ICAP-Metadata)","x-icap-respmod-header(X-ICAP-Metadata)"
Can you search “fields” in your proxy logs where you will see header of the logs, bluecoat version etc. if you found that, please share here.