Splunk for Bluecoat ProxySG: After upgrade to ProxySG 6.6.23, why does automatic key-value extraction not work for user and date?

[mkuhlmann]

Hello all,

It seems that the auto key-value extraction for the "Splunk for Blue Coat" APP does not work with ProxySG 6.6.xx

[auto_kv_for_bluecoat_v6_5_x]
REGEX = (?:"([^"]+)"|(\S+))\s+(?:"(\d{1,2}:\d{1,2}:\d{1,2})"|(\d{1,2}:\d{1,2}:\d{1,2}))\s+(?:"(\d+)"|(\d+))\s+(?:"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"|(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\....

After migration to .6.6, the extraction for User and date don't work. Also, the filter result is set to proxied whatever happened.

Does anyone have new extractions for 6.6.xx???

Splunk Version 6.1
Splunk for Blue Coat 3.0.7

Regards
Michael

[ryandg]

Without you posting your config -- I can't be certain but I ran into this issue last week. Which sourcetype are you using? File or Syslog? In our case the issue was that even though the logs were coming in via syslog it actually still required the file sourcetype in order to correctly parse.

One of the showing signs for me was that the user field was replaced with a 3 digit number (the sc_status value).

If that doesn't resolve it, can you post your inputs.conf?

[Richfez]

While you have included part of the REGEX for an event, I doubt many of us have a handy Blue Coat log line to compare with. If you could include both the above REGEX in its entirety and at least one (if not two or three depending on the length) of the log lines it's trying to parse, I think you'll have a much better response!

[ridglerj]

The official app only supports up to 6.5, I've had to modify the regex to the following:

REGEX = (?:"([^"]+)"|(\S+))\s+(?:"(\d{1,2}:\d{1,2}:\d{1,2})"|(\d{1,2}:\d{1,2}:\d{1,2}))\s+(?:"(\d+)"|(\d+))\s+(?:"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"|(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))

That seems to capture all the fields, but the names don't appear to match up. I don't completely understand how bluecoat assigns the values to the string but I currently have this:

FORMAT = date::$1 date::$2 time::$3 time::$4 time_taken::$5 time_taken::$6 c_ip::$7 c_ip::$8 cs_username::$9 cs_username::$10 cs_auth_group::$11 cs_auth_group::$12 dest_ip::$15 dest_ip::$16 s_action::$23 s_action::$24 cs_categories::$25 cs_categories::$25 cs_Referer::$27 cs_Referer::$28 sc_status::$29 sc_status::$30 cs_method::$33 cs_method::$34 rs_Content_Type::$35 rs_Content_Type::$36 cs_uri_scheme::$37 cs_uri_scheme::$38 cs_uri_path::$39 cs_uri_path::$40 cs_uri_port::$41 cs_uri_port::$42 cs_uri_extension::$43 cs_uri_extension::$44 cs_uri_query::$45 cs_uri_query::$46 cs_User_Agent::$49 cs_User_Agent::$50 cs_host::$51 cs_host::$52 sc_bytes::$53 sc_bytes::$54 cs_bytes::$53 cs_bytes::$54 s_ip::$51 s_ip::$52 sc_bytes::$53 sc_bytes::$54 cs_bytes::$55 cs_bytes::$56 x_bluecoat_application_name::$59 x_bluecoat_application_name::$60

It's not perfect, but I hope this helps you.

[Richfez]

Oh, and the rest of the [auto_kv_for_bluecoat_v6_5_x] stanza: there's other things in there we need to know.