topic Re: How to extract fields from bluecoat logs? in Splunk Search

How to extract fields from bluecoat logs?

TheGU — Fri, 05 Nov 2010 12:08:37 GMT

How to extract the log example below:

2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200

I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.

for extraction example :

2010-09-29
16:23:44
2
172.16.106.54
exam.ple
Filter-ID==4
-
OBSERVED
"Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
http://market.example.com/product/1439306
200

Re: How to extract fields from bluecoat logs?

ziegfried — Fri, 05 Nov 2010 18:15:20 GMT

You could use a regular expression to extract the fields. Something like this:

props.conf

[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)

Re: How to extract fields from bluecoat logs?

TheGU — Thu, 02 Dec 2010 14:58:43 GMT

I found that transaction may be up to 2kB. Very hard to write a regex to cover all of them. However your answer is Ok. Thanks.

Re: How to extract fields from bluecoat logs?

Peterman — Mon, 10 Oct 2022 13:03:28 GMT

@TheGU wrote:
How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
2010-09-29
16:23:44
2
172.16.106.54
exam.ple
Filter-ID==4
-
OBSERVED
"Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
http://market.example.com/product/1439306
200

bluecoat server is awsome man