Are you a member of the Splunk Community?
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- How to extract fields from bluecoat logs?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
- 2010-09-29
- 16:23:44
- 2
- 172.16.106.54
- exam.ple
- Filter-ID==4
- -
- OBSERVED
- "Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
- http://market.example.com/product/1439306
- 200
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@TheGU wrote:How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
- 2010-09-29
- 16:23:44
- 2
- 172.16.106.54
- exam.ple
- Filter-ID==4
- -
- OBSERVED
- "Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
- http://market.example.com/product/1439306
- 200
bluecoat server is awsome man
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I found that transaction may be up to 2kB. Very hard to write a regex to cover all of them. However your answer is Ok. Thanks.
Community Content Calendar, September edition
Splunkbase Unveils New App Listing Management Public Preview
Leveraging Automated Threat Analysis Across the Splunk Ecosystem
