Join the Conversation
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: How to extract fields from bluecoat logs?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
- 2010-09-29
- 16:23:44
- 2
- 172.16.106.54
- exam.ple
- Filter-ID==4
- -
- OBSERVED
- "Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
- http://market.example.com/product/1439306
- 200
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@TheGU wrote:How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
- 2010-09-29
- 16:23:44
- 2
- 172.16.106.54
- exam.ple
- Filter-ID==4
- -
- OBSERVED
- "Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
- http://market.example.com/product/1439306
- 200
bluecoat server is awsome man
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I found that transaction may be up to 2kB. Very hard to write a regex to cover all of them. However your answer is Ok. Thanks.
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas
What Is the Name of the USB Key Inserted by Bob Smith? (BOTS Hint, Not the Answer)
Automating Threat Operations and Threat Hunting with Recorded Future
