Are you a member of the Splunk Community?
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: How to extract fields from bluecoat logs?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
- 2010-09-29
- 16:23:44
- 2
- 172.16.106.54
- exam.ple
- Filter-ID==4
- -
- OBSERVED
- "Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
- http://market.example.com/product/1439306
- 200
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@TheGU wrote:How to extract the log example below:
2010-09-29 16:23:44 2 172.16.106.54 exam.ple Filter-ID==4 - OBSERVED "Search Engines/Portals;News/Media" http://market.example.com/product/1439306 200
I got this from bluecoat server. I can't use DELIMS=" " to extract KV because some fields have a " " (spacebar) too.
for extraction example :
- 2010-09-29
- 16:23:44
- 2
- 172.16.106.54
- exam.ple
- Filter-ID==4
- -
- OBSERVED
- "Search Engines/Portals;News/Media" <<<<<<<<<< this field make a problem!!!
- http://market.example.com/product/1439306
- 200
bluecoat server is awsome man
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You could use a regular expression to extract the fields. Something like this:
props.conf
[your_sourcetype]
EXTRACT-fields=^[\d\-]+) [\d:]+ (?<FIELD_1>\d+) (?<clientip>[\d\.]+) (?<domain>\S+) Filter\-ID==(?<filter_id>\S+) "(?<url_category>[^"]+)" (?<url>\S+) (?<status>\d+)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I found that transaction may be up to 2kB. Very hard to write a regex to cover all of them. However your answer is Ok. Thanks.
Tech Talk Recap | Mastering Threat Hunting
Observability for AI Applications: Troubleshooting Latency
Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?
