All Apps and Add-ons

Why am I unable to extract fields in Splunk for Squid app?

bortnikl
New Member

I am using Splunk version 6.1.3 and Squid App 0.2v.
My props.conf contain:

[squid]
TIME_FORMAT = %s.%3N
MAX_TIMESTAMP_LOOKAHEAD = 15
KV_MODE = none
SHOULD_LINEMERGE = false
REPORT-squid = squid

My transforms.conf contain:

[squid]
REGEX = ^\d+\.\d+\s+(\d+)\s+([0-9\.]*)\s+([^/]+)/(\d+)\s+(\d+)\s+(\w+)\s+((?:([^:]*)://)?([^/:]+):?(\d+)?(/?[^ ]*))\s+(\S+)\s+([^/]+)/([^ ]+)\s+(.*)$
FORMAT = duration::$1 clientip::$2 action::$3 http_status::$4 bytes::$5 method::$6 uri::$7 proto::$8 uri_host::$9 uri_port::$10 uri_path::$11 username::$12 hierarchy::$13 server_ip::$14 content_type::$15

I am receiving squid logs at UDP 514. Sourcetype is manually set to "squid" (I just wrote squid in the field)
When I try search sourcetype="squid", things goes fine and I have results, but if I put command sourcetype="squid" action="*" , I do not have any results.
When I run squid app, there are no results too. Job inspector shows me this information:

This search has completed, but did not match any events. The terms specified in the highlighted portion of the search and this part is highlighted:

search sourcetype="squid" action="*" | eval reqcount=1

and

The following messages were returned by the search subsystem:

DEBUG: base lispy: [ AND sourcetype::squid ]
DEBUG: search context: user="admin", app="SplunkforSquid", bs-pathname="/opt/splunk/etc"

As a test, I tried to change sourcetype to "syslog" and run search

"*search sourcetype="syslog" action="**" 

.. and It gave me results, but when I tried to put search like:

*search sourcetype="syslog" clientip="*" uri_host="*" uri_path="*"*

.. I didn`t get any results. There are no clientip, uri_host or uri_path fields extracted, or even indexed.

Can you help me with this problem?

0 Karma
1 Solution

tskinnerivsec
Contributor

What format are your squid logs? More than likely, the regex isn't matching. This app was designed to work with a custom squid log format, which is shown in the readme file included with the app. That custom format is recommended to provide all of the enterprise security fields. Please submit a small sample of your squid logs and I can take a look.

View solution in original post

tskinnerivsec
Contributor

What format are your squid logs? More than likely, the regex isn't matching. This app was designed to work with a custom squid log format, which is shown in the readme file included with the app. That custom format is recommended to provide all of the enterprise security fields. Please submit a small sample of your squid logs and I can take a look.

bortnikl
New Member

Here is a sample: Aug 25 12:39:35 x.x.x.x Aug 25 12:39:37 somename (squid): 1408963177.419 138515 X.X.X.X TCP_MISS/000 10925 CONNECT safebrowsing-cache.google.com:443 - DEFAULT_PARENT/servername -

0 Karma

Ayn
Legend

Well not custom really, it's what Squid comes with by default out of the box. But yeah, assumptions regarding the format are definitely made.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...