All Apps and Add-ons

Why am I unable to extract fields in Splunk for Squid app?

bortnikl
New Member

I am using Splunk version 6.1.3 and Squid App 0.2v.
My props.conf contain:

[squid]
TIME_FORMAT = %s.%3N
MAX_TIMESTAMP_LOOKAHEAD = 15
KV_MODE = none
SHOULD_LINEMERGE = false
REPORT-squid = squid

My transforms.conf contain:

[squid]
REGEX = ^\d+\.\d+\s+(\d+)\s+([0-9\.]*)\s+([^/]+)/(\d+)\s+(\d+)\s+(\w+)\s+((?:([^:]*)://)?([^/:]+):?(\d+)?(/?[^ ]*))\s+(\S+)\s+([^/]+)/([^ ]+)\s+(.*)$
FORMAT = duration::$1 clientip::$2 action::$3 http_status::$4 bytes::$5 method::$6 uri::$7 proto::$8 uri_host::$9 uri_port::$10 uri_path::$11 username::$12 hierarchy::$13 server_ip::$14 content_type::$15

I am receiving squid logs at UDP 514. Sourcetype is manually set to "squid" (I just wrote squid in the field)
When I try search sourcetype="squid", things goes fine and I have results, but if I put command sourcetype="squid" action="*" , I do not have any results.
When I run squid app, there are no results too. Job inspector shows me this information:

This search has completed, but did not match any events. The terms specified in the highlighted portion of the search and this part is highlighted:

search sourcetype="squid" action="*" | eval reqcount=1

and

The following messages were returned by the search subsystem:

DEBUG: base lispy: [ AND sourcetype::squid ]
DEBUG: search context: user="admin", app="SplunkforSquid", bs-pathname="/opt/splunk/etc"

As a test, I tried to change sourcetype to "syslog" and run search

"*search sourcetype="syslog" action="**" 

.. and It gave me results, but when I tried to put search like:

*search sourcetype="syslog" clientip="*" uri_host="*" uri_path="*"*

.. I didn`t get any results. There are no clientip, uri_host or uri_path fields extracted, or even indexed.

Can you help me with this problem?

0 Karma
1 Solution

tskinnerivsec
Contributor

What format are your squid logs? More than likely, the regex isn't matching. This app was designed to work with a custom squid log format, which is shown in the readme file included with the app. That custom format is recommended to provide all of the enterprise security fields. Please submit a small sample of your squid logs and I can take a look.

View solution in original post

tskinnerivsec
Contributor

What format are your squid logs? More than likely, the regex isn't matching. This app was designed to work with a custom squid log format, which is shown in the readme file included with the app. That custom format is recommended to provide all of the enterprise security fields. Please submit a small sample of your squid logs and I can take a look.

bortnikl
New Member

Here is a sample: Aug 25 12:39:35 x.x.x.x Aug 25 12:39:37 somename (squid): 1408963177.419 138515 X.X.X.X TCP_MISS/000 10925 CONNECT safebrowsing-cache.google.com:443 - DEFAULT_PARENT/servername -

0 Karma

Ayn
Legend

Well not custom really, it's what Squid comes with by default out of the box. But yeah, assumptions regarding the format are definitely made.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Data Persistence in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. What happens if the OpenTelemetry collector ...

Introducing Splunk 10.0: Smarter, Faster, and More Powerful Than Ever

Now On Demand Whether you're managing complex deployments or looking to future-proof your data ...

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...