All Apps and Add-ons
Highlighted

transform sourcetype(squid) from UDP 514

Explorer

I'd use syslog to send squid log and used squid apps. So I need transform sourcetype to squid.
Here's my props.conf and tranforms.conf

[props.conf]

[squid]

TIME_FORMAT = %s.%3N

MAXTIMESTAMPLOOKAHEAD = 15

KV_MODE = none

SHOULD_LINEMERGE = false

REPORT-squid = squid

[[source::udp:514]

TRANSFORMS-sqsourcetype= sq_sourcetyper

[transforms.conf]

[squid]

REGEX = ^\d+.\d+\s+(\d+)\s+([0-9.])\s+([^/]+)/(\d+)\s+(\d+)\s+(\w+)\s+((?:([^:])://)?([^/:]+):?(\d+)?(/?[^ ]))\s+(\S+)\s+([^/]+)/([^ ]+)\s+(.)$

FORMAT = duration::$1 clientip::$2 action::$3 httpstatus::$4 bytes::$5 method::$6 uri::$7 proto::$8 urihost::$9 uriport::$10 uripath::$11 username::$12 hierarchy:
:$13 serverip::$14 contenttype::$15

[sq_sourcetyper]

DEST_KEY = MetaData:Sourcetype

REGEX = SquidProxyLog

FORMAT = sourcetype::squid

And event source
Dec 7 15:20:58 ipaddressfrom Dec 7 15:19:57 hostnamehere SquidProxyLog 0 1323242396.113 11 ipaddress TCP_MISS/200 773 GET http://xxx.search.yahoo.net/ip- -DIRECT/119.160.251.5 application/javascript

After restart splunk, there's no sourcetype named squid, did I miss anything?
Thanks

0 Karma
Highlighted

Re: transform sourcetype(squid) from UDP 514

Splunk Employee
Splunk Employee

I didn't see a SOURCE_KEY in your configuration. Instead of using SquidProxyLog as your regex, perhaps you can match it with the Host of the Squid server as returned by the "Host" field.

In this case, part of your transforms.conf would look something like the below, assuming the host is 192.168.1.1.

[sq_sourcetyper] 
SOURCE_KEY = MetaData:Host 
REGEX = 192.168.1.1
DEST_KEY = MetaData:Sourcetype 
FORMAT= sourcetype::squid

I also noticed a double [[ brackets in your props.conf and I'm assuming this is a cut-and-paste error.

[[source::udp:514]
TRANSFORMS-sqsourcetype= sq_sourcetyper

View solution in original post

Highlighted

Re: transform sourcetype(squid) from UDP 514

Explorer

It's worked thanks 😄

0 Karma
Highlighted

Re: transform sourcetype(squid) from UDP 514

Explorer

Instead of a [source::udp:514] instanza, may I use [syslog]? My tcp and udp ports 514 are both manually sourcetyped to "syslog", Will splunk parse again after changing sourcetype? Or in this case will it ignore the first instanza of the props.conf?

0 Karma