I'd use syslog to send squid log and used squid apps. So I need transform sourcetype to squid.
Here's my props.conf and tranforms.conf
[props.conf]
[squid]
TIME_FORMAT = %s.%3N
MAX_TIMESTAMP_LOOKAHEAD = 15
KV_MODE = none
SHOULD_LINEMERGE = false
REPORT-squid = squid
[[source::udp:514]
TRANSFORMS-sqsourcetype= sq_sourcetyper
[transforms.conf]
[squid]
REGEX = ^\d+.\d+\s+(\d+)\s+([0-9.])\s+([^/]+)/(\d+)\s+(\d+)\s+(\w+)\s+((?:([^:])://)?([^/:]+):?(\d+)?(/?[^ ]))\s+(\S+)\s+([^/]+)/([^ ]+)\s+(.)$
FORMAT = duration::$1 clientip::$2 action::$3 http_status::$4 bytes::$5 method::$6 uri::$7 proto::$8 uri_host::$9 uri_port::$10 uri_path::$11 username::$12 hierarchy:
:$13 server_ip::$14 content_type::$15
[sq_sourcetyper]
DEST_KEY = MetaData:Sourcetype
REGEX = SquidProxyLog
FORMAT = sourcetype::squid
And event source
Dec 7 15:20:58 ipaddress_from Dec 7 15:19:57 hostname_here SquidProxyLog 0 1323242396.113 11 ip_address_ TCP_MISS/200 773 GET http://xxx.search.yahoo.net/ip- -DIRECT/119.160.251.5 application/javascript
After restart splunk, there's no sourcetype named squid, did I miss anything?
Thanks
I didn't see a SOURCE_KEY in your configuration. Instead of using SquidProxyLog as your regex, perhaps you can match it with the Host of the Squid server as returned by the "Host" field.
In this case, part of your transforms.conf would look something like the below, assuming the host is 192.168.1.1.
[sq_sourcetyper]
SOURCE_KEY = MetaData:Host
REGEX = 192.168.1.1
DEST_KEY = MetaData:Sourcetype
FORMAT= sourcetype::squid
I also noticed a double [[ brackets in your props.conf and I'm assuming this is a cut-and-paste error.
[[source::udp:514]
TRANSFORMS-sqsourcetype= sq_sourcetyper
I didn't see a SOURCE_KEY in your configuration. Instead of using SquidProxyLog as your regex, perhaps you can match it with the Host of the Squid server as returned by the "Host" field.
In this case, part of your transforms.conf would look something like the below, assuming the host is 192.168.1.1.
[sq_sourcetyper]
SOURCE_KEY = MetaData:Host
REGEX = 192.168.1.1
DEST_KEY = MetaData:Sourcetype
FORMAT= sourcetype::squid
I also noticed a double [[ brackets in your props.conf and I'm assuming this is a cut-and-paste error.
[[source::udp:514]
TRANSFORMS-sqsourcetype= sq_sourcetyper
Instead of a [source::udp:514] instanza, may I use [syslog]? My tcp and udp ports 514 are both manually sourcetyped to "syslog", Will splunk parse again after changing sourcetype? Or in this case will it ignore the first instanza of the props.conf?
It's worked thanks 😄