Splunk Search

Is my sourcetype override messing up my field extraction, or am I?

Communicator

My sourcetype override is working, but my field extractions are not.

props.conf

[source::udp:514]
TRANSFORMS-changesourcetype = set_sourcetype_barracuda_sf

[barracuda_sf]
KV_MODE=none
REPORT-bsf = bsf_scan, bsf_send, bsf_recv

transforms.conf

[set_sourcetype_barracuda_sf]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(10.1.5.49|10.1.5.50)[\w\.\-]*\]?\s
FORMAT = sourcetype::barracuda_sf
DEST_KEY = MetaData:Sourcetype

[bsf_scan]
REGEX = (?:[^\s\n]*\s){5}([\w/]*)\[(\d*)\]:\s(.*\]|127.0.0.1)\s([\w\d-]*)\s(\d*)\s(\d*)\s(SCAN)\s(.*)
FORMAT = barracuda_process::$2  barracuda_pid::$3 client_ip::$4 message_id::$5 start_time::$6 end_time::$7 service::$8 info::$9

[bsf_send]
REGEX = (?:[^\s\n]*\s){5}([\w/]*)\[(\d*)\]:\s(.*\]|127.0.0.1)\s([\w\d-]*)\s(\d*)\s(\d*)\s(SCAN)\s(.*)
FORMAT = barracuda_process::$2  barracuda_pid::$3 client_ip::$4 message_id::$5 start_time::$6 end_time::$7 service::$8 info::$9

[bsf_recv]
REGEX = (?:[^\s\n]*\s){5}([\w/]*)\[(\d*)\]:\s(.*\]|127.0.0.1)\s([\w\d-]*)\s(\d*)\s(\d*)\s(SEND)\s(.*)
FORMAT = barracuda_process::$2  barracuda_pid::$3 client_ip::$4 message_id::$5 start_time::$6 end_time::$7 service::$8 info::$9
0 Karma

Contributor

Are the extracts not working at all?
I ran into some troubles as well with one big syslogfeed on udp so now i use a sysloghost with a forwarder (rolled files monitor) and push this as one sourcetype to the indexer (splunksyslog).
There i use exact the same method as you are using ( making 8+ sourcetype overrides) and have dozens of fields extracts on those new ones.
So I am not sure if this is working only for cooked data,,,my concern was the load ( 50G a day,so wanted a store and forward before parsing)
I def. want to test your setup cause I have some planned deployments with this as well!

0 Karma

Legend

I think you have been caught by way that stanzas in props.conf are processed; Splunk only makes one pass. You probably shouldn't count on the transformed sourcetype to be available for use in the second stanza.

But there is an easy cure for your problem. You can eliminate the second stanza altogether, unless you already have some barracuda_sf events from some other input.

[source::udp:514]
TRANSFORMS-changesourcetype = set_sourcetype_barracuda_sf
KV_MODE=none
REPORT-bsf = bsf_scan, bsf_send, bsf_recv

[barracuda_sf]
KV_MODE=none
REPORT-bsf = bsf_scan, bsf_send, bsf_recv
0 Karma

Contributor

Here's another thing, isn't my sourcetype override happening at index time and my field extractions happening at search time? :
yeah thats what i think,,,btw why the KV_MODE=none setting
??

0 Karma

SplunkTrust
SplunkTrust

If you're going to pump syslog directly into Splunk, there is nothing at all wrong with defining multiple syslog ports on a per-sourcetype basis. Use (for example) 5140 for barracuda, 5141 for VMWare ESXi, 5142 for Cisco ASA, etc ...

0 Karma

Communicator

Here's another thing, isn't my sourcetype override happening at index time and my field extractions happening at search time?

0 Karma

Communicator

I tried putting REPORT-bsf = bsf_scan, bsf_send, bsf_recv in my [source::udp:514], but unfortunately I still didn't get my field extractions.

0 Karma

Communicator

My concern would be that my REPORT and KV_MODE keywords would affect all of my syslog stuff.

Maybe this is another example of why one shouldn't pump syslog directly into Splunk? 😕

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!