I believe this might be caused by having a specific UDP listener defined as:
X.X.X.X:514 cisco:wsa:w3c
But then later after that I have a generic UDP listener defined as:
514 cisco:asa
Anyone else have this same issue and get them to file out separately? Because Splunk sticks these under the cisco:asa type, it doesn't populate the CiscoSecSutie dashboards.
Just have the one server, and yes:
drwxr-xr-x 8 splunk splunk 4.0K Apr 29 18:23 Splunk_CiscoSecuritySuite/
drwx--x--x 9 splunk splunk 4.0K Oct 8 11:52 Splunk_TA_cisco-asa/
drwx--x--x 9 root root 4.0K Oct 8 09:31 Splunk_TA_cisco-wsa/
And I've been changing everything in 'Splunk_TA_cisco-wsa'. My props and transforms are still default.
# cat props.conf
[source::udp:514]
TRANSFORMS-change_cisco_wsa = set_sourcetype_cisco_wsa, set_index_cisco_wsa
root@splunk: local # cat transforms.conf
######Access logs in squid format######
[kv_for_cisco_wsa_squid]
REGEX = ([0-9.]*) *[0-9]* ([0-9.]*) ([A-Z_]*)/([0-9]*) ([0-9]*) ([A-Z]*) ([^ ]*) ([^ ]*) ([^/]*)/([^ ]*) ([^ ]*) ([^ ]+) <([^,]+),([^,]+),"*([0-9]{0,2}|\-|\w+)"*,"([^"]+)",[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,"([^"]+)",[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,"([^"]+)",[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^,]+,[^>]+>\s*\-\s*"?([^"]+)"?$
FORMAT = src_ip::$2 txn_result_code::$3 status::$4 bytes_in::$5 http_method::$6 url::$7 user::$8 server_contact_mode::$9 dest::$10 http_content_type::$11 acltag::$12 x_webcat_code_abbr::$13 wbrs_score::$14 x_webroot_scanverdict::$15 webroot_threat_name::$16 mcafee_virus_name::$17 malware_category::$18 vendor_suspect_user_agent::$19
[cisco_wsa_category_lookup]
filename = cisco_wsa_category_map_lookup.csv
[cisco_wsa_vendor_info_lookup]
filename = cisco_wsa_vendor_lookup.csv
[cisco_wsa_malware_action_lookup]
filename = cisco_wsa_malware_action_lookup.csv
[cisco_wsa_proxy_action_lookup]
filename = cisco_wsa_proxy_action_lookup.csv
######L4TM logs######
[kv_for_cisco_wsa_Firewall_l4tm]
REGEX = [A-Za-z]* ([A-Za-z]* +[0-9]* [0-9:]* [0-9]*) [A-Za-z]*: Firewall ([A-Za-z]*) ([A-Z]+).* data from ([0-9a-z.]*)(:([0-9a-z]*)){0,1} to ([0-9a-z.]*)(\(([A-Za-z0-9 -\_]*)\)){0,1}(:([^.]+)){0,1}.
FORMAT = vendor_action::$2 transport::$3 src::$4 src_port::$6 dest::$7 dest_domain::$9 dest_port::$11
[kv_for_cisco_wsa_Address_l4tm]
REGEX = [A-Za-z]* ([A-Za-z]* +[0-9]* [0-9:]* [0-9]*) [A-Za-z]*: Address ([0-9.:]*) [A-Za-z]* [A-Za-z]* ([A-Za-z0-9.\_\-]*)( \([A-Za-z0-9 .\_\-]*\)){0,1} [A-Za-z]* [A-Za-z]* firewall ([A-Za-z ]*)
FORMAT = dest::$2 dest_domain::$3 vendor_action::$5
[kv_for_cisco_wsa_removed_l4tm]
REGEX = [A-Za-z]* ([A-Za-z]* +[0-9]* [0-9:]* [0-9]*) [A-Za-z]*: Address ([0-9.:]*) [A-Za-z]* ([A-Za-z0-9.\-\_]*)( \([A-Za-z0-9 .\-\_]*\)){0,1} ([A-Za-z]*) [A-Za-z ]*
FORMAT = dest::$2 dest_domain::$3 vendor_action::$5
[cisco_wsa_traffic_action_lookup]
filename = cisco_wsa_traffic_action_lookup.csv
########W3C Logs########
##Uncomment the following lines and follow the instructions provided on http://docs.splunk.com/Documentation/AddOns/released/CiscoWSA/Configurew3clogfieldextractions.
[auto_kv_for_cisco_wsa_w3c]
DELIMS = " "
FIELDS = timestamp,x_elapsed_time,s_ip,x_acltag,c_ip,sc_result_code,x_resultcode_httpstatus,sc_http_status,cs_mime_type,cs-bytes,sc-bytes,cs_method,cs_url,cs_username,x_webcat_code_abbr,x_wbrs_score,x_req_dvs_scanverdict,x_webroot_threat_name,x_mcafee_virus_name
Using btool I see them getting listed. Though I'm not seeing any real red flags about them. Stepped through the link you provided.
Hi @jasmilner and @jconger
Glad you two are having a robust dialogue here on Answers 🙂 For future reference, please be sure to click on "Add comment" under the actual posted answer or "Reply" on a comment if you're responding to each other. Right now, you've both been posting a new answer each time through the "Your Answer" space at the bottom of the page. I've converted most of your answers to comments appropriately, but I can't do that for posts more than 1500 characters (like this current "Answer" I'm commenting on :P). This will help with other users seeing the proper flow of the conversation. Thanks!
Patrick
So I tore it all out, and started from scratch. Now it works. At least with the classifying the sourcetype. (Which is bitter sweet, because I'm not sure what was the culprit)
Now all the fields are miss-parsed. Which I'm guessing is because of:
[auto_kv_for_cisco_wsa_w3c]
DELIMS = " "
FIELDS = timestamp,x_elapsed_time,s_ip,x_acltag,c_ip,sc_result_code,x_resultcode_httpstatus,sc_http_status,cs_mime_type,cs-bytes,sc-bytes,cs_method,cs_url,cs_username,x_webcat_code_abbr,x_wbrs_score,x_req_dvs_scanverdict,x_webroot_threat_name,x_mcafee_virus_name
Since, the time stamp in the front of the raw logs have spaces, and the delimiter is a space... back to Google for round two!
So the way this all works (having multiple sourcetypes coming in on the same port) is like this:
Here is basically what you should have:
inputs.conf
[udp://:514]
connection_host = dns
props.conf
[source::udp:514]
TRANSFORMS-change_cisco_wsa = set_sourcetype_cisco_wsa
transforms.conf
[set_sourcetype_cisco_wsa]
DEST_KEY = MetaData:Sourcetype
REGEX = some regex that will match your WSA events
FORMAT = sourcetype::cisco:wsa
So when I kill my:
[udp://514]
source = cisco_syslog
sourcetype = cisco:asa
disabled = false
connection_host = dns
And just leave the WSA module running:
[udp://xx.xx.xx.xx:514]
connection_host = dns
source = cisco_syslog
sourcetype = cisco:wsa:w3c
acceptFrom = 170.193.90.106
disabled = false
[udp://yy.yy.yy.yy:514]
connection_host = dns
source = cisco_syslog
sourcetype = cisco:wsa:w3c
acceptFrom = 170.193.90.107
disabled = false
the logs are now tagged with a sourcetype of "udp:514" not "cisco:asa" nor "cisco:wsa:w3c"
I don't see a corresponding set_sourcetype_cisco_wsa in transforms.conf.
Oh you know I once had this in transforms:
[set_sourcetype_cisco_wsa]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(x.x.x.x|y.y.y.y)[\w\.\-]*\]?\s
FORMAT = sourcetype::cisco_wsa_w3c
DEST_KEY = MetaData:Sourcetype
[set_index_cisco_wsa]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(x.x.x.x|y.y.y.y)[\w\.\-]*\]?\s
DEST_KEY = _MetaData:Index
FORMAT = cisco
But that never seemed to make a difference. Also, the original I found had:
FORMAT = sourcetype::cisco_wsa_squid
But I'm not using squid.
the Xs and Ys were actual IPs 😉
Putting it back in the top of transforms and restarting Splunk didn't seem to do anything.
I haven't read that yet, just for one in props.conf. What would I need to do in transforms?
You can use props.conf and transforms.conf to inspect the event and assign the correct sourcetype (rather than defining the sourcetype in inputs.conf).
Check out the docs for the Cisco Security Suite for an example -> https://apps.splunk.com/app/525/#/documentation
About 1/3 of the way down the page, see the notes under "Splunk Add-On for Cisco ASA Configuration Tip"
Yep. I followed those directions already.
Are you using the ASA and WSA add-ons on your Splunk server or a forwarder? Can you post your props.conf and transforms.conf?
Also, you might want to try out the btool command to ensure your props and transforms are applied correctly:
http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Usebtooltotroubleshootconfigurati...