unit_hostname not being extracted properly

[gcrawford_newba]

Hi All,

Hope someone can help me here.

We are configuring Splunk for F5 Security and we can't get the field extractions to work properly. It's to do with the syslog data at the front of the string and this looks like it's taken care of in the delimiters in the appropriate transforms.conf however it does not appear to be doing its thing.

I have adjusted the asm extract to suite F5 OS v11. The transforms.conf entry is below: -

[asm_extract_11]
DELIMS = ","
FIELDS = "syslog_specific_data":"unit_hostname","management_ip_address","web_application_name","policy_name","policy_apply_date","violations","support_id","request_sta
tus","response_code","src_ip","method","protocol","uri","request","query_string","x_forwarded_for_value","sig_ids","sig_names","date_time","severity"

The issue appears to be that the syslog_specific_data is delimited via a different delimiter (:) than the rest of the data - as such the field is being extracted as syslog_specific_data_unit_hostname with all the syslog data and the unit_hostname as one big field... this doesn't work very well with the app or anything else for that matter.

Has anyone else experienced this and if so how did you get around it? Did you manage to strip out the syslog data via a regex in the transforms.conf or something similar?

Example input being index is as below and we want the unit_hostname to be identified as blah.host.local

Feb 4 16:05:08 a.b.c.d Feb 4 16:04:24 blah.host.local ASM:"blah.host.local","a.b.c.d","","","2014-02-04 15:00:40","Illegal URL length,Illegal request length,Illegal file type,Modified domain cookie(s)","12288077832457980502","alerted","404","w.x.y.z","GET","HTTPS","/robots.txt","GET /robots.txt HTTP/1.1\r\nHost: external.example.com\r\nConnection: close, TE\r\nTE: trailers\r\nUser-Agent: Mozilla/5.0 (compatible; Funnelback)""1""\r\n\r\n","","N/A","","","2014-02-04 16:04:24","Critical"

[gcrawford_newba · Accepted solution]

Never mind peeps, I got around it by using a SEDCMD on input to change the ASM: to ASM, then changed the transforms.conf such that "syslog_specific_data":"unit_hostname" became "syslog_specific_data","unit_hostname"

Now all is well.

[davebo1896]

a tweak to the asm_tokenizer can clean this up, as well.
local/transforms.conf

[asm_tokenizer]
REGEX = ([^=,:]+)="([^.]+)|([^\"]+)"
FORMAT = $1::$2