I am trying to configure the props and transforms conf files for logs that's in .csv format that we're querying via a virtual index in Hunk.
My props.conf file is configured as follows:
[source::/LogCentral/Applications/Shibboleth_PRD/*] sourcetype = shibboleth [shibboleth] REPORT-manual-shib = manual-shib
My transforms.conf file is configured as follows:
[manual-shib] DELIMS = "," FIELDS = username,mfa_service_code,date_requested,mfa_ip_address,mfa_value_returned,mfa_required,zip,latitude,longitude,timezone,country_code,country,city,isp,organization_name,as_name,region,region_name,audit_date_created,audit_created_by,audit_date_modified,audit_modified_by
I restart Hunk/Splunk service and when I run a search (index=shibboleth), the fields aren't being extracted.
Any help on what I'm missing or fat-fingered in my configs would be greatly appreciated.
I even modified my props.con file as follows and still no automated field extraction:
[source::/LogCentral/Applications/Shibboleth_PRD/*] sourcetype = shibboleth REPORT-manual-shib = manual-shib KV_MODE = none NO_BINARY_CHECK = true SHOULD_LINEMERGE = false category = Custom description = Comma-separated value format. Set header and other settings in "Delimited Settings" pulldown_type = true
Thx for pointing out that error. I fixed that source to /... and then re-ran the query but the fields still aren't being extracted. I opened a case with Splunk as all sources I have (properly set to /...) are also no longer having fields automatically extracted. This issue seemed to have arose when I updated to version 6.4.2.
Based on this link:
http://docs.splunk.com/Documentation/Hunk/6.4.1/HunkReleaseNotes/Releasenotes -- See ERP-1901 - there was an issue with CSV extraction but it was fixed in 6.4.1.
I am checking to see if that same fixed is also in 6.4.2
Thx for checking.
I have three other sources - Cisco ISE, Cisco ASA, and Windows Event Logs - whose fields were being extracted before I updated to version 6.4.2. For those three sources I was using the relevant add-ons: SplunkCiscoISE (with SplunkTAcisco-ise), SplunkTAwindows, and SplunkTA_cisco-asa.
Here are the stanzas for those three from my props.conf file:
[source::/LogCentral/Applications/ISE_PRD/...] sourcetype = cisco:ise:syslog DATETIME_CONFIG = NO_BINARY_CHECK = true SHOULD_LINEMERGE = false [source::/LogCentral/Applications/Firewall_PRD/...] sourcetype = cisco:asa [source::/LogCentral/WindowsEvent/PRD/...] sourcetype = windows_snare_syslog
I tried 6.4.2 with a CSV file, and Hunk did the right thing without any entry in the props.conf. Hunk auto translated the CSV to a JSON visualization automatically