topic Re: Need help with regex to read all the value in all the lines in Splunk Search

How to write regex to read all the value in all the lines?

fajri1203 — Tue, 20 Sep 2022 14:37:17 GMT

Hi Everyone,

I am desperately seeking help for my new query in SPLUNK. The search result will look like the below:

"pluginid","alertRef","alert","name","riskcode","confidence","riskdesc","confidencedesc","desc","instances","count","solution","otherinfo","reference","cweid","wascid","sourceid" "100001","100001","Unexpected Content-Type was returned","Unexpected Content-Type was returned","1","3","Low (High)","High","A Content-Type of text/html was returned by the server.This is not one of the types expected to be returned by an API.Raised by the 'Alert on Unexpected Content Types' script","System.Xml.XmlElement","933","","","","-1","-1","20420" "100000","100000","A Client Error response code was returned by the server","A Client Error response code was returned by the server","0","3","Informational (High)","High","A response code of 401 was returned by the server.This may indicate that the application is failing to handle unexpected input correctly.Raised by the 'Alert on HTTP Response Code Error' script","System.Xml.XmlElement","2831","","","","388","20","70"

My aim is to have a table in Splunk that can categorize each the value with the new field. For example:

pluginid alertRef alert 100001 100001 Unexpected Content-Type was returned","Unexpected Content-Type was returned 100000 100000 A Client Error response code was returned by the server

So my regex should be able to read all the new line inside the csv search result..

My current solution is not really capable (as it only read single line, not multiple lines) as you can see below (I skipped the column name) :

^"\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+"\s+"(?P<plugin_id>\d+)","(?P<alert_ref>\d+)

Please help me to get the regex able to read all the new line in my CSV search result

Need help with regex to read all the value in all the lines

venkatasri — Tue, 20 Sep 2022 02:39:13 GMT

Hi @fajri1203 You don't need a regex you could do it two ways if .csv is being forwarded from UF then set

inside props.conf

[sourcetypename]

INDEXED_EXTRACTIONS = csv

At search-time, on search-head you shall do following

https://www.splunk.com/en_us/blog/tips-and-tricks/quick-n-dirty-delimited-data-sourcetypes-and-you.html?301=/blog/2013/03/11/quick-n-dirty-delimited-data-sourcetypes-and-you.html&_gl=1*1q6w3af*_ga*NDMxNjgzMDU4LjE2Mjc2MDY0MDg.*_gid*NDUyOTcwODc3LjE2NjM1NDI5NDI.&_ga=2.10800753.452970877.1663542942-431683058.1627606408&locale=en_us

Re: Need help with regex to read all the value in all the lines

fajri1203 — Tue, 20 Sep 2022 02:57:53 GMT

Hi @venkatasri ,

Thanks for the quick reply. May I know what is UF?

Currently our data is being taken from Azure Storage Account using the plugin called "Splunk Add-on for Microsoft Cloud Services" as you can see at the below image:

Re: Need help with regex to read all the value in all the lines

venkatasri — Tue, 20 Sep 2022 04:55:13 GMT

@fajri1203 Try search-time approach, UF is a universal forwarder not applicable for your case. I wonder Add-on Microsoft Cloud Services don't have the extraction by default.

You have to configure props.conf, and transforms.conf on search-head (SH) under $SPLUNK_HOME/etc/<app_name>/local OR $SPLUNK_HOME/etc/system/local. If you are having SH cluster and using SH deployer you must know how to bundle push or contact your splunk admin.

In standalone splunk SH the restart is required post changes.

[mscs:storage:blob]

REPORT-extract-csv-fields = extract-csv-fields

[extract-csv-fields]
DELIMS=","
FIELDS = "pluginid","alertRef","alert","name","riskcode","confidence","riskdesc","confidencedesc","desc","instances","count","solution","otherinfo","reference","cweid","wascid","sourceid"