How to write regex to read all the value in all the lines?

[fajri1203]

Hi Everyone,

I am desperately seeking help for my new query in SPLUNK. The search result will look like the below:

"pluginid","alertRef","alert","name","riskcode","confidence","riskdesc","confidencedesc","desc","instances","count","solution","otherinfo","reference","cweid","wascid","sourceid"
"100001","100001","Unexpected Content-Type was returned","Unexpected Content-Type was returned","1","3","Low (High)","High","<p>A Content-Type of text/html was returned by the server.</p><p>This is not one of the types expected to be returned by an API.</p><p>Raised by the 'Alert on Unexpected Content Types' script</p>","System.Xml.XmlElement","933","","","","-1","-1","20420"
"100000","100000","A Client Error response code was returned by the server","A Client Error response code was returned by the server","0","3","Informational (High)","High","<p>A response code of 401 was returned by the server.</p><p>This may indicate that the application is failing to handle unexpected input correctly.</p><p>Raised by the 'Alert on HTTP Response Code Error' script</p>","System.Xml.XmlElement","2831","","","","388","20","70"

My aim is to have a table in Splunk that can categorize each the value with the new field. For example:

pluginid alertRef alert
100001	 100001	  Unexpected Content-Type was returned","Unexpected Content-Type was returned
100000	 100000   A Client Error response code was returned by the server

So my regex should be able to read all the new line inside the csv search result..

My current solution is not really capable (as it only read single line, not multiple lines) as you can see below (I skipped the column name) :

^"\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+"\s+"(?P<plugin_id>\d+)","(?P<alert_ref>\d+)

Please help me to get the regex able to read all the new line in my CSV search result

[venkatasri]

@fajri1203  Try search-time approach, UF is a universal forwarder not applicable for your case. I wonder Add-on Microsoft Cloud Services don't have the extraction by default. 

You have to configure props.conf, and transforms.conf on search-head (SH) under $SPLUNK_HOME/etc/<app_name>/local OR  $SPLUNK_HOME/etc/system/local. If you are having SH cluster and using SH deployer you must know how to bundle push or contact your splunk admin. 

In standalone splunk SH the restart is required post changes.

[mscs:storage:blob]

REPORT-extract-csv-fields = extract-csv-fields

[extract-csv-fields]
DELIMS=","
FIELDS = "pluginid","alertRef","alert","name","riskcode","confidence","riskdesc","confidencedesc","desc","instances","count","solution","otherinfo","reference","cweid","wascid","sourceid"

[venkatasri]

Hi @fajri1203  You don't need a regex you could do it two ways if .csv is being forwarded from UF then set

inside props.conf

[sourcetypename]

INDEXED_EXTRACTIONS = csv

At search-time, on search-head you shall do following

https://www.splunk.com/en_us/blog/tips-and-tricks/quick-n-dirty-delimited-data-sourcetypes-and-you.h...

[fajri1203]

Hi @venkatasri ,

Thanks for the quick reply. May I know what is UF?

Currently our data is being taken from Azure Storage Account using the plugin called "Splunk Add-on for Microsoft Cloud Services" as you can see at the below image: