Splunk Search

How to write regex to read all the value in all the lines?

fajri1203
Loves-to-Learn

Hi Everyone,


I am desperately seeking help for my new query in SPLUNK. The search result will look like the below:

 

fajri1203_0-1663639729351.png

 

 

 

"pluginid","alertRef","alert","name","riskcode","confidence","riskdesc","confidencedesc","desc","instances","count","solution","otherinfo","reference","cweid","wascid","sourceid"
"100001","100001","Unexpected Content-Type was returned","Unexpected Content-Type was returned","1","3","Low (High)","High","<p>A Content-Type of text/html was returned by the server.</p><p>This is not one of the types expected to be returned by an API.</p><p>Raised by the 'Alert on Unexpected Content Types' script</p>","System.Xml.XmlElement","933","","","","-1","-1","20420"
"100000","100000","A Client Error response code was returned by the server","A Client Error response code was returned by the server","0","3","Informational (High)","High","<p>A response code of 401 was returned by the server.</p><p>This may indicate that the application is failing to handle unexpected input correctly.</p><p>Raised by the 'Alert on HTTP Response Code Error' script</p>","System.Xml.XmlElement","2831","","","","388","20","70"

 

 

 

My aim is to have a table in Splunk that can categorize each the value with the new field. For example:

 

 

 

pluginid alertRef alert
100001	 100001	  Unexpected Content-Type was returned","Unexpected Content-Type was returned
100000	 100000   A Client Error response code was returned by the server

 

 

 

So my regex should be able to read all the new line inside the csv search result..

My current solution is not really capable (as it only read single line, not multiple lines) as you can see below (I skipped the column name) :

 

 

^"\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+","\w+"\s+"(?P<plugin_id>\d+)","(?P<alert_ref>\d+)

 

 

Please help me to get the regex able to read all the new line in my CSV search result

Labels (3)
0 Karma

venkatasri
SplunkTrust
SplunkTrust

@fajri1203  Try search-time approach, UF is a universal forwarder not applicable for your case. I wonder Add-on Microsoft Cloud Services don't have the extraction by default. 

You have to configure props.conf, and transforms.conf on search-head (SH) under $SPLUNK_HOME/etc/<app_name>/local OR  $SPLUNK_HOME/etc/system/local. If you are having SH cluster and using SH deployer you must know how to bundle push or contact your splunk admin. 

In standalone splunk SH the restart is required post changes.

[mscs:storage:blob]

REPORT-extract-csv-fields = extract-csv-fields

[extract-csv-fields]
DELIMS=","
FIELDS = "pluginid","alertRef","alert","name","riskcode","confidence","riskdesc","confidencedesc","desc","instances","count","solution","otherinfo","reference","cweid","wascid","sourceid"

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @fajri1203  You don't need a regex you could do it two ways if .csv is being forwarded from UF then set

inside props.conf

[sourcetypename]

INDEXED_EXTRACTIONS = csv

At search-time, on search-head you shall do following

https://www.splunk.com/en_us/blog/tips-and-tricks/quick-n-dirty-delimited-data-sourcetypes-and-you.h...

0 Karma

fajri1203
Loves-to-Learn

Hi @venkatasri ,


Thanks for the quick reply. May I know what is UF?

 

Currently our data is being taken from Azure Storage Account using the plugin called "Splunk Add-on for Microsoft Cloud Services" as you can see at the below image:

 

image.png

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...