Splunk Search

How to remove double quotes from events ?

sivakumargik
New Member

sample event

"USR_LOGIN","USR_EMP_NO","USR_LAST_NAME","USR_FIRST_NAME","USR_DISPLAY_NAME","USR_STATUS","USR_EMAIL","USR_TRANSIT","USR_EMPLOYEEMANAGER","USR_IDENTITYSECURITYID","USR_UDF_EMPLOYER","USR_EMPLOYERCODE"

Tags (1)
0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval raw="\"USR_LOGIN\",\"USR_EMP_NO\",\"USR_LAST_NAME\",\"USR_FIRST_NAME\",\"USR_DISPLAY_NAME\",\"USR_STATUS\",\"USR_EMAIL\",\"USR_TRANSIT\",\"USR_EMPLOYEEMANAGER\",\"USR_IDENTITYSECURITYID\",\"USR_UDF_EMPLOYER\",\"USR_EMPLOYERCODE\"" 
| rename COMMENT AS "this is sample data" 
| makemv delim="," raw 
| mvexpand raw 
| eval tmp=1 
| xyseries tmp raw _time 
| fields - tmp 
| rename COMMENT AS "this is sample data" 
| rename \"*\" as *

I tried to remove double quotes for field names.

0 Karma

mayurr98
Super Champion

the easiest way to do at index time is by using SEDCMD script:
This is run anywhere search to test the script:

| makeresults 
| eval _raw="\"USR_LOGIN\",\"USR_EMP_NO\",\"USR_LAST_NAME\",\"USR_FIRST_NAME\",\"USR_DISPLAY_NAME\",\"USR_STATUS\",\"USR_EMAIL\",\"USR_TRANSIT\",\"USR_EMPLOYEEMANAGER\",\"USR_IDENTITYSECURITYID\",\"USR_UDF_EMPLOYER\",\"USR_EMPLOYERCODE\"" 
| rex mode=sed "s/\"(\w+)\"/\1/g"

You would need to do this using CLI:

1) On the machine that runs Splunk Enterprise, create a props.conf in the $SPLUNK_HOME/etc/system/local directory. If the file already exists, proceed to the next step.
2) Open $SPLUNK_HOME/etc/system/local/props.conf with a text editor.
3) Add the following stanza to reference the transform that you created in inputs.conf to do the masking transformation.

[your_sourcetype]
SEDCMD-remove_dquotes= s/\"(\w+)\"/\1/g

Save the file and close it.
Restart Splunk Enterprise.
0 Karma

rajashaey
Explorer

Hi,

please specify what to insert in Transforms.conf

0 Karma

sivakumargik
New Member

Hey Mayur,

The data is already ingested. I would need to do this in search time.

and the below is just the field names but there are around 100k events with actual data in which i need to extract the data without the double quotes

"USR_LOGIN","USR_EMP_NO","USR_LAST_NAME","USR_FIRST_NAME","USR_DISPLAY_NAME","USR_STATUS","USR_EMAIL","USR_TRANSIT","USR_EMPLOYEEMANAGER","USR_IDENTITYSECURITYID","USR_UDF_EMPLOYER","USR_EMPLOYERCODE"

0 Karma

to4kawa
Ultra Champion
| makeresults
| eval _raw="\"USR_LOGIN\",\"USR_EMP_NO\",\"USR_LAST_NAME\",\"USR_FIRST_NAME\",\"USR_DISPLAY_NAME\",\"USR_STATUS\",\"USR_EMAIL\",\"USR_TRANSIT\",\"USR_EMPLOYEEMANAGER\",\"USR_IDENTITYSECURITYID\",\"USR_UDF_EMPLOYER\",\"USR_EMPLOYERCODE\""
| rename COMMENT AS "this is sample data"
| eval _raw=replace(_raw,"\"","")

Hi, how about it?

0 Karma

sivakumargik
New Member

the below is just the field names but there are around 100k events with actual data in which i need to extract the data without the double quotes

"USR_LOGIN","USR_EMP_NO","USR_LAST_NAME","USR_FIRST_NAME","USR_DISPLAY_NAME","USR_STATUS","USR_EMAIL","USR_TRANSIT","USR_EMPLOYEEMANAGER","USR_IDENTITYSECURITYID","USR_UDF_EMPLOYER","USR_EMPLOYERCODE"

0 Karma
Get Updates on the Splunk Community!

Fall Into Learning with New Splunk Education Courses

Every month, Splunk Education releases new courses to help you branch out, strengthen your data science roots, ...

Super Optimize your Splunk Stats Searches: Unlocking the Power of tstats, TERM, and ...

By Martin Hettervik, Senior Consultant and Team Leader at Accelerate at Iver, Splunk MVPThe stats command is ...

How Splunk Observability Cloud Prevented a Major Payment Crisis in Minutes

Your bank's payment processing system is humming along during a busy afternoon, handling millions in hourly ...