Getting Data In

SEDCMD extract multiple field from line

ryastrebov
Communicator

Hello!

I have a log file containing a string with the values ​​separated by commas. For Example:
,345,af04,,,01,78932223442,,,,,24,

08,532,bcc01,,,,,345s,,,,93,

Between , is _ symbol (,_,). I want to extract from all lines only certain fields. For example, field number 2,3,7,12.

Result is:

345,af04,78932223442,24

532,bcc01,_,93

Field for extraction can contain _ symbol.

How I can make it?

My knowledges about regular expression is very small...

Best Regards,
Roman

Tags (2)
0 Karma
1 Solution

eashwar
Communicator

hello i hope this will help you!!

| eval fieldname =",345,af04,,,01,78932223442,,,_,,24,_" | rex field=fieldname mode=sed "s/(,,,|,,)/,/g s/(^,)//g"

above worked for me in my splunk instance

add this line to your props.conf

SEDCMD-removecommas=s/(,,,|,,)/,/g s/(^,)//g 

the above regular expression says if 3 commas or 2 commas found together replace with one comma. then find the coma in the line start and remove it.

yours,

eashwar raghunathan

View solution in original post

eashwar
Communicator

Extraction are done in index time and search time.

i am also a new to splunk.

if you call a transform.conf variable using REPORT form props.conf it will do the extraction in search time.

if you call a transforms.conf variable using TRANSFORMS from props.conf it will do the extraction in index time.

if you are using TRANSFORMS in your props.conf after the data is indexed make sure you do the following to reindex all the data.

step 1 : stop splunk ./splunk stop

step 2 : clean the index ./splunk clean eventdata <yourindexname>

step 3: start splunk ./splunk start

if you want to clean your full splunk instance replace step 2 by ./splunk clean all

hope this will help you,

yours,

eashwar raghunathan

happy splunking

0 Karma

eashwar
Communicator

sure i can help you with it.
all you have to do is. update your question with one sample event. followed by field=value.
what value should be assigned to what field.
consider creating a new thread so that splunk guys dont get annoyed of hijacking threads.
happy splunking,
comment the link to the new thread hear
yours,
eashwar raghunathan

0 Karma

ryastrebov
Communicator

Thank you for your great help, eashwar!

You described approach is very good.

I am a new in regular expression and I have difficulties with write regex in transforms.conf file for my situation.

Can you help me with it?

0 Karma

eashwar
Communicator

hello i hope this will help you!!

| eval fieldname =",345,af04,,,01,78932223442,,,_,,24,_" | rex field=fieldname mode=sed "s/(,,,|,,)/,/g s/(^,)//g"

above worked for me in my splunk instance

add this line to your props.conf

SEDCMD-removecommas=s/(,,,|,,)/,/g s/(^,)//g 

the above regular expression says if 3 commas or 2 commas found together replace with one comma. then find the coma in the line start and remove it.

yours,

eashwar raghunathan

ryastrebov
Communicator

And what of the presented method is recommended for performance? I need extract this fields before indexing...

0 Karma

ryastrebov
Communicator

Thank you for your answer!

I'll look for a solution to my problem using the TRANSFORMS

0 Karma

eashwar
Communicator

hello there,
you if you can create a new thread with a sample event in it i can help you with your extract.
you may have to use EXTRACT-xxxxx in props.conf
or you can also use DELIMIT in

DELIMS = ","
FIELDS = "field1", "field2", "field3"
this is transforms.conf

consider voting for the answer above if the answer is correct and helped you.

thank you,
eashwar raghunathan

0 Karma

ryastrebov
Communicator

Hello eashwar!

Thank you for your help!

Unfortunately I need a little more...

I do not need to remove all empty values. I need to extract the field with certain numbers, regardless of whether there is anything in this field or not. For example,source string of the form:
1,2,3,4,5,6,7,8,9,0

I need to get in result string (extract field №2,№5,№9):
2,5,9

All field can contain a-z,A-Z,0-9 and _

It is a problem...

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...