topic Re: SEDCMD extract multiple field from line in Getting Data In

SEDCMD extract multiple field from line

ryastrebov — Tue, 26 Mar 2013 08:07:54 GMT

Hello!

I have a log file containing a string with the values separated by commas. For Example:
,345,af04,,,01,78932223442,,,,,24,

08,532,bcc01,,,,,345s,,,,93,

Between , is _ symbol (,_,). I want to extract from all lines only certain fields. For example, field number 2,3,7,12.

Result is:

345,af04,78932223442,24

532,bcc01,_,93

Field for extraction can contain _ symbol.

How I can make it?

My knowledges about regular expression is very small...

Best Regards,
Roman

Re: SEDCMD extract multiple field from line

eashwar — Tue, 26 Mar 2013 09:15:20 GMT

hello i hope this will help you!!

| eval fieldname =",345,af04,,,01,78932223442,,,_,,24,_" | rex field=fieldname mode=sed "s/(,,,|,,)/,/g s/(^,)//g"

above worked for me in my splunk instance

add this line to your props.conf

SEDCMD-removecommas=s/(,,,|,,)/,/g s/(^,)//g

the above regular expression says if 3 commas or 2 commas found together replace with one comma. then find the coma in the line start and remove it.

yours,

eashwar raghunathan

Re: SEDCMD extract multiple field from line

ryastrebov — Tue, 26 Mar 2013 09:33:16 GMT

Hello eashwar!

Thank you for your help!

Unfortunately I need a little more...

I do not need to remove all empty values. I need to extract the field with certain numbers, regardless of whether there is anything in this field or not. For example,source string of the form:
1,2,3,4,5,6,7,8,9,0

I need to get in result string (extract field №2,№5,№9):
2,5,9

All field can contain a-z,A-Z,0-9 and _

It is a problem...

Re: SEDCMD extract multiple field from line

eashwar — Tue, 26 Mar 2013 09:46:28 GMT

hello there,
you if you can create a new thread with a sample event in it i can help you with your extract.
you may have to use EXTRACT-xxxxx in props.conf
or you can also use DELIMIT in

DELIMS = ","
FIELDS = "field1", "field2", "field3"
this is transforms.conf

consider voting for the answer above if the answer is correct and helped you.

thank you,
eashwar raghunathan

Re: SEDCMD extract multiple field from line

ryastrebov — Tue, 26 Mar 2013 10:19:18 GMT

Thank you for your answer!

I'll look for a solution to my problem using the TRANSFORMS

Re: SEDCMD extract multiple field from line

ryastrebov — Tue, 26 Mar 2013 10:46:24 GMT

And what of the presented method is recommended for performance? I need extract this fields before indexing...

Re: SEDCMD extract multiple field from line

eashwar — Tue, 26 Mar 2013 11:40:20 GMT

Extraction are done in index time and search time.

i am also a new to splunk.

if you call a transform.conf variable using REPORT form props.conf it will do the extraction in search time.

if you call a transforms.conf variable using TRANSFORMS from props.conf it will do the extraction in index time.

if you are using TRANSFORMS in your props.conf after the data is indexed make sure you do the following to reindex all the data.

step 1 : stop splunk ./splunk stop

step 2 : clean the index ./splunk clean eventdata <yourindexname>

step 3: start splunk ./splunk start

if you want to clean your full splunk instance replace step 2 by ./splunk clean all

hope this will help you,

yours,

eashwar raghunathan

happy splunking

Re: SEDCMD extract multiple field from line

ryastrebov — Tue, 26 Mar 2013 11:54:50 GMT

Thank you for your great help, eashwar!

You described approach is very good.

I am a new in regular expression and I have difficulties with write regex in transforms.conf file for my situation.

Can you help me with it?

Re: SEDCMD extract multiple field from line

eashwar — Tue, 26 Mar 2013 12:02:10 GMT

sure i can help you with it.
all you have to do is. update your question with one sample event. followed by field=value.
what value should be assigned to what field.
consider creating a new thread so that splunk guys dont get annoyed of hijacking threads.
happy splunking,
comment the link to the new thread hear
yours,
eashwar raghunathan