topic Re: Field delimitation using character position in Splunk Search

Field delimitation using character position

changux — Wed, 21 Sep 2016 03:17:30 GMT

Hi all.

I have some log files like this:

265964455 00000000000000028000000002Fuerza      R              1     0000000100

Field delimitation rules are (in this case, i have similar logs with other character distribution):

First 10 characters or 265964455 = FIELD1 
Next 18 characters or 000000000000000280 = FIELD2
Next 8 characters or 00000002 = FIELD3
Next 12 charcaters or Fuerza    = FIELD4
...

I tried with:

sourcetype=rsrs | rex field=MultiField "(?<FIELD1>.{10}) (?<FIELD2>.{18}) (?<FIELD3>.{8}) (?<FIELD4>.{12}) ..."

But didn't work. Anyone can help me please?

Re: Field delimitation using character position

sundareshr — Wed, 21 Sep 2016 03:56:38 GMT

Try this

sourcetype=rsrs | rex  "(?<field1>\w{1,10})\s*(?<field2>\w{1,18})\s*(?<field3>\d{1,8})\s*(?<field4>\w{1,12})\s*"

Re: Field delimitation using character position

changux — Wed, 21 Sep 2016 12:14:11 GMT

Works great!
Thanks!
How i can do permanent it? Sorry for disturb.

Re: Field delimitation using character position

inventsekar — Wed, 21 Sep 2016 12:27:23 GMT

permanent it meaning? learning rex?

Re: Field delimitation using character position

changux — Wed, 21 Sep 2016 12:29:05 GMT

Thanks. I mean, putting correctly in props.conf. Do you can please help me with the stanza? I don't know if i need transforms.conf also.

Re: Field delimitation using character position

sundareshr — Wed, 21 Sep 2016 13:10:50 GMT

You can use this regex in the Field Extraction UI (IFX) OR Add this to your props under the appropriate sourcetype stanza

EXTRACT-fields = (?<field1>\w{1,10})\s*(?<field2>\w{1,18})\s*(?<field3>\d{1,8})\s*(?<field4>\w{1,12})

Re: Field delimitation using character position

changux — Wed, 21 Sep 2016 13:43:13 GMT

Thanks a lot. If some field contains symbols, for example -, how i must catch?

Data:

000000000001614636IObser4AI-TP

Command:

 | rex "(?<FIELD1>\w{1,10})\s*(?<FIELD2>\w{1,18})\s*(?<FIELD3>\w{1,6})\s*(?<FIELD4>\w{1,6})\s*"

FIELD4 returns 4AI and should be 4AI-TP. The symbol - is a non word character and the expression stops.

Re: Field delimitation using character position

changux — Wed, 21 Sep 2016 13:55:42 GMT

Self answered, \S{w1,6} works! Thanks again!

Re: Field delimitation using character position

aakwah — Wed, 21 Sep 2016 14:32:28 GMT

Hello,

In props.conf add the following:

[rsrs]
REPORT-extraction = field_extraction

transforms.conf

[field_extraction]
REGEX = (?<field1>\w{1,10})\s*(?<field2>\w{1,18})\s*(?<field3>\d{1,8})\s*(?<field4>\w{1,12})\s*

Regards

Re: Field delimitation using character position

changux — Wed, 21 Sep 2016 15:57:08 GMT

Thank you!

Re: Field delimitation using character position

changux — Wed, 21 Sep 2016 20:39:24 GMT

Last question, how capture the 6 digits/numbers/letters/spaces or whatever.

I tried using rex with \S, \V and \X but doesn't work.

Data looks like:

https://www.dropbox.com/s/t5lg8tkzt1prje0/sample.txt?raw=1

And my rex expression:

... | rex "(?<DATA_ID_NRO_DATA>\S{1,10})\s* ..."

One of my problems is with the DATA_LUG_EXPfield that must be return empty (character positions: 141-147 in this line are empty) and returns the value of the next field DATA_ID_COD_RES, but in general, the data is very complex to extract it perfectly 😞

Re: Field delimitation using character position

changux — Wed, 21 Sep 2016 21:29:20 GMT

@sundareshr any idea to solve this? The data string has spaces, big text sequences and well, i don't know how proceed.

Thanks!

Re: Field delimitation using character position

sundareshr — Tue, 29 Sep 2020 11:06:58 GMT

Not sure I understand. In the data example you posted, what would the value for DATA_LUG_EXP be if extracted correctly

Re: Field delimitation using character position

changux — Thu, 22 Sep 2016 12:50:34 GMT

No, in the position 141 you doesn't have data, so, the field must be null. I can publish a small dataset to explain in a better way, ok?