topic field extraction from raw data in Splunk Search

field extraction from raw data

jatin_patel — Wed, 01 May 2013 20:26:49 GMT

Hi There,

I have below data that i will like to extract as key-value pair from a custom event source i have created.

for example i have

sourcetype=DBData

and in each result i have below data that i will like to extract. the data is in XML format

< Name > username1 < /Name >

< Name > username2 < /Name >

< Name > username3 < /Name >

I want to call Field Name as UersName and value as per above from the data.

I have tried the Interactive Field Extractor for this source type and added bunch of values as shown above but when i click on generate it give me below errors. I am not good at all with regex as of now. This sounds very simple task i am not sure why IFX is not able to take care of this.

Invalid search job/offset specified. [HTTP 404] https://127.0.0.1:8089/services/search/jobs/1367438277.12976; [{'text': 'Unknown sid.', 'type': 'FATAL', 'code': None}]. Defaulting to using values from the first result of the search string: ""
No regex could be learned. Try providing different examples or restriction.

Re: field extraction from raw data

bmacias84 — Wed, 01 May 2013 22:38:58 GMT

@jatin_patel, using the field extractor is the wrong way to go with this data type. Here you will want to use props.conf file for advanced configuration.



#props.conf

[DBData]

KV_MODE = xml #xml : automatically extracts fields from XML data

FIELDALIAS-Name = Name AS UserName #Use this to apply aliases to a field. The original field is not removed. This just means that the original field can be searched on using any of its aliases.

Read through the props.conf for more information. Propsconf

This should get your started and or help. Dont forget to vote up and accept anwsers that help.

Cheers,

Re: field extraction from raw data

jatin_patel — Thu, 02 May 2013 02:55:12 GMT

I tried but that is creating too many unwanted fields like over 5000. I only need few fields from the XML data. Search also becomes pretty slow.

Re: field extraction from raw data

Ayn — Thu, 02 May 2013 06:56:32 GMT

How about:

<Name>(?<UserName>[^<]+)

Re: field extraction from raw data

jatin_patel — Tue, 07 May 2013 17:50:48 GMT

sorry for late reply, but it did not work..

Re: field extraction from raw data

Ayn — Tue, 07 May 2013 19:21:08 GMT

Is your data formatted exactly as above, with spaces after and before the tags? In that case you need to change the regex accordingly.