How do I write the regex to extract fields from another existing field?

[skender27]

Hi,

I need to extract a field from another field, no metadata fields.

The existing field (let's call it existing_field) has the following value:
class = 'blablabla' AND category = 'blablabla' AND ...

As you see the new two fields I need to extract are class and category and they are separated from AND.
What is the regex to extract them so I can add it to the .conf file?

Thanks,
Skender

[richgalloway · Accepted solution]

Try this:

... | rex field=existing_field "class = '(?P<class>\w+)' AND category = '(?P<category>\w+)'" | ...

[skender27]

It returns no errors but it doesn't work.
this regex is ok:
(?P\w+)\s\=\s\'\w+\s\w+\s\w+\'\sAND\s(?P\w+)

and here is a piece of sample data:
"existing_field": "class = 'Servizio...' AND category = 'Materiale...' AND ( ticket_type = 'Change Request' and ticket_impact_code = '2' ) AND ( ticket_type = 'Change Request' and ticket_urgency_code = '2' )"...

[richgalloway]

This regex will work if the fields contain only word characters. Try this as an alternative:

class = '(?P<class>[^ ]+)' AND category = '(?P<category>[^ ]+)'

[skender27]

And how about extracting entire strings (with white spaces included), not only words?

[richgalloway]

This regex extracts fields with spaces from your example.

class = '(?P<class>.*?)' AND category = '(?P<category>.*?)'

[skender27]

Thanks a lot:

I resolved it this way:

| rex field=existing_field  "class = (?P.*?) AND category = (?P.*?) AND"

Skender

[skender27]

I comfirm: the values are only word characters.

I tried this but I get no new fields extracted:
| rex field=sql_where_clause "class = '(?P[^ ]+)' AND category = '(?P[^ ]+)'"

should I cancel the extraction row I added in relative sourcetype in the props.conf?

Skender

[skender27]

sql_where_clause is the existing_field

[skender27]

I added in the sourcetype in my props.conf:

EXTRACT-my_extraction = (?P\w+)\s\=\s\'\w+\s\w+\s\w+\'\sAND\s(?P\w+) in existing_field

but I do not see the new fields yet...

[richgalloway]

You need to name the fields you are extracting (perhaps you did so and the editor dropped them). What's more, the capturing groups need to be around the right side of the equals sign or all you will capture is the field name.

[skender27]

Here is a sample of the data Iquinn:
"existing_field": "class = 'jhaskjdhsakjdhsakjdh' AND category = 'dhjkashdjkahdkajhdkaj' AND (hdsgahsdgasdgadgjjasgdhagdhasgd"...

as far as I know this is part of JSON data...

Thanks,
Skender

[skender27]

Ok I wrote this one and it works for the sample:

^(?P\w+)\s\=\s\'\w+\s\w+\s\w+\'\sAND(?P\s\w+)

Here you have the sample text:

NAME OF THE FIELD
existing_field_from_json
VALUE
class = 'kdjaldja' AND category = 'shdgahgdhadgjad' AND some_other_text_here... 

Now, how to put this regex to extract the information from the existing field?
And how to insert the eval stanza in the props.conf?

Skender

[DMohn]

You could use the rex command
your_search_here | rex field=existing_field "^.*\= '(?\w+)'.*\= '(?\w+)'.*$"

More details on the command can be found here: http://docs.splunk.com/Documentation/Splunk/6.2.5/SearchReference/rex

Otherwise try to use the graphical field extractor: http://docs.splunk.com/Documentation/Splunk/6.2.5/Knowledge/ExtractfieldsinteractivelywithIFX

[lquinn]

Can you give a sample of a whole event?

[skender27]

here is some sample text:

"existing_field: class = 'Service One...' AND category = 'Materials Two...' AND ( ticket_type = 'Change Request' and ticket_impact_code = '2' ) AND ( ticket_type = 'Change Request' and ticket_urgency_code = '2' )"...