Splunk Search

How to extract and create multiple fields for the same log line

vnarapuram
Explorer

If I have the log line:

WEB 1.1.1.1/2.2.2.2/3.3.3.3
and I want to use extract fields to map:
WEB -> field1
1.1.1.1/2.2.2.2/3.3.3.3 -> field2
1.1.1.1 -> field3

How do I do this?
I am able to extract field1 and field2 but not field3.

0 Karma
1 Solution

jpolvino
Builder

Leveraging what aberkow wrote, you could just do 2 rexes:

| makeresults count=1
| eval _raw="Jun 30 14:47:09 1.1.1.1/2.2.2.2/3.3.3.3 WEB: 14:47:09,000XXBB,SYSTEM"
| rex field=_raw "^(?<field1>\w+\s+\d+\s+\d+:\d+:\d+)\s(?<field2>[\S]+)\s(?<field3>\w+)"
| rex field=field2 "(?<field4>[^/]+)"

Or if you REALLY REALLY want it to be one regex:

| makeresults count=1
| eval _raw="Jun 30 14:47:09 1.1.1.1/2.2.2.2/3.3.3.3 WEB: 14:47:09,000XXBB,SYSTEM"
| rex field=_raw "^(?<field1>\w+\s+\d+\s+\d+:\d+:\d+)\s(?<field2>(?<field4>\d+\.\d+\.\d+\.\d+)(/\d+\.\d+\.\d+\.\d+){2})\s(?<field3>\w+)"

The first way is much more readable in my opinion!

View solution in original post

jpolvino
Builder

Leveraging what aberkow wrote, you could just do 2 rexes:

| makeresults count=1
| eval _raw="Jun 30 14:47:09 1.1.1.1/2.2.2.2/3.3.3.3 WEB: 14:47:09,000XXBB,SYSTEM"
| rex field=_raw "^(?<field1>\w+\s+\d+\s+\d+:\d+:\d+)\s(?<field2>[\S]+)\s(?<field3>\w+)"
| rex field=field2 "(?<field4>[^/]+)"

Or if you REALLY REALLY want it to be one regex:

| makeresults count=1
| eval _raw="Jun 30 14:47:09 1.1.1.1/2.2.2.2/3.3.3.3 WEB: 14:47:09,000XXBB,SYSTEM"
| rex field=_raw "^(?<field1>\w+\s+\d+\s+\d+:\d+:\d+)\s(?<field2>(?<field4>\d+\.\d+\.\d+\.\d+)(/\d+\.\d+\.\d+\.\d+){2})\s(?<field3>\w+)"

The first way is much more readable in my opinion!

View solution in original post

vnarapuram
Explorer

Thanks a lot. That works.
I know it's off-topic but do you know by any chance how to create another additional field for every log line. So, in this case, along with the field1 to field4, I also want field5 to set to 'Palo Ato' for all log lines. I am not good at Regex stuff. Do you know how to do in a single regex?
Appreciate your response in advance. Thanks again.

0 Karma

aberkow
Builder

| eval field5 = "Palo ato"? Or is Palo Ato a different field?

vnarapuram
Explorer

so, that would look like:
^(?\w+\s+\d+\s+\d+:\d+:\d+)\s(?(?\d+.\d+.\d+.\d+)(/\d+.\d+.\d+.\d+){2})\s(?\w+)|eval field5 = "Palo Alto"
?

0 Karma

aberkow
Builder

If you're trying to do this through a regex command, you can do it with something like this:

| makeresults count=1
| eval _raw="WEB 1.1.1.1/2.2.2.2/3.3.3.3"
| rex field=_raw "(?<field1>.*)\s(?<field2>.*)"
| rex field=_raw ".*\s(?<field3>.*?)\/.*"

As this will allow you to run multiple regex commands over the same code and extract fields as it finds them. In this example, I'm taking everything before the whitespace as "field1", everything after as "field2", and then another cut is everything after the whitespace before the first slash. Less clear if you're trying to run an extraction on index or at search time.

Hope this helps!

vnarapuram
Explorer

Thanks!
My actual log looks like this:
Jun 30 14:47:09 1.1.1.1/2.2.2.2/3.3.3.3 WEB: 14:47:09,000XXBB,SYSTEM

And the regex splunk generated for me is:
^(?P\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P[^ ]+)\s+(?P\w+)

This regex split my log line in to:
field1 -> Jun 30 14:47:09
field2 -> 1.1.1.1/2.2.2.2/3.3.3.3
field3 -> WEB

Here, if I want to include in same regex to generate field4 with value 1.1.1.1, how would I do that?
Appreciate your response in advance. Thanks!

0 Karma

aberkow
Builder

I don't know if it's possible to extract two things out of one line. In this case, I would just run this in your actual Splunk search:

| rex field=field2 ".*\s(?<field4>.*?)\/.*"

As you can take any field in to apply a regex command on, not just _raw. https://docs.splunk.com/Documentation/Splunk/8.0.0/SearchReference/Rex#Examples

0 Karma

jpolvino
Builder

It is possible to nest extracts on one line, but it makes readability worse. See possible answer below.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!