Splunk Search

Multi Line field extraction

Path Finder

Trying to do an expression that would extract IP's that are below the Client IP: line. Im looking to pull out each IP that is separated by a comma individually. In my raw event log there is a blank line above the Client IP line and also below the line that gives the IP addresses. Its possible I can have more than one or two IP's per log.

Client IP:
0.0.0.0,0.0.0.0

I also have another field in the same log that gives a username. There is a blank line above the error message and below the username. I was looking to extract everything after the : and before the -The user name.
I tried UserName="Error message:(/w)-The user name or password is incorrect"

Error message:
username@test.com-The user name or password is incorrect

0 Karma

Esteemed Legend

Like this:

... | rex "(?ms)Client\s+IP:[\r\n\s]*(?<ClientIPs>[\d\.\,]+)" | rex "(?ms)Error\s+message:[\r\n\s]*(?<EmailAddress>[^\s\-]+)"
0 Karma

Contributor

From time to time I run into an issue extracting fields from _raw with Windows logs containing new lines, but sometimes what I'm looking for is in the Message field bunched together too. If that's the case for you. Give this a try:

 | rex field=Message "(IP\:\s+|\d+\,)(?P<ClientIP>.[^\s]*)"
 | makemv delim="," ClientIP 

This should extract the IPs and separate them into their own values. If you don't want the IPs as their own values, remove the makemv portion of it.

Also, someone posted a log the other day from an ADFS log where the Client IP had a . at the end of the second IP. If that appears in your events, give this a try:

 | rex field=Message "(IP\:\s+|\d+\,)(?P<ClientIP>.*)."
 | makemv delim="," ClientIP 

For the username, give one of these a try:

| rex field=Message "message\:\s(?P<username>.*)-"

or

  | rex field=Message "message\:\s(?P<username>.[\s]*)"

or

 | rex field=Message "message\:\s(?P<username>.[\-]*)"

If none of that works, would you mind providing a sanitized sample of the entire log?

0 Karma

Legend

Are using trying the rex command or IFX?
In either case you can turn on Access Flag (?s), due to the same \s+ will match newline characters also. Following is an example:

rex field=_raw "(?s)Error message:\s+(?<Username>.*)-The user name or password is incorrect" 

Both rex and IFX worked for me... If Rex is not working for you please try Interactive Field Extraction in Splunk and let Splunk come up with required regular expression. Following is the regular expression IFX came up with ^(?:[^ \n]* ){4}\s+(?P<Username>[^\-]+). You can also try this instead in rex to test.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Path Finder

Just tried running this to extract but did not get any results. The \s+ command is telling my extraction to match one or more times to whitespace which means go from one line to the next line until you hit your first character thats not whitespace correct?

0 Karma

Legend

Yes... I tried to attach image to my answer since both rex as well as IFX worked for me. However, I am unable to do so... I have added regular expression that IFX came up with to my answer. Please test that in rex. If not Try Field Extraction yourself.

One of the other option would be to remove newline character from your _raw events using eval with replace, but that will be too expensive.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Path Finder

Not sure whats going on with my extract. I tried the below and got results the first time through extract preview with just the username@test.com. I put in the extraction for all users and when going back in the search the field wasent there. I deleted the extract and readded it and its not finding results again.

^(?:[^ \n]* ){4}\s+(?P[^-]+)

0 Karma

Legend

Using Extract fields method in your Splunk Search you can create Field Extraction yourself using Regular Expression on your data. Make sure there are no non-matches in your selected events or you would need to adjust the regular expression by including non-matched events.

Follow the documentation to come up with your field extraction: http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/ExtractfieldsinteractivelywithIFX

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma