topic Re: Regex not appearing to work in Splunk Search

Regex not appearing to work

mhale1982 — Fri, 04 Jan 2013 01:58:31 GMT

I'm having some issues with using regex to define the host of some events from an ASA. The events are in the format below:

Jan 3 17:14:29 10.111.11.111 %ASA-6-302016: Teardown UDP connection 10101576 for external_untrusted:10.111.111.111/111 to external_untrusted:10.111.111.111/111 duration 0:00:00 bytes 0 (asdfasdf)

I'm using the following regex:

(\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)?

This works fine here: Regex pal

But my event still shows:

host=rel-splunk.roll.ad | sourcetype=cisco_asa | source=/var/log/syslog/asa

Any thoughts on what I'm doing wrong?

Re: Regex not appearing to work

Rob — Fri, 04 Jan 2013 02:06:16 GMT

Have you tried this regex with the Splunk 'rex' command?

Also, I think you may have left out a space after the IP address. (This is assuming that you are looking for the first IP address from your sample event that follows the timestamp.)

\b is a zero-width character assertion so your regex would be looking for the end of the IP address to have a new word immediately follow it which could become confusing depending on the regular expression engine.

Try using (\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s)? instead and see if that works out a bit better.

I still recommend testing your regex with the Splunk search language as it will give you a better idea of how the regex is interpreted. E.g.

|rex field=_raw "(?(?:\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s)?)"

Re: Regex not appearing to work

mloven_splunk — Fri, 04 Jan 2013 02:47:43 GMT

mhale1982, I'm assuming that you want to use the field immediately following the timestamp as your host value?

Try this regex with the rex command that Rob mentioned above. So, in your case, it would be something like:

your search goes here | rex ^(\S+\s+){3}(?[^\s]+)

And see if a field called "aaaa" gets extracted. If so, the regex is good, and you can slap it into your host override.

HTH

Re: Regex not appearing to work

Rob — Fri, 04 Jan 2013 02:51:38 GMT

Nice regex!

It also works well since if the first IP is the one we are looking for then the space character serves as a nice anchor. And it made me realize that my slashes are missing in my previous answer.

Re: Regex not appearing to work

mloven_splunk — Fri, 04 Jan 2013 03:03:39 GMT

ah, weird, mine are too... maybe some odd formatting going on...
\test

Re: Regex not appearing to work

mloven_splunk — Fri, 04 Jan 2013 03:07:20 GMT

So mhale1982, there's some odd formatting going on that seems to be eating the backslashes from our responses, so, for clarity, I'll try to pop it into a code block to see if it accepts it, but there should be a backslash preceding each of the "s" or "S" characters.

your search goes here | rex ^(\S+\s+){3}(?<aaaa>[^\s]+)

Re: Regex not appearing to work

mhale1982 — Fri, 04 Jan 2013 18:01:27 GMT

All,

Thanks for your replies so far. I'm getting some issues running the expression using rex field=_raw. It does seem to give me some results if I use regex _raw=<expression>. I haven't tried using this in my data input section yet...that'll happen later today. Will report back on how it works.

Thanks again!

Re: Regex not appearing to work

Rob — Fri, 04 Jan 2013 18:26:26 GMT

In case you haven't already checked it out, here is the docs link for rex: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Rex

For regex, its a little different:
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Regex

The syntax is different for the two as you have noted as the rex command is used for extracting a field and the regex command is used for filtering results.

Re: Regex not appearing to work

mhale1982 — Fri, 04 Jan 2013 22:37:53 GMT

I just realized that I was doing something Splunk isn't designed for.

I was attempting to modify the 'hosts' value based on entries within the log. What I failed to notice on the data input screen, the regex is based on the path of the file, not the contents.

I've since modified rsyslog to log to files with the IP address as the name of the file and used the following regex to generate the host IP: (\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)

Thanks again for the assistance and the info on regex and rex. That'll be helpful moving forward.