topic Re: host regex does not seem to work in Splunk Search

host regex does not seem to work

stefan_radovano — Wed, 28 May 2014 08:19:14 GMT

Hello all,

I am new to Splunk and I am currently evaluating 6.1. We collect logs from a bunch of devices (routersand switches) to a central syslog server (syslog-ng) and currently splunk runs on this server. I am trying to get it to detect the hostname of the device from the log filename but I can't seem to get it to work.

I went through a lot of the questions already posted here and it seems to me what I am doing should work, but it doesn't.

This is the entry I have in /apps/search/local/inputs.conf:

[monitor:///data/log/Core/*]
blacklist = \.(gz|bz2|z|zip|\d)$
disabled = false
followTail = 0
host =
whitelist = \.cnt.int.log$
host_regex = ^/data/log/Core/(.*)\.cnt\.int\.log$
sourcetype = cisco:ios

(this was added by the web gui)

The files look like this:

/data/log/Core/router1.cnt.int.log
/data/log/Core/router2.cnt.int.log
/data/log/Core/router3.cnt.int.log
/data/log/Core/router4.cnt.int.log
/data/log/Core/router4.cnt.int.log.1
/data/log/Core/router4.cnt.int.log.2.gz
/data/log/Core/router4.cnt.int.log.3.gz
/data/log/Core/router5.cnt.int.log
/data/log/Core/router6.cnt.int.log
/data/log/Core/router7.cnt.int.log

The regex looks fine to me, it checks out ok in RegExr. Despite all this, when I go to the Web gui, search and click on Data summary, I only see the syslog server hostname. There is none of those router1, router2 and so on hostnames which I expected to see.

Any idea why this is not working ?

Regards,
Stefan

Re: host regex does not seem to work

richgalloway — Wed, 28 May 2014 12:04:53 GMT

I did not get a match in RegExr using your regex string and your sample file names. I had better luck with

\/data\/log\/Core\/(.*)\.cnt\.int\.log

Re: host regex does not seem to work

stefan_radovano — Wed, 28 May 2014 12:24:34 GMT

Unfortunately it's not working either. I don't think the escape is needed to be honest. For example, I can type this into the search bar:

index=main | rex field=source ^/data/log/Core/(?.*).cnt.int.log$

and it produces entries with the host extracted correctly, so the regex is fine. I just don't understand why it's not being applied on indexing.

Re: host regex does not seem to work

stefan_radovano — Wed, 28 May 2014 12:25:52 GMT

The command above actually contains backslashes behind the dots at the end, they are just removed by this site apparently.

Re: host regex does not seem to work

richgalloway — Wed, 28 May 2014 12:30:39 GMT

When I tried your regex in RegExr, I did not get a match until I removed the anchor tags (^$). Have you tried that?

Re: host regex does not seem to work

stefan_radovano — Wed, 28 May 2014 12:38:23 GMT

Well, this is weird, it works when I remove the anchor tags but I could SWEAR that I tried without too. And I am pretty sure I've seen examples in here with people using anchor tags. In any case, thank you!