topic Re: Host transforms not working in Getting Data In

Host transforms not working

edwardrose — Wed, 15 May 2019 18:24:02 GMT

Hello All,

I have the following props and transfroms

Props.conf

[host::splunk-sh1]
TRANSFORMS-vdisyslogs = set_host

Transforms.conf

[set_host]
REGEX = [ies|wv|inn].*.mentorg.com
DEST_KEY = MetaData:Host
FORMAT = host::$1

But the host value is set to $1 and not the ies|wv|inn.*.mentorg.com. It works when I run the following search:

index="remoteaccess" sourcetype="vdi:syslogs" 
| rex field=_raw "(?<host>[ies|wv|inn].*.mentorg.com)"

What do I have wrong and why is it wrong?

Thanks
ed

Re: Host transforms not working

ddrillic — Wed, 15 May 2019 20:28:55 GMT

Please try - REGEX = ([ies|wv|inn]).*.mentorg.com for the capture group.

Re: Host transforms not working

MuS — Wed, 15 May 2019 20:43:34 GMT

almost 😉

Set the capturing group to be ([ies|wv|inn].*.mentorg.com) to be used as $1

cheers, MuS

Re: Host transforms not working

edwardrose — Wed, 15 May 2019 20:44:00 GMT

Nope that did not work. The host field still shows up as $1

Re: Host transforms not working

koshyk — Wed, 15 May 2019 20:56:55 GMT

hi,

I could see two issues.

1) You regex may be too greedy sometimes (or incorrect). Please see regex sample on what all your regex will match https://regex101.com/r/xSWLH1/2 .

Better regex is : https://regex101.com/r/d5QXlN/2

2) Capture group is a MUST if you put FORMAT

 [set_host]
 REGEX = ([ies|wv|inn].*?\.mentorg\.com)
 DEST_KEY = MetaData:Host
 FORMAT = host::$1

Re: Host transforms not working

MuS — Wed, 15 May 2019 21:00:47 GMT

check for typos
did you restart Splunk after the change?
check $SPLUNK_HOME/bin/splunk btool props list --debug and $SPLUNK_HOME/bin/splunk btool transforms list --debug to see if your config is used
Make sure the host name in the props stanza matches the entire name, not just a substring (and it is case sensitive 😉 )
Should not be a show stopper, but in the regex use a \. to match a .

Beside that, out of ideas right now ¯\_(ツ)_/¯

Re: Host transforms not working

edwardrose — Wed, 15 May 2019 21:26:27 GMT

@MuS

Your answer was correct and worked.

Thanks
ed

Re: Host transforms not working

ddrillic — Wed, 15 May 2019 21:30:11 GMT

@MuS - why my cheerful REGEX = ([ies|wv|inn]).*.mentorg.com is broken? ; -) after all we want just ies or wv or inn?

Re: Host transforms not working

edwardrose — Wed, 15 May 2019 21:36:38 GMT

@ddrillic

I was looking for the entire FQDN that would start with ies, wv or inn.

thanks
ed

Re: Host transforms not working

MuS — Wed, 15 May 2019 21:37:27 GMT

Your regex is technically correct, but the example shows "(?<host>[ies|wv|inn].*.mentorg.com)" as regex where it will capture either its,wv, or inn followed by anything followed by mentors.com. In other words it captures the FQDN not just the host.

Does that make sense?

Re: Host transforms not working

ddrillic — Wed, 15 May 2019 21:41:58 GMT

Perfect @MuS ; -)

Re: Host transforms not working

ddrillic — Wed, 15 May 2019 21:43:20 GMT

Got it, great @edwardrose - good luck and keep us posted.

Re: Host transforms not working

MuS — Wed, 15 May 2019 22:00:50 GMT

Thanks, converted to answer - feel free to accept it 🙂

cheers, MuS