topic Re: Append Domain name at index time? in Splunk Search

Append Domain name at index time?

daniel333 — Wed, 20 Sep 2017 20:18:08 GMT

All,

I have logs coming in from /var/log/messages and /var/log/maillog which have the hostname not the FQDN. There is just too much change control and politics to get them fixed at the source. Looking for a way at index time to just make the correction.

Server names are well formed 12 characters ending in three numbers.

So I need to create a props.conf/transforms.conf on my indexer, just not sure what it will look like.

If host = .*\n\n\n then append mycompany.com

Any ideas what that might look like?

Re: Append Domain name at index time?

alemarzu — Wed, 20 Sep 2017 20:23:03 GMT

Are u listening syslog directly from the Syslog server or you have the syslogd writing to a file in your Splunk server ?

Re: Append Domain name at index time?

daniel333 — Wed, 20 Sep 2017 20:27:51 GMT

Splunk for Nix is installed. So it's being gathered locally.

Re: Append Domain name at index time?

DalJeanis — Wed, 20 Sep 2017 20:31:40 GMT

This should match any host that does not end with .com, and append mycompany.com onto the current value

[yourstanzaname]
SOURCE_KEY = MetaData:Host
DEST_KEY  = MetaData:Host
REGEX = (.*$)(?<!\.com$)
FORMAT = host::$1mycompany.com

...or...

REGEX = .*$(?<!\.com$)
FORMAT = host::$0mycompany.com

Updated, because you meant THAT Host.

Just kidding, I just forgot we were talking about an ultra special snowflake of a MetaData field.

https://answers.splunk.com/answers/492863/what-happens-if-dest-key-metadatahost.html

Updated one more time, to change \0 to $0 and \1 to $1 because contextually we're in a .conf file and not a rex in sed mode.

(sigh)

Re: Append Domain name at index time?

s2_splunk — Wed, 20 Sep 2017 20:37:12 GMT

This post will show you the general approach to modifying your metadata field 'host', adjust RegEx to match properly.

Re: Append Domain name at index time?

daniel333 — Wed, 20 Sep 2017 22:43:06 GMT

Doens't seem to be flying. Tried this config as well as swapping host for MetaData:Host in your DEST_Key. I feel like I am missing something key here.

#props.conf
[syslog]
TRANSFORMS-FIELDS = syslog_fix_fqdn

#transforms.conf
[syslog_fix_fqdn]
SOURCE_KEY = host
DEST_KEY  = host
REGEX = .*$(?<!\.com$)
FORMAT = \0ilovethecompany.com

Re: Append Domain name at index time?

DalJeanis — Thu, 21 Sep 2017 00:00:17 GMT

@daniel333 - try the new code.

Re: Append Domain name at index time?

daniel333 — Thu, 21 Sep 2017 01:10:17 GMT

Thanks for replying!

I tried the new code and I ended up with my hostnames getting changed to

\1mycompany.com

Not following how that capture should work there. How does \1 and \0 refer back to the above regex?

Re: Append Domain name at index time?

DalJeanis — Thu, 21 Sep 2017 18:19:05 GMT

@daniel333 - my bad. Just rechecked the docs, and those slashes should be $s in this location. Updated.

Re: Append Domain name at index time?

lfedak_splunk — Thu, 21 Sep 2017 23:10:38 GMT

Hey @daniel333 if they solved your problem, please don't forget to accept an answer! You can upvote posts as well. (Karma points will be awarded for either action.) Happy Splunking!