topic Re: Host field in Splunk Search

Host field

pfabrizi — Thu, 05 Oct 2017 17:44:58 GMT

one of my data sources has host field in the raw packet. However when we search the events the host field is the name of the forwarder. Where do I rename that? I do use a transform, so can it be done there on ingestion?

What would be the syntak? in the props.conf file?

Re: Host field

s2_splunk — Thu, 05 Oct 2017 17:50:50 GMT

Can you please review this part of our documentation and let us know if you run into any trouble doing it like that?

Re: Host field

DalJeanis — Thu, 05 Oct 2017 17:55:16 GMT

The words you were searching for are "override host"

You will put code stanzas in props.conf and transforms.conf.

Here's a pretty good answer...

https://answers.splunk.com/answers/91933/can-you-override-host-for-an-input.html

And for you it will look something like this...

props.conf

[sourcetype_stanza_name]
TRANSFORMS-host_rename = host_rename_stanza

transforms.conf:

[host_rename_stanza]
REGEX = some regex that finds the host=([^\s]+)\s
DEST_KEY = MetaData:Host
FORMAT = host::$1

Re: Host field

yannK — Tue, 29 Sep 2020 16:11:11 GMT

If the event does not have a host extraction, it will use the default host field in the forwarder inputs.conf.
Do a ./splunk btool inputs list --debug on the forwarder to figure

To change the host you can :
- if all your events fro one input are from the same host,
add on the inputs.conf, under the input stanza a setting host=myactualhostname
-if your host has to be extracted from the event itself, then use a props/transforms rule on the indexers (or the first heavy forwarder if any)

If you are in a syslog data situation, look at the default syslog sourcetypes, they try to do that at indextime.
Look at the defaults [syslog] definitions in $SPLUNK_HOME/etc/system/default/props.conf and [syslog-host] in $SPLUNK_HOME/etc/system/default/transforms.conf ) for examples.

Re: Host field

pfabrizi — Thu, 05 Oct 2017 19:27:05 GMT

so the 'host' in the raw packet is not splunk forwarder or host of the device sending the event. Here is an example:
|host=teemfsw1.spt.com,

this is the server and domain space of the offending event.

I was converting it to shost using this: #EXTRACT-shost = CEF:\s+\d(?:|[^|]+){6}|+host=(?[a-zA-Z0-9.-_]+)

But it sounds like host is a built in field for device sending to the indexer?

Re: Host field

pfabrizi — Tue, 29 Sep 2020 16:06:33 GMT

I get this stanza error:
Undocumented key used in transforms.conf; stanza='hostoverride' setting='DEST_KEY' key='MetData:Host'

Here is my transforms:
[hostoverride]
DEST_KEY = MetData:Host
REGEX = CEF:\s+\d(?:|[^|]+)(6)|+host=([a-zA-Z0-9.-_]+)
FORMAT = host::$1

Re: Host field

yannK — Thu, 05 Oct 2017 19:45:03 GMT

The required meta fields in splunk are :
source, sourcetype, host (we could consider index and _time to also be always populated)

If you want to use a field at search time for the "original offending host", maybe you could use a different name to distinguish is from the "event sender host" .
example with : offender_host

 EXTRACT-shost = CEF\:\s+\d(?:\|[^\|]+){6}\|+host=(?<offender_host>[a-zA-Z0-9.-_]+)

Re: Host field

s2_splunk — Thu, 05 Oct 2017 19:46:25 GMT

It's supposed to be MetaData::Host

you are missing an 'a' and a ':'

Re: Host field

pfabrizi — Tue, 29 Sep 2020 16:06:54 GMT

I fixed the missing 'a'. I also checked my regex in regex101 and found that was not correct, I validated my regex finds the value. It is still not giving me host as I want it.
2 questions:
1. I do the transforms at the forwarder and not the indexer?
2. I have 2 transforms. First is the host over ride and the second is to parse the event as it is a custom CEF formatted string. Below is my props.conf for this index.

[netwitness]
FIELDALIAS-severity_as_id = severity as severity_id
FIELDALIAS-dst_as_dest = dst as dest
EVAL-app = netwitness
EXTRACT-subject = CEF:\s+\d(?:|[^|]+){4}|(?[^|]+)
EXTRACT-shost = CEF:\s+\d(?:|[^|]+){6}|+host=(?[a-zA-Z0-9.-_]+)
TRANSFORMS-ho=hostoverride
TRANSFORMS = netwitness-extractions

Re: Host field

s2_splunk — Fri, 06 Oct 2017 20:36:37 GMT

props/transforms go where the parsing happens, i.e. the indexer. Unless you use a heavy forwarder, of course. But I am assuming you are collecting using a universal forwarder that sends directly to your indexer(s). If there is any intermediary forwarder in your forwarding chain that is a HEAVY forwarder, you need to put props/transforms there.

Oh, and I just noticed your second TRANSFORMS is not valid. Should be something like TRANSFORMS-net or sumsuch.
You can also write: TRANSFORMS-ho=hostoverride, netwitness-extractions

What does your transforms.conf look like now?
Can you share a sample event as well?

Re: Host field

pfabrizi — Sat, 07 Oct 2017 07:43:01 GMT

I am using a light forwarder to send the log to the indexer, so I am guessing that is a intermediary?

[hostoverride]
REGEX = host=([a-zA-Z0-9.-_]+)
DEST_KEY = MetaData:Host
FORMAT = host::$1

[netwitness-extractions]
REGEX = CEF:\d+|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|([^|])|
FORMAT = nullQueue

sample Event:

Re: Host field

yannK — Mon, 09 Oct 2017 17:08:41 GMT

"light forwarder" and "Universal forwarders" do not parse the events (except a few exceptions)
so the props/transforms has to be setup on your indexers (or on your first heavy forwarders is any are chained)

Re: Host field

pfabrizi — Tue, 10 Oct 2017 17:30:00 GMT

Thank You, That worked.