Getting Data In

After configuring props and transforms to extract the host from events, why are no results returned searching the host field?

schose
Contributor

Hi Forum,

i'm dealing with collectd data. This data generates events looking like this:

hostname.cpu-0.cpu-wait 0 1426497157
hostname.cpu-0.cpu-idle 99.25 1426497157
hostname.cpu-0.cpu-system 0.03 1426497157
hostname.cpu-0.cpu-nice 0 1426497157
hostname.cpu-0.cpu-user 0 1426497157

format it hostname.instance.metric value timestamp

As all data is piped into Splunk using udp import, I'm extracting the hostname from the event using props.conf and transforms.conf
write it to metadata:host.

props.conf:

[collectd]
TIME_PREFIX = ^.+\..+\..+\s.+\s
TRANSFORMS-mask= mask-collectd
SHOULD_LINEMERGE=false

transforms.conf

[mask-collectd]
REGEX = ^([^\.]+)\..+\..+\s.+\s.+
DEST_KEY = MetaData:Host
FORMAT = $1

All Data is located in Index collectd. When doing a search (e.g.: index=collectd) it shows me the hosts from the events at field hosts.
When doing a search with index=collectd host=* I don't get back any(!) results. Even when clicking the field and choosing a hostname.

when using the metadata command:

| metadata type=hosts index=collectd 

I get the hostnames from the event again.
For me it looks like only in some kind of metadata the hostname is rewritten. The search result shows the correct host field, but it's not usable. 😞

Any hints and best regards,

Andreas

1 Solution

somesoni2
Revered Legend

Try this in transforms.conf (props.can remain same)

[mask-collectd]
REGEX = ^([^\.]+)\..+\..+\s.+\s.+
DEST_KEY = MetaData:Host
FORMAT = host::$1

View solution in original post

somesoni2
Revered Legend

Try this in transforms.conf (props.can remain same)

[mask-collectd]
REGEX = ^([^\.]+)\..+\..+\s.+\s.+
DEST_KEY = MetaData:Host
FORMAT = host::$1

View solution in original post

nmohammed
Contributor
0 Karma

schose
Contributor

yes, perfect forgot the "host::" at FORMAT = ... thx, wouldn't find it for my own! 🙂

0 Karma

tachifelix
Path Finder

Use only props.conf in your app directory. try this following stanza:

[my_sourcefile]
EXTRACT-extract_hostname =( ?<hostname> ^([^.]+)..+..+s.+s.+)
0 Karma

schose
Contributor

yes, but then i would create a new field hostname. I want to replace the field "host".

background: the app is used in different enviroments. some environment have proper reverse dns resolution, some have not. For those who haven't i want to extract the hostname from the event. This is also done for sourcetype syslog in default.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!