I am trying to pull incoming tcp data into the Metrics Store using this information:
Currently the data is coming in to an event index and the _raw field looks like this:
2017-11-29 18:43:22.807 +0000 Tag="Channel1.Device1.Tag1" Value="1523" Quality="good"
We would like to pull this into the Metrics Store with these fields
metric_name Tag
_value Value
_time _time
(We also want the Quality, but I'm ignoring that until I get the basics.)
Created a new port for the data to come in.
Source Sourcexxxxx
Source Type Typexxxxx
index indexxxxxx
The index (indexxxxx) is a metrics index
I edit configuration files as below and restart Splunk. I get two errors:
Index Processor: Metric value= is not valid for source=Sourcexxxxx, sourcetype=Typexxxxx, host=Hostxxxxx, index=index51115. Metric event data with an invalid metric value would not be indexed. Ensure the input metric data is not malformed.
Index Processor: Metric name is missing for source=Sourcexxxxx, sourcetype=Typexxxxx, host=Hostxxxxx, index=indexxxxxx. Metric event data without metric name is invalid and would not be indexed. Ensure the input metric data is not malformed.
What am I doing wrong? Thanks in advance for your help.
props.conf
[Typexxxxx]
TIME_FORMAT = %F %H:%M:%S.%Q %z
TRANSFORMS-test_metric_name = test_name
TRANSFORMS-test_metric_value = test_value
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
pulldown_type = 1
category = Metrics
transforms.conf
[test_name]
REGEX = "Channel1.Device1.(\S+)"
FORMAT = metric_name::$1
WRITE_META = true
[test_value]
REGEX = Value="(\S+)"
FORMAT = _value::$1
WRITE_META = true
fields.conf
[metric_name]
INDEXED=true
[_value]
INDEXED=true
Hi walkerhound,
One thing I spot is that the docs mentions this value for the metric_name FORMAT
to be FORMAT = metric_name::graphite.$1
maybe try to use FORMAT = metric_name::channel1.$1
in your case.
For the _value, try this regex Value=\"([^\"]+)\"
or if you really have literary Value="1234"
in your logs try this:
[test_value]
REGEX = (.+)
SOURCE_KEY = Value
FORMAT = _value::$1
WRITE_META = true
Hope this helps ...
cheers, MuS
It turns out that my problem had to do with permissions. Splunk could not read the fields.conf file. When I corrected that problem, this worked.
Nice finding! How about writing that up as new answer and accept it so it can help other finding the right solution? I suspect you were doing this on a Windows Splunk instance?
Thank you for the suggestions. I experimented with several varieties of your ideas but I continue to get the same two errors indicating that the metric name and metric values are missing or unset.