Getting Data In
Highlighted

How can I get my TCP data into the metrics index?

Path Finder

I am trying to pull incoming tcp data into the Metrics Store using this information:

http://docs.splunk.com/Documentation/Splunk/7.0.0/Metrics/GetMetricsInOther#Get_metrics_in_from_clie...

Currently the data is coming in to an event index and the _raw field looks like this:
2017-11-29 18:43:22.807 +0000 Tag="Channel1.Device1.Tag1" Value="1523" Quality="good"

We would like to pull this into the Metrics Store with these fields
metric_name Tag
_value Value
_time _time

(We also want the Quality, but I'm ignoring that until I get the basics.)

Created a new port for the data to come in.

Source Sourcexxxxx
Source Type Typexxxxx
index indexxxxxx

The index (indexxxxx) is a metrics index

I edit configuration files as below and restart Splunk. I get two errors:

Index Processor: Metric value= is not valid for source=Sourcexxxxx, sourcetype=Typexxxxx, host=Hostxxxxx, index=index51115. Metric event data with an invalid metric value would not be indexed. Ensure the input metric data is not malformed.

Index Processor: Metric name is missing for source=Sourcexxxxx, sourcetype=Typexxxxx, host=Hostxxxxx, index=indexxxxxx. Metric event data without metric name is invalid and would not be indexed. Ensure the input metric data is not malformed.

What am I doing wrong? Thanks in advance for your help.

props.conf

[Typexxxxx]
    TIME_FORMAT = %F %H:%M:%S.%Q %z
    TRANSFORMS-test_metric_name = test_name
    TRANSFORMS-test_metric_value = test_value
    NO_BINARY_CHECK = true
    SHOULD_LINEMERGE = false
    pulldown_type = 1
    category = Metrics

transforms.conf
[testname]
REGEX = "Channel1.Device1.(\S+)"
FORMAT = metric
name::$1
WRITE_META = true

[test_value]
    REGEX = Value="(\S+)"
    FORMAT = _value::$1
    WRITE_META = true

fields.conf
[metric_name]
INDEXED=true

    [_value]
    INDEXED=true
0 Karma
Highlighted

Re: How can I get my TCP data into the metrics index?

SplunkTrust
SplunkTrust

Hi walkerhound,

One thing I spot is that the docs mentions this value for the metricname FORMAT to be `FORMAT = metricname::graphite.$1maybe try to useFORMAT = metric_name::channel1.$1` in your case.

For the _value, try this regex Value=\"([^\"]+)\" or if you really have literary Value="1234" in your logs try this:

 [test_value]
 REGEX = (.+)
 SOURCE_KEY = Value
 FORMAT = _value::$1
 WRITE_META = true

Hope this helps ...

cheers, MuS

0 Karma
Highlighted

Re: How can I get my TCP data into the metrics index?

Path Finder

Thank you for the suggestions. I experimented with several varieties of your ideas but I continue to get the same two errors indicating that the metric name and metric values are missing or unset.

0 Karma
Highlighted

Re: How can I get my TCP data into the metrics index?

Path Finder

It turns out that my problem had to do with permissions. Splunk could not read the fields.conf file. When I corrected that problem, this worked.

0 Karma
Highlighted

Re: How can I get my TCP data into the metrics index?

SplunkTrust
SplunkTrust

Nice finding! How about writing that up as new answer and accept it so it can help other finding the right solution? I suspect you were doing this on a Windows Splunk instance?

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.