I have following data file which I want to ingest as metrics data. Is it possible?
NodeName, IfIndex, datetime, ifInOctets, ifOutOctets, ifInErrors, ifOutErrors, locIfCarTrans, locIfInCRC, locIfOutputQueueDrops, locIfInputQueueDrops, IfName, IfType, IfSpeed, IfAlias
server1.com,066,20190821 13:01:10,0,350782,0,0,0,0,0,0,Gi2/42,ethernetCsmacd,1 Gbps, eth6-22.214.171.124
server2.com,051,20190821 13:01:10,0,0,0,0,0,0,0,0,Gi2/3,ethernetCsmacd,1 Gbps, Gig1.6-Private-intf
server3.com,111,20190821 13:01:25,0,0,0,0,0,0,0,0,Gi3/39,ethernetCsmacd,1 Gbps, ETH0 decom
server4.com,003,20190821 13:01:30,15179,4690,0,0,0,0,0,0,Gi0/1,ethernetCsmacd,40 Mbps, Gi0/2.3412 : MetroE 40M
Notice, datatime format is yyyymmdd HH:MM:SS format and its in 3rd column. NOT in first column.
metric names are ifInOctets, ifOutOctets, ifInErrors, ifOutErrors, locIfCarTrans, locIfInCRC, locIfOutputQueueDrops, locIfInputQueueDrops
remaining other fields NodeName, IfIndex, AND IfName, IfType, IfSpeed, IfAlias need to use as dimensions.
Your CSV is not in the expected format. See https://docs.splunk.com/Documentation/Splunk/7.3.1/Metrics/GetMetricsInOther#Get_metrics_in_from_fil....
Another method is to define a log-to-metric sourcetype. Go to Settings->Source types and create a new type. Be sure to choose "Log to Metrics" as the Category. Then click on the Metrics tab and enter your measures and dimensions. Next, create props and transforms for your sourcetype to extract the fields.
to simplify, I re-arranged columns as shown below with time stamp as first field, then metrics and then dimensions.
I created sourcetype logtomonitor and used ifOutOctets,ifInErrors,ifOutErrors,locIfCarTrans,locIfInCRC
,locIfOutputQueueDrops,locIfInputQueueDrops as metrics. I added monitor on splunk forwarder.
However, I am not seeing any data into metric index.
is this due to date_time format coming from log files?
20190821 17:53:10,300,348,0,0,0,0,0,0,server1.com,001,Gi0/0/0,ethernetCsmacd,40 Mbps,Link to zzz : Gi2/2/0.3421: MetroE 40M : Vlan3424 :
20190821 17:53:40,57611,23173,0,0,0,0,0,0,server2.com,001,Gi0/0/0,ethernetCsmacd,20 Mbps,Link to yyy : Vlan3596 : MetroE 20M : Vlan3596 : IPC
20190821 17:53:40,62129,36565,0,0,0,0,0,0,server3.net,001,Gi0/0/0,ethernetCsmacd,40 Mbps,BC : Link [w/Nano] to xxx : G0/0/10 : iLite
If you have
TIME_FORMAT = %Y%m%d %H:%M:%S in your props.conf file for that sourcetype then the date_time column should not be a factor. What are your props.conf and transforms.conf settings?
props.conf on indexer server
INDEXEDEXTRACTIONS = csv
LINEBREAKER = ([\r\n]+)
METRIC-SCHEMA-TRANSFORMS = metric-schema:5secpollerinterfacemetrics1566415584485
NOBINARYCHECK = true
category = Log to Metrics
description = 5secpollerinterfacemetrics
pulldowntype = 1
METRIC-SCHEMA-MEASURES = ifInOctets,ifOutOctets,ifInErrors,ifOutErrors,locIfCarTrans,locIfInCRC,locIfOutputQueueDrops,locIfInputQueueDrops
That looks like it would work. Verify the fields are extracted properly by commenting-out the
METRIC-SCHEMA-TRANSFORMS line in props.conf and sending the data to an events index.
It's a best practice to always include
TIME_FORMAT along with
TRUNCATE in all props.conf stanzas.