Props.conf Multi Extraction Not working

[daniel333]

Hello,

Props.conf is not functioning like I expect. I have a scripted input that looks like this

CONTAINERID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
0dce14da9952        centos              "/bin/bash -c 'while "   16 minutes ago      Up 16 minutes                           grave_mayer
5969bffa8ad2        centos              "/bin/bash -c 'while "   46 minutes ago      Up 46 minutes                           distracted_jepsen

So I created a props.conf looking like this

[docker:ps]
SEDCMD-fixspace1= s/CONTAINER ID/CONTAINERID/g
SHOULD_LINEMERGE=false
LINE_BREAKER=(^$|[\r\n]+[\r\n]+)
TRUNCATE=1000000
DATETIME_CONFIG = CURRENT
KV_MODE = multi

Any idea why I am not getting an extraction for this?

[jkat54]

You're not specifying any key values to extract, just the mode of multi.

You'll nee more work on your props... something like this:

 [docker:ps]
 SHOULD_LINEMERGE=false
 LINE_BREAKER=(^$|[\r\n]+[\r\n]+)
 TRUNCATE=1000000
 DATETIME_CONFIG = CURRENT
 SEDCMD-removefirstline= s/CONTAINER.*$//g
 SEDCMD-extractFields = ^(?<containerid>\w+)\s+(?<image>\w+)\s+"(?<command>.+)"\s+(?<created>.+) ago\s+(?<status>.+)\s(minutes|hours|days|weeks|months|years)\s+(?<ports>\S+)\s+(?<names>.*)$

You'll note we removed the "header" / first line completely and used regex to specify capture groups which will be the fields extracted in splunk. There are some places where my example will fail. For example it will extract "up 40" if it's "up 40 minutes", "up 40 days", etc. To fix that, fix the regex... something like below will work:

 ^(?<containerid>\w+)\s+(?<image>\w+)\s+"(?<command>.+)"\s+(?<created>.+) ago\s+(?<status>(Up|Down|Bad|Ugly)).+\s(minutes|hours|days|weeks|months|years)\s+(?<ports>\S+)\s+(?<names>.*)$

Where status is expected to be one of the 4 Up, Down, Bad, Ugly...

[daniel333]

Thanks for getting back to me. Only recent weeks has Splunk data normalization been part of my jobs so I'm getting myself into corners lately.

So I seem to be having trouble with the first part there. the SEDCMD which is supposed to wipe out the first row. Splunk does not seem to respect it.

my props.conf

[docker:ps]
SEDCMD-fixspace1 = s/CONTAINER.*$//g
#SHOULD_LINEMERGE=false
#LINE_BREAKER=(^$|[\r\n]+[\r\n]+)
#TRUNCATE=1000000
#DATETIME_CONFIG = CURRENT
#SEDCMD-extractFields = ^(?<containerid>\w+)\s+(?<image>\w+)\s+"(?<command>.+)"\s+(?<created>.+) ago\s+(?<status>.+)\s(minutes|hours|days|weeks|months|years)\s+(?<ports>\S+)\s+(?<names>.*)$

My results have the text in line after restarting Splunk

2/15/16
12:19:10.000 PM 

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
0dce14da9952        centos              "/bin/bash -c 'while "   11 hours ago        Up 11 hours                             grave_mayer
5969bffa8ad2        centos              "/bin/bash -c 'while "   11 hours ago        Up 11 hours                             distracted_jepsen

host = localhost.localdomain  
 source = /opt/splunk/etc/apps/SA-docker/bin/getPS.sh  
 sourcetype = docker:ps