Multivalue Field Extraction issue.

[jorvinmartinez]

I've been working on a complicated set of log files that collect performance stats for a number of counters. It requires regex work to be done.

The format for the logs files is :

[Source_Identifier]servername[TYPE]-instanceofcounter#counter#start_time_of_collection#end_time_of_collection#numSamples-__#MULIPLE_VALUES_SEPARATED_BY_Comma

Below is a sample :

[virtualserver]Server1.domain.com[HostSystem]-vmnic0#net.transmitted.average#2013-12-31T13:43:00EST#2013-12-31T13:57:40EST#numSamples-45#14,8,26,26,20,14,2,8,2,16,9,3,54,9,14,2,8,2,26,9,14,25,8,14,4,9,26,27,20,15,24,8,2,13,2,21,14,8,3,4,20,3,42,8,15

I created the following field extractions :

perfmon : EXTRACT-perfmon_counter Inline (?i)^[^#]#(?P[^#]+)
perfmon : EXTRACT-perfmon_date_start Inline (?i)..?#(?P\d+-\d+-\d+\w+:\d+:\d+\w+)(?=#)

perfmon : EXTRACT-perfmon_device Inline (?i)[*]-(?P[^#]+)
perfmon : EXTRACT-perfmon_entity Inline (?i)[virtualserver](?P[^[]+)

perfmon : EXTRACT-perfmon_value Inline (?i)\d+\,(?[^,]+)

The problem I'm having is that the perfmon_value field should be a multivalue field. Only the first value is displayed when I run the following search :

index=perfmon_manager sourcetype=perfmon perfmon_counter="net.transmitted.average" perfmon_entity="Server1.domain.com" | mvexpand perfmon_value | table _time perfmon_entity,perfmon_counter, perfmon_device, perfmon_value

_time perfmon_entity perfmon_counter perfmon_device perfmon_value

1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 14

I would like to get a table each value as a separate event similar to :

_time perfmon_entity perfmon_counter perfmon_device perfmon_value

1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 14
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 8
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 26
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 26
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 20
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 14
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 2
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 8
1 1/2/14 11:47:20.000 PM Server1.domain.com net.transmitted.average vmnic0 2

THanks for your assistance.

[Ledion_Bitincka · Accepted solution]

This is similar to this question . You can do what you need by using REPORT in props.conf and transforms.conf to define the extraction rule, e.g.

props.conf
[sourcetype]
...
REPORT-xyz = perfmon-vals

transforms.conf
[perfmon-vals]
REGEX = ,(\d+)
MV_ADD = true