I'm trying to extract two index-time fields from the input stream. Both should be multivalued. I successfully extracted the first one, and it is multivalued, just like I wanted. However, the second field, which is to be extracted from the first one (like a short code, which is a suffix of its full version), uses only the first value of it.
Here is a quick example I've created:
transforms.conf
[mainKey]
REGEX = record(?:\.\d+)?\.code="(?P<mainKey>[^"]+)"
#FORMAT = mainKey::$1
WRITE_META = true
REPEAT_MATCH = true
LOOKAHEAD = 1048576
MV_ADD = 1
[subKey]
REGEX = (?m-s)(?<=^|\s)[a-zA-Z]*(?P<subKey>\d+)(?=\s|$)
#FORMAT = subKey::$1
SOURCE_KEY = field:mainKey
WRITE_META = true
REPEAT_MATCH = true
MV_ADD = 1
props.conf
[testIndexFields]
DATETIME_CONFIG =
NO_BINARY_CHECK = true
category = Custom
description = Testing multivalue index-time fields
pulldown_type = true
TRANSFORMS-mainKey = mainKey
TRANSFORMS-subKey = subKey
Where testIndexFields is a sourcetype I'm importing this data to.
I prepared the following file as a data sample:
2016-12-13 17:07:20, record.1.code="MAIN132" record.2.code="PRE9087", record.3.code="1405"
2016-12-13 17:07:40, record.code="SingleCode0123456"
2016-12-13 17:08:00, record.1.code="123BadOne", record.2.code="GoodOne1", record.3.code="NoSubKey"
2016-12-13 17:08:20, record.1.code="!alsobad123",record.2.code="TryThis1508"
2016-12-13 17:07:20, record.code="Unnumbered0001", record.code="Unnumbered0002", record.code="Unnumbered0003"
I'm expecting the data to be extracted like that:
mainKey=MAIN132 mainKey=PRE9087 mainKey=1405 subKey=132 subKey=9087 subKey=1405
mainKey=SingleCode0123456 subKey=0123456
mainKey=123BadOne mainKey=GoodOne1 mainKey=NoSubKey subKey=1
mainKey=Unnumbered0001 mainKey=Unnumbered0002 mainKey=Unnumbered0003 subKey=0001 subKey=0002 subKey=0003
However, I'm getting this:
mainKey = MAIN132 mainKey = PRE9087 mainKey = 1405 subKey = 132
mainKey = SingleCode0123456 subKey = 0123456
mainKey = 123BadOne mainKey = GoodOne1 mainKey = NoSubKey
mainKey = !alsobad123 mainKey = TryThis1508
mainKey = Unnumbered0001 mainKey = Unnumbered0002 mainKey = Unnumbered0003 subKey = 0001
As you can see, the subKey is extracted from the first occurrence of mainKey only. Is there a way to change this behavior?
... View more