Hello Splunkers,
I have an heavy forwarder that receives millions of events in json format. In order to save space and license I'd like to send to indexer only some interesting fields.
I tried different combinations with props and transforms but without success. What I obtained is simply the match of first group of regex and nothing else.
Here below the configuration (I reduced the regex to only 3 fields and the event structure to make it easy), an event example and what is the output now
transforms.conf
[KeepJsonFields]
REGEX = ("Field001":".*?".)|("Field002":".*?".)||("Field003":".*?".)
FORMAT = {$1$2$3}
DEST_KEY = _raw
props.conf
[st_json_fields]
DATETIME_CONFIG = KV_MODE = json
MAX_TIMESTAMP_LOOKAHEAD = 34
NO_BINARY_CHECK = true
TRANSFORMS-KeepJsonFields = KeepJsonFields
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y%m%d%H%M%S"
TIME_PREFIX = "FieldForTime":"
pulldown_type = true
TZ = UTC
Event sample
{
"Block1":{
"Field001":"Value001",
"Field002":"Value002 ",
"Field003":1000,
"Field004":"Value004",
"Field005":1000,
"Field00N":1000
},
"Block2":{
"Field-001":"Value-001",
"Field-002":"Value-002",
"Field-003":"Value-003",
"Field-00N":"Value-004"
},
"Block3":{
"Field_001":"Value_001",
"Field_002":"Value_002",
"Field_003":"Value_003"
}
}
Event in splunk
{"Field001":"Value001",}
Here below main problems I've encountered:
The regex seems to stop at the first occurence
How to manage commas to rebuild the correct json format (is it correct to include them in the capture group?)
Thanks in advance for the help you can give me
... View more