Hi @scelikok Thanks a lot for your reply, it was most helpful, and it helped me finding a solution. However, I realised that the snippet I had provided had some subtle differences with the actual ...
See more...
Hi @scelikok Thanks a lot for your reply, it was most helpful, and it helped me finding a solution. However, I realised that the snippet I had provided had some subtle differences with the actual data, and so I had to slightly adapt your solution. That being said, I was under the impression that your regex was not quite right either as I ran it through regex101 first and it only matched the first xml block (I stripped the beginning of the square bracket line to emulate the line breaker in props.conf) So, to recap, here is a more accurate example of the log: [1][DATA]BEGIN --- - 06:03:09[012] <?xml version="1.0" encoding="UTF-8"?> <root> <tag1>value</tag1> <nestedTag> <tag2>another value</tag2> </nestedTag> </root> [1][DATA]END --- - 06:03:09[012] [1][DATA]BEGIN --- - 07:03:09[123] <?xml version="1.0" encoding="UTF-8"?> <root> <tag1>some stuff</tag1> <nestedTag> <tag2>other stuff</tag2> </nestedTag> </root> [1][DATA]END --- - 07:03:09[123] [1][DATA]BEGIN --- - 08:03:09[456] <?xml version="1.0" encoding="UTF-8"?> <root> <tag1>some more data</tag1> <nestedTag> <tag2>fooband a bit more</tag2> </nestedTag> </root> [1][DATA]END --- - 08:03:09[456] Here is the props.conf I ended up using (as per @scelikok's suggestion): [my_sourcetype] LINE_BREAKER = (\[1\]\[DATA\]BEGIN[-\s]+) SHOULD_LINEMERGE = false TRANSFORM-transform2xml = transform2xml KV_MODE = xml And here is the corresponding transforms.conf, slightly tweaked - I ended up being a bit more explicit on the end of the event and removed some of the capturing groups: [transform2xml] REGEX = ^([^\[]+)\[\d+\][\r\n]+(<\?xml.*>[^\[]+)\[1\]\[DATA\]END --- - [\d:]+\[\d+\][\r\n]* FORMAT = <time>$1</time>$2 DEST_KEY = _raw It may not be a perfect xml, but that it works as expected and the xml is now automatically parsed. Thanks again for your help @scelikok !