topic Re: Field Extraction: Regex global flag/modifier in Splunk Search

Field Extraction: Regex global flag/modifier

konnex — Thu, 10 Oct 2019 15:22:10 GMT

Hi Splunkers,

I know that it is possible to match multiple times using rex (using max_match=0).

Can I apply the same logic to a field extraction? I tried .../g,/.../g, (?g)..., none of these work.

Re: Field Extraction: Regex global flag/modifier

richgalloway — Thu, 10 Oct 2019 16:51:07 GMT

If you mean an index-time field extraction then you probably want to add MV_ADD=true to your transforms.conf file.

Re: Field Extraction: Regex global flag/modifier

konnex — Thu, 10 Oct 2019 17:05:56 GMT

No, I am talking about field extraction configured in props.conf...

Re: Field Extraction: Regex global flag/modifier

richgalloway — Thu, 10 Oct 2019 18:03:12 GMT

How are you doing them in props? Please share your config.

Re: Field Extraction: Regex global flag/modifier

konnex — Thu, 10 Oct 2019 18:33:17 GMT

[aws:sqs]
KV_MODE = none
TIME_PREFIX = \"eventTimestamp\\\":\\\"
#2017-07-19T16:03:42.195Z
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3Q%Z
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\n\r]+)\{\"MD5
TRUNCATE = 999999
CHARSET = UTF-8

EXTRACT-eventAction,eventCategory,eventLabel,eventTimestamp,adid,userid = (?:\\"eventAction\\" ?: ?\\"(?<eventAction>(?:(?!\\").)*)|\\"eventCategory\\" ?: ?\\"(?<eventCategory>(?:(?!\\").)*)|\\"eventLabel\\" ?: ?\\"(?<eventLabel>(?:(?!\\").)*)|\\"eventTimestamp\\" ?: ?\\"(?<eventTimestamp>(?:(?!\\").)*)|\\"adid\\" ?: ?\\"(?<adid>(?:(?!\\").)*)|\\"userid\\" ?: ?\\"(?<userid>(?:(?!\\").)*))+

Re: Field Extraction: Regex global flag/modifier

konnex — Fri, 11 Oct 2019 14:36:14 GMT

I managed to get the output I want using multiple EXTRACTs in props.conf:

EXTRACT-eventAction = \\"eventAction\\" ?: ?\\"(?<eventAction>(?:(?!\\").)*)
EXTRACT-eventCategory = \\"eventCategory\\" ?: ?\\"(?<eventCategory>(?:(?!\\").)*)
EXTRACT-eventLabel = \\"eventLabel\\" ?: ?\\"(?<eventLabel>(?:(?!\\").)*)
EXTRACT-eventTimestamp = \\"eventTimestamp\\" ?: ?\\"(?<eventTimestamp>(?:(?!\\").)*)
EXTRACT-adid = \\"adid\\" ?: ?\\"(?<adid>(?:(?!\\").)*)
EXTRACT-userid = \\"userid\\" ?: ?\\"(?<userid>(?:(?!\\").)*)

I did not know you could specify an extraction with multiple expressions.