I know this is a really old post, but in case someone else comes across this, to create a multivalue _meta field, use this syntax: _meta = new_field::value1  new_field::value2 ... View more

This was incredibly useful. Thank you! ... View more

Almost 5 years to the day after the last reply to this thread, and this issue still hasn't been resolved in Splunk. Our situation is slightly different;  we have raw csv data which is processed via regex captures in in-line field extractions (EXTRACT-)  in props.conf.  The caveat for this type of field extraction is that only one single regex can be used. For years this has worked great, but recently for the same "source", 2 additional fields have been added, making the existing extraction not work. To compensate, I've retooled the regex using a conditional if/then/else lookahead to test if N+2 fields exist.  If so, process through the "then" regex, otherwise the "else". Here's the regex: (?J)^[^,\n]*,(?(?=([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+))((?P<retailer>[^,]+),(?P<storeNumber>[^,]+),(?P<displayName>[^,]+),(?P<button>[^,]+),(?P<buttonSelection>[^,]+),(?P<activeDevice>[^,]+),(?P<swVersion>[^,]+),(?P<contentVersion>[^,]+),(?P<platform>[^,]+),(?P<configType>[^,]+),(?P<presses>[^,]+))|((?P<retailer>[^,]+),(?P<storeNumber>[^,]+),(?P<displayName>[^,]+),(?P<button>[^,]+),(?P<swVersion>[^,]+),(?P<contentVersion>[^,]+),(?P<platform>[^,]+),(?P<configType>[^,]+),(?P<presses>[^,]+))) Here are examples of the two possible data: Original: 20210727,some_retailer,950,display-10,someButton,7.99.9966,2021.07.11-USA,platformX,PLAYER_MASTER,35 New data: 20210727,some_retailer,950,display-10,someButton,THIS IS THE BUTTON SELECTION,BIG-DEVICE,7.99.9966,2021.07.11-USA,platformX,PLAYER_MASTER,35 This works very well on regex101.com.  Captures populate as expected depending on the number of comma-delimited values present. Sadly, when I push the change to my dev Splunk, it doesn't work.  For the original data (i.e., with 2 less fields), all fields get extracted.  However for the newer data, ONLY the two new fields get extracted.   It's as if the Splunk regex engine isn't implementing PCRE2 correctly, in particular the (?J). Unfortunately I don't have the privilege of writing tickets, but I do believe it's a Splunk error. With all that being said, do any of you gurus out there know another way to write a single regex that I could try to accomplish this? Many thanks in advance!  ... View more

Thanks for the quick response. And thanks again for supporting Splunk (and Splunkers) with your TA. It's greatly appreciated. G. ... View more

it will be available in the local path $SPLUNK_HOME/etc/..../local because its fully user-specific ... View more

Once I placed the props.conf in the indexers....BOOM...works, thanks! ... View more

I will give that a try. I tried BREAK_ONLY_BEFORE = ^\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d,\d\d\d\s*[[]\d*[]] and that seemed to work. Many thanks ... View more

You need to avoid transaction if at all possible. ... View more

No worries. I'm pleased that it helped. Enjoy your Splunk journey! (And if you're happy with the answer, just mark the question as 'answered' - as it helps the rest of the community know that this one is sorted.) ... View more

This would work too. A little improved version of yours: |inputlookup Historic-80014-temps.csv | eval todaysMonthString=strftime(now(),"%b") | eval todaysDateString=strftime(now(),"%e") | rex field="rec-date" "(?<testDate>.+?)-<testMonth>.+?)" | where testMonth==todaysMonthString AND testDate==todaysDateString Also, see if my updates answer works for you now. ... View more

@duyanhtr... Anytime, did mktime work for you despite the spooky character? ... View more

Great to hear that you got it working! If you're happy that the question is resolved, just accept this as the answer, then it gets marked a 'done' when others are looking. Happy Splunking! ... View more

I was asking if there is limit on regex formula, may be like maximum total characters allowed, ... View more

Well done for sticking with it. Everything looks OK, so my guess would be that whatever you're doing to extract all of your original field names, is being done at 'Search time'. This would mean that they wouldn't be available yet for the EXTRACT commands in props.conf to reference. No worries, back to basics. Just extract them all in the sourcetype stanzas. So delete the EXTRACT lines that you have and try this instead: EXTRACT-full = ^(?<timeStamp>[^,]*),(?<elapsed>[^,]*),"(?<label>[^,]*),(?<ACL>[^"]*)",(?<responseCode>[^,]*),(?<responseMessage>[^,]*),(?:(?<targetHost>[^\s]*)\s(?<JMeterThread>[^,]*))?,(?<dataType>[^,]*),(?<success>[^,]*),(?<failureMessage>[^,]*),(?<bytes>[^,]*),(?<sentBytes>[^,]*),(?<grpThreads>[^,]*),(?<allThreads>[^,]*),(?<Latency>[^,]*),(?<Hostname>[^,]*),(?<IdleTime>[^,]*),(?<Connect>[^$]*) That should encompass all of your original fields and the 'splits' that you were looking for in your original question. I'm not sure if there's a way to see, "if the EXTRACTs are loaded by Splunk". But you can use the btool command to see the total configuration that is being applied to a sourcetype. You need console / command line access, but you could try running: bin/splunk btool props list Augsburg --debug This will give you the total config which is being applied to the 'Augsburg' sourcetype/stanza. In a test in my laptop, this gives: /Applications/Splunk/etc/system/local/props.conf [test-extract] /Applications/Splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True /Applications/Splunk/etc/system/default/props.conf AUTO_KV_JSON = true /Applications/Splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE = /Applications/Splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True /Applications/Splunk/etc/system/default/props.conf CHARSET = UTF-8 /Applications/Splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml /Applications/Splunk/etc/system/local/props.conf EXTRACT-full = ^(?<timeStamp>[^,]*),(?<elapsed>[^,]*),"(?<label>[^,]*),(?<ACL>[^"]*)",(?<responseCode>[^,]*),(?<responseMessage>[^,]*),(?:(?<targetHost>[^\s]*)\s(?<JMeterThread>[^,]*))?,(?<dataType>[^,]*),(?<success>[^,]*),(?<failureMessage>[^,]*),(?<bytes>[^,]*),(?<sentBytes>[^,]*),(?<grpThreads>[^,]*),(?<allThreads>[^,]*),(?<Latency>[^,]*),(?<Hostname>[^,]*),(?<IdleTime>[^,]*),(?<Connect>[^$]*) /Applications/Splunk/etc/system/default/props.conf HEADER_MODE = /Applications/Splunk/etc/system/default/props.conf LEARN_MODEL = true /Applications/Splunk/etc/system/default/props.conf LEARN_SOURCETYPE = true /Applications/Splunk/etc/system/default/props.conf LINE_BREAKER_LOOKBEHIND = 100 /Applications/Splunk/etc/system/default/props.conf MATCH_LIMIT = 100000 /Applications/Splunk/etc/system/default/props.conf MAX_DAYS_AGO = 2000 /Applications/Splunk/etc/system/default/props.conf MAX_DAYS_HENCE = 2 /Applications/Splunk/etc/system/default/props.conf MAX_DIFF_SECS_AGO = 3600 /Applications/Splunk/etc/system/default/props.conf MAX_DIFF_SECS_HENCE = 604800 /Applications/Splunk/etc/system/default/props.conf MAX_EVENTS = 256 /Applications/Splunk/etc/system/default/props.conf MAX_TIMESTAMP_LOOKAHEAD = 128 /Applications/Splunk/etc/system/default/props.conf MUST_BREAK_AFTER = /Applications/Splunk/etc/system/default/props.conf MUST_NOT_BREAK_AFTER = /Applications/Splunk/etc/system/default/props.conf MUST_NOT_BREAK_BEFORE = /Applications/Splunk/etc/system/default/props.conf SEGMENTATION = indexing /Applications/Splunk/etc/system/default/props.conf SEGMENTATION-all = full /Applications/Splunk/etc/system/default/props.conf SEGMENTATION-inner = inner /Applications/Splunk/etc/system/default/props.conf SEGMENTATION-outer = outer /Applications/Splunk/etc/system/default/props.conf SEGMENTATION-raw = none /Applications/Splunk/etc/system/default/props.conf SEGMENTATION-standard = standard /Applications/Splunk/etc/system/default/props.conf SHOULD_LINEMERGE = True /Applications/Splunk/etc/system/default/props.conf TRANSFORMS = /Applications/Splunk/etc/system/default/props.conf TRUNCATE = 10000 /Applications/Splunk/etc/system/default/props.conf detect_trailing_nulls = false /Applications/Splunk/etc/system/default/props.conf maxDist = 100 /Applications/Splunk/etc/system/default/props.conf priority = /Applications/Splunk/etc/system/default/props.conf sourcetype = The useful thing about this is you can see which configuration file the config is coming from. Splunk's config system is a layered approach, and sometimes is tricky to see what's coming from where. Have a go with the EXTRACT above, but also try btool for yourself to see if there's some other configuration being picked up. ... View more

This is essentially the same as my "convoluted" solution. In my opinion, it has general usage and should not take that much code to do such a trivial work. strptime does get the correct epoch time (UTC), just need an elegant way to convert it in any given time zone. Python added astimezone() method in dateutil (not datetime), so it has demand. Thanks a lot for your time! ... View more

Yup. That worked. Thanks. ... View more

Very strange. This was the props.conf which I created to test with: [not-breaking] TRUNCATE = 0 SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)(?:FATAL[\s]+|ERROR[\s]+|INFO[\s]+|WARN[\s]+|DEBUG[\s]+) MAX_EVENTS = 100000 TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N Are you using something similar? ... View more

I had a typo! I meant rex , not regex ! Try the fixed answer now! ... View more

gvmorley, Thank you for your help, it worked! ... View more

Especially when you use a regex in your code, you really need to mark the code as such so the web engine will not delete your terms that are in angle brackets. I made a guess at the missing regex code and formatted it like this index=tmol_uk sourcetype=apt.event [search sourcetype=pxy.access index=tmol_uk "POST /metrics HTTP/1.1" referer="/member/edit_profile" | fields sid] "email succesfully submitted to NTF" | fields sid email_address | join sid [search index=tmol_uk sourcetype=apt.event | rex field=_raw "pid.?\t(?<mid>[0-9]*)"] | table mid email_address sid so what your final search does is this - a) search in sourcetype=pxy.access with referer="/member/edit_profile" for the words "POST /metrics HTTP/1.1" from all those records, return only the sid and _time fields to the second search ( I prefer to return the values using "| table _time sid" rather than "| fields sid", so that I can be absolutely sure that no other returned values will affect the search. ) b) search in sourcetype=apt.event for the words "email succesfully submitted to NTF" on a record that has the sid and _time values from the first search. c) pass _time, sid and email_address along d) join that to the "mid" value pulled from the first record on sourcetype=apt.event that has the same sid e) drop all other unneeded fields from the join (I'd suggest that you put the commands "|table sid mid |dedup sid" at the end of the second subsearch, inside the brackets. Since you won't be using anything but the mid value that gets passed back, you want to tell splunk not to keep the other stuff.) ... View more

I suspect that this app will give you much insight: https://splunkbase.splunk.com/app/2871/ ... View more