Hi
I have a problem in Splunk's regex and I can't figure it out for the life of me.
I'm going to simplify my problem a bit. Imagine this is my data:
|a|b|
If 'a' exists, I want my regex to pick out 'a' only, otherwise I want it to pick out 'b' only.
So in this case: |a|b| my regex should pick out 'a'. But, if for some reason 'a' didn't exist in my data, I want the same bit of regex to pick out 'b'.
|a|b| ---> should pick out 'a' only
|c|b| ---> should pick out 'b' only
'b' always exists, but 'a' sometimes doesn't.
Any ideas?
Thanks a lot!
Phil
Try this (I may be taking you too literally, but the pieces that you need are here):
(?J)(?:^|[\r\n])\|(?:(?<MyField>a)\|b\|)|(?:c\|(?<MyField>b)\|)
| makeresults
| eval _raw="2014-08-29 16:09:00,830 INFO Order filled for 8=FIX.4.4|9=667|15=GBP|55=USD
2014-08-27 16:11:19,373 INFO Order filled for 8=FIX.4.4|9=667|55=USD"
| makemv delim="
" _raw
| stats count by _raw
`comment("this is sample log")`
`comment("the logic")`
| rex "^(?<_time>[\w-]+ \d\d:\d\d:\d\d,\d{3}) .+ for (?<message>.+)"
| eval _time=strptime(_time,"%F %T")
| eval message=split(message,"|")
| stats count by _time message
| rex field=message "(?<key>\w+)=(?<value>.+$)"
| eval {key} = value
| fields - message count key value
| stats values(*) as * by _time
This is @philallen1 's sample.
I think it is unnecessary to use conditional REGEX .
Here. _
fields are a little tricky so I would eval
/rename
them like I did here.
index=myindex |
eval no_referrer_regex="MYREGEX1" |
eval referrer_regex="MYREGEX2" |
eval regex=if(_time < 1579250700,no_referrer_regex,referrer_regex) | eval raw=_raw |
map maxsearches=10000 search="| makeresults | eval mapped_raw=\"$$raw$$\" | rex field=mapped_raw \"$$regex$$\"" | table pst pst_epoch id action path num desc browser referrer
A second approach would just be to use ad-hoc searches in SimpleXML to set token values.
I would try something like
... | rex "| \d+=(?
In theory this will see the pipe followed by a number, equal sign, and then upper case letters and only do that for the first time it runs into that pattern. I forget if all currency has a 3 letter designate but you could potentially change the [A-Z]+ part to [A-Z]{3}.
Am going off the top of my head with no caffeine though so YMMV.
try this: [your search query] | rex _raw "15=(?P<currency_a>\w+)" | rex _raw "55=(?P<currency_b>\w+)" | eval return_value = if( isnull(currency_a), currency_b, currency_a)
assuming the 'a' field is always preceded by '15=' and 'b' field is always preceded by '55='
Probably better (definitely simpler) to do | eval return_value = coalesce(currency_a, currency_b)
I'm not sure if I need to put these @ signs in to get people's attention, so I'm giving it a go!
@ppablo_splunk
@sk314
@lguinn
@strive
Hi guys
Thanks for the responses. Here are some sample logs.
2014-08-29 16:09:00,830 INFO Order filled for 8=FIX.4.4|9=667|15=GBP|55=USD
2014-08-27 16:11:19,373 INFO Order filled for 8=FIX.4.4|9=667|55=USD
So the fields I am referring to are "15=..." and "55=...". I'm interested in the currency that each of these fields give.
The "15=..." field is the equivalent to my "a" field - i.e. it doesn't always exist. When it does exist I want the currency that it gives, otherwise I want the currency that "55=..." gives.
I hope that makes sense?
Phil
You question clearly states that a and b are two separate fields, but your example shows that a is value of a field.
Assuming you will have a and b as two distinct fields, you need coalesce function.
try like this:
Your base search...| eval RequiredField = coalesce(a,b)
Read more on coalesce here: http://docs.splunk.com/Documentation/Splunk/6.1.3/SearchReference/Commonevalfunctions
As @ppablo_splunk mentioned, if you can give sample log events, it will be helpful for us to answer your question.
Why does it have to be "the same bit of regex"??
Since "'b' always exists, but 'a' sometimes doesn't"
Have you tried using fillnull on field 'a'? like so:
[your search] | fillnull value=0 a | rex a "regex for field a" | rex b "regex for field b" | eval return_value = if(isnull(a), b, a)
Worked perfectly for my situation
Hi @philallen1
This is a great question. It'll be really helpful if you could paste a sample of your data so people can help you figure out an accurate regex for your use case.