I have logs with to and from email address like so:
..... from=<mickey.mouse@disney.com> .....
..... to=<minnie.mouse@disney.com> .....
As you can guess, Splunk nicely parses them like so:
from = <mickey.mouse@disney.com>
to = <minnie.mouse@disney.com>
how do I get rid of the gt and lt signs?
rex in the search field works:
rex field=_raw "from=<(?<sender>.+?)>"
But won't work in props.conf
EXTRACT-sender = from=<(?<sender>.+?)>
The sender field never appears, like the extraction didn't work.
I thought about using SEDCMD and tried with rex first
rex mode=sed field=_raw "s/from=</from=/g"
but then I wasn't sure how to get the trailing >
So far none of this has worked.
I'm not sure if this should be done in just the props.conf, using props and transforms, or using SECCMD at the HF.
Suggestions appreciated.
Thanks
So you are using the props
on the search head, right? If you are distributing that with an app, that works. But if you want to make it more generally accessible (and editable by someone with out admin access to your servers), I'd create a auto field extraction for it (you can use the Settings -> Field Extractions or the Splunk Field Extraction Tool for that). It's also easier to make adjustments there if you need to. Then you don't have to use rex
.
So you are using the props
on the search head, right? If you are distributing that with an app, that works. But if you want to make it more generally accessible (and editable by someone with out admin access to your servers), I'd create a auto field extraction for it (you can use the Settings -> Field Extractions or the Splunk Field Extraction Tool for that). It's also easier to make adjustments there if you need to. Then you don't have to use rex
.
right. But I have a three server cluster, so the props is in a custom app that I'm sending out from my deployment server. If I create a field extraction using that method, won't it get overwritten next time I do a deployment?
You are using the deployment server to push out configs to the either the indexer cluster or the search head cluster? Those are not supported configs. We tried doing that a few years ago, and we had problems and stopped doing it when we found out it was not supported. Your CM should not get configs from the DS, and the deployer shouldn't either. If you are not doing this, ignore my concerns and let me know, and I'll modify this posting.
If you use the auto field extractions, they will not get overwritten by an update. They are maintained separately within the configuration tree, and won't get overwritten. We have dozens of field extractions that never get overwritten when we upgrade, and we've done updates about 6 times in the last year alone.
First, not sure what you mean by CM. Cluster Master?
I get what you are saying about the field extractions. I will try that and let you know what happens.
Yes, the CM is the cluster master. Are you deploying the app to the SHC (search head cluster) through your DS (deployment server)? or are you using just the deployer for the SHC?
Using the field extraction tool via the web gui did work and it did distribute to all servers in the cluster. Thanks.
try this,
| makeresults | eval from="<mickey.mouse@disney.com>" | eval to="<minnie.mouse@disney.com>" | eval from=replace(from,"<|>","") | eval to=replace(to,"<|>","")
Settings->>Fields » Calculated fields » Add new
Name: from
Eval expression = replace(from,"<|>","")
or
props.conf
[sourcetype]
EVAL-from = replace(from,"<|>","")
EVAL-to = replace(to,"<|>","")
The eval replace works great in the search line, but doesn't in the props.conf. Not sure if it's related to what @cpetterborg said above...
I always ask this on questions like this, but here goes. "Do you really need to extract the field at index time?" Unless you really need to, do it at search time. Moving on.
Did you try to put a backslash before the angle brackets in your EXTRACT line?
This shows the SEDCMD format that you probably need:
rex mode=sed field=_raw "s/from=<(.*)>/from=\\1/g"
Again, you may want to try it in the SEDCMD with backslashes before the angle brackets.
I'd like to do it at search time, but as I said, the EXTRACT in my props.conf won't work.