Getting Data In

Remove beginning less-than and trailing greater-than in field

reswob4
Builder

I have logs with to and from email address like so:

..... from=<mickey.mouse@disney.com> .....
..... to=<minnie.mouse@disney.com> .....

As you can guess, Splunk nicely parses them like so:

from = <mickey.mouse@disney.com>
to = <minnie.mouse@disney.com>

how do I get rid of the gt and lt signs?

rex in the search field works:

 rex field=_raw "from=<(?<sender>.+?)>"

But won't work in props.conf

EXTRACT-sender = from=<(?<sender>.+?)>

The sender field never appears, like the extraction didn't work.

I thought about using SEDCMD and tried with rex first

rex mode=sed field=_raw "s/from=</from=/g"

but then I wasn't sure how to get the trailing >

So far none of this has worked.

I'm not sure if this should be done in just the props.conf, using props and transforms, or using SECCMD at the HF.

Suggestions appreciated.

Thanks

0 Karma
1 Solution

cpetterborg
SplunkTrust
SplunkTrust

So you are using the props on the search head, right? If you are distributing that with an app, that works. But if you want to make it more generally accessible (and editable by someone with out admin access to your servers), I'd create a auto field extraction for it (you can use the Settings -> Field Extractions or the Splunk Field Extraction Tool for that). It's also easier to make adjustments there if you need to. Then you don't have to use rex.

View solution in original post

cpetterborg
SplunkTrust
SplunkTrust

So you are using the props on the search head, right? If you are distributing that with an app, that works. But if you want to make it more generally accessible (and editable by someone with out admin access to your servers), I'd create a auto field extraction for it (you can use the Settings -> Field Extractions or the Splunk Field Extraction Tool for that). It's also easier to make adjustments there if you need to. Then you don't have to use rex.

reswob4
Builder

right. But I have a three server cluster, so the props is in a custom app that I'm sending out from my deployment server. If I create a field extraction using that method, won't it get overwritten next time I do a deployment?

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

You are using the deployment server to push out configs to the either the indexer cluster or the search head cluster? Those are not supported configs. We tried doing that a few years ago, and we had problems and stopped doing it when we found out it was not supported. Your CM should not get configs from the DS, and the deployer shouldn't either. If you are not doing this, ignore my concerns and let me know, and I'll modify this posting.

If you use the auto field extractions, they will not get overwritten by an update. They are maintained separately within the configuration tree, and won't get overwritten. We have dozens of field extractions that never get overwritten when we upgrade, and we've done updates about 6 times in the last year alone.

0 Karma

reswob4
Builder

First, not sure what you mean by CM. Cluster Master?

I get what you are saying about the field extractions. I will try that and let you know what happens.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Yes, the CM is the cluster master. Are you deploying the app to the SHC (search head cluster) through your DS (deployment server)? or are you using just the deployer for the SHC?

0 Karma

reswob4
Builder

Using the field extraction tool via the web gui did work and it did distribute to all servers in the cluster. Thanks.

0 Karma

sbbadri
Motivator

try this,

| makeresults | eval from="<mickey.mouse@disney.com>" | eval to="<minnie.mouse@disney.com>" | eval from=replace(from,"<|>","") | eval to=replace(to,"<|>","")

0 Karma

sbbadri
Motivator

Settings->>Fields » Calculated fields » Add new
Name: from
Eval expression = replace(from,"<|>","")

or

props.conf

[sourcetype]
EVAL-from = replace(from,"<|>","")
EVAL-to = replace(to,"<|>","")

0 Karma

reswob4
Builder

The eval replace works great in the search line, but doesn't in the props.conf. Not sure if it's related to what @cpetterborg said above...

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I always ask this on questions like this, but here goes. "Do you really need to extract the field at index time?" Unless you really need to, do it at search time. Moving on.

Did you try to put a backslash before the angle brackets in your EXTRACT line?

This shows the SEDCMD format that you probably need:

rex mode=sed field=_raw "s/from=<(.*)>/from=\\1/g"

Again, you may want to try it in the SEDCMD with backslashes before the angle brackets.

0 Karma

reswob4
Builder

I'd like to do it at search time, but as I said, the EXTRACT in my props.conf won't work.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...