search time field extractions using props for an Indexed field

[Raghav2384]

Hello Experts,
We have a field xyz which holds mac addresses. Problem is, some of the mac addresses are of xx:xx:xx:xx:xx:xx format and some are xxxx.xxxx.xxxx & xx-xx-xx-xx-xx-xx & xxxx-xxxx-xxxx-xxxx (Every weirdest pattern you can imagine). We used
rex field=_raw mode=sed s/[-|.|:]//g to make it all numbers. I know Props.conf documentation reads, SEDCMD is only used at index time. Is there any other alternative procedures you recommend.
I tried EVAL-foo = replace(CALLING_STATION_ID,":*","") in props.conf to no avail.
Thanks in advance.

Raghav

[aweitzman · Accepted solution]

Does this create a field foo with none of the colons, dashes or dots?

EVAL-foo = replace(CALLING_STATION_ID,"([-:\.])","")

[Raghav2384]

Tried, didn't work.
EVAL-MACID = replace(CALLING_STATION_ID,"([-:.])","") in props.conf, restarted splunk and MACID is not in there.
Also tried ...=replace(CALLING_STATION_ID,"([-|:|.])",""), no luck here either.
Any different route you recommend?
Thanks,
Raghav

[aweitzman]

Is CALLING_STATION_ID a field in the actual data, or is it a generated field (with a different EVAL- or FIELDALIAS- entry in props.conf)? It won't work if it's a generated field.

I did what I proposed above and it worked, so I'm not sure what might be going on in your environment. Can you try with just a colon within the brackets and see if it at least does that?

[Raghav2384]

This is what i found in the logs :'EVAL-foo' in stanza [aaa]: The expression is malformed. An unexpected character is reached at '”:”,””)'.

[Raghav2384]

CALLING_STATION_ID is a field in the actual data. Example log
XXXXXXXXX= S13456
PORT-ID = 1//1/1/1
FRAMED-IP = 1.2.3.4
NAS-PORT-TYPE = 5
CALLING-STATION-ID = 12:ae:45:y2:35:3d
NAS-IP-ADDRESS = x.y.x.xy
And is extracted by splunk at the index time.

[aweitzman]

Do you see any errors regarding the MACID field in splunkd.log?

[Raghav2384]

Could you please post me your props stanza and sample log you created to replicate this? I want to see where the heck i am doing wrong.

[aweitzman]

Here's the run-anywhere example based on the props stanza, just to prove that the regex is correct:

| gentimes start=-1 
| eval CALLING_STATION_ID="123-456-789,12:34:56:78,12.345.6789,12.34:56-78" 
| makemv delim="," CALLING_STATION_ID 
| mvexpand CALLING_STATION_ID 
| eval MACID = replace(CALLING_STATION_ID,"([-:\.])","") 
| table CALLING_STATION_ID MACID

If you take that exact replace string (hyphen must be first inside the brackets, and there must be a backslash prior to the period) and put it in props.conf like so:

EVAL-MACID = replace(CALLING_STATION_ID,"([-:\.])","")

it should work the same way. I also tested this props line on similar actual data in my environment and it worked.

[Raghav2384]

Thanks a ton @aweitzman. Culprit here is me, just realized that my config file reads "props.confs".
See if you can throw a punch at my face through your next comment 🙂

[aweitzman]

Ha! 🙂

Happens to the best of us. Glad it's working now.