topic Re: search time field extractions using props for an Indexed field in Getting Data In

search time field extractions using props for an Indexed field

Raghav2384 — Mon, 28 Sep 2020 18:09:10 GMT

Hello Experts,
We have a field xyz which holds mac addresses. Problem is, some of the mac addresses are of xx:xx:xx:xx:xx:xx format and some are xxxx.xxxx.xxxx & xx-xx-xx-xx-xx-xx & xxxx-xxxx-xxxx-xxxx (Every weirdest pattern you can imagine). We used
rex field=_raw mode=sed s/[-|.|:]//g to make it all numbers. I know Props.conf documentation reads, SEDCMD is only used at index time. Is there any other alternative procedures you recommend.
I tried EVAL-foo = replace(CALLING_STATION_ID,":*","") in props.conf to no avail.
Thanks in advance.

Raghav

Re: search time field extractions using props for an Indexed field

aweitzman — Wed, 12 Nov 2014 17:03:57 GMT

Does this create a field foo with none of the colons, dashes or dots?

EVAL-foo = replace(CALLING_STATION_ID,"([-:\.])","")

Re: search time field extractions using props for an Indexed field

Raghav2384 — Mon, 28 Sep 2020 18:09:19 GMT

Tried, didn't work.
EVAL-MACID = replace(CALLING_STATION_ID,"([-:.])","") in props.conf, restarted splunk and MACID is not in there.
Also tried ...=replace(CALLING_STATION_ID,"([-|:|.])",""), no luck here either.
Any different route you recommend?
Thanks,
Raghav

Re: search time field extractions using props for an Indexed field

aweitzman — Wed, 12 Nov 2014 18:43:38 GMT

Is CALLING_STATION_ID a field in the actual data, or is it a generated field (with a different EVAL- or FIELDALIAS- entry in props.conf)? It won't work if it's a generated field.

I did what I proposed above and it worked, so I'm not sure what might be going on in your environment. Can you try with just a colon within the brackets and see if it at least does that?

Re: search time field extractions using props for an Indexed field

Raghav2384 — Mon, 28 Sep 2020 18:09:24 GMT

CALLING_STATION_ID is a field in the actual data. Example log
XXXXXXXXX= S13456
PORT-ID = 1//1/1/1
FRAMED-IP = 1.2.3.4
NAS-PORT-TYPE = 5
CALLING-STATION-ID = 12:ae:45:y2:35:3d
NAS-IP-ADDRESS = x.y.x.xy
And is extracted by splunk at the index time.

Re: search time field extractions using props for an Indexed field

aweitzman — Wed, 12 Nov 2014 19:16:57 GMT

Do you see any errors regarding the MACID field in splunkd.log?

Re: search time field extractions using props for an Indexed field

Raghav2384 — Wed, 12 Nov 2014 22:59:15 GMT

This is what i found in the logs :'EVAL-foo' in stanza [aaa]: The expression is malformed. An unexpected character is reached at '”:”,””)'.

Re: search time field extractions using props for an Indexed field

Raghav2384 — Thu, 13 Nov 2014 01:16:52 GMT

Could you please post me your props stanza and sample log you created to replicate this? I want to see where the heck i am doing wrong.

Re: search time field extractions using props for an Indexed field

aweitzman — Thu, 13 Nov 2014 13:47:45 GMT

Here's the run-anywhere example based on the props stanza, just to prove that the regex is correct:

| gentimes start=-1 
| eval CALLING_STATION_ID="123-456-789,12:34:56:78,12.345.6789,12.34:56-78" 
| makemv delim="," CALLING_STATION_ID 
| mvexpand CALLING_STATION_ID 
| eval MACID = replace(CALLING_STATION_ID,"([-:\.])","") 
| table CALLING_STATION_ID MACID

If you take that exact replace string (hyphen must be first inside the brackets, and there must be a backslash prior to the period) and put it in props.conf like so:

EVAL-MACID = replace(CALLING_STATION_ID,"([-:\.])","")

it should work the same way. I also tested this props line on similar actual data in my environment and it worked.

Re: search time field extractions using props for an Indexed field

Raghav2384 — Thu, 13 Nov 2014 15:04:31 GMT

Thanks a ton @aweitzman. Culprit here is me, just realized that my config file reads "props.confs".
See if you can throw a punch at my face through your next comment 🙂

Re: search time field extractions using props for an Indexed field

aweitzman — Thu, 13 Nov 2014 15:12:28 GMT

Ha! 🙂

Happens to the best of us. Glad it's working now.