Cleaning raw data at index time or search time?

aapittts — Thu, 21 Feb 2013 21:07:48 GMT

I have raw data that looks like this: (4)example(3)domain(3)com(0). In my search, I've been using a macro that looks like this:

eval $name$=replace($name$, "\(\d+\)",".")|eval $name$=replace($name$, "^\.", "")

This produces the desired result. However, when I try and pipe the output of the macro to a lookup table it doesn't work. I've narrowed the issue down to the regex bc if I put the example domain above in my lookup table I get the proper results. That is not the solution bc I have hundreds of domains in the lookup table and can not change them all. So my question is is there a way to pass the output of the regex properly or is this something that needs to be taken care of in the props or transforms?

Re: Cleaning raw data at index time or search time?

aapittts — Fri, 22 Feb 2013 13:27:01 GMT

After fighting with the regex more, I realized I wasn't replacing the final '.' from the domain name thus not getting any matches against my look up table.

eval $name$=replace($name$, "\(\d+\)",".")|eval $name$=replace($name$, "^\.|$\.", "")

topic Re: Cleaning raw data at index time or search time? in Splunk Search

Cleaning raw data at index time or search time?

Re: Cleaning raw data at index time or search time?