adding lookup or eval to transforms.conf

[fk319]

I would like to do an eval on every log entry, from a certian sourcetype. In this case I have a real number that I would like to quantize.

I also have a series of IPs that I would like to group by location. I am using a lookup at search, but I am only interested one or two locations for a search. Some of the IP ranges are /24, so I would also like to use wildcards.

I would like to make these changes when the log is loaded and not at search time so that I can search by location. Do I do this in transforms.conf? or is there another location?

[bwooden · Accepted solution]

You can not automatically call an eval out of the box using transforms.conf. You could create a macro though to minimize your typing. Instead of typing

... | eval some_var=<some eval opeartion>

you would type

... `macro_name`

---

Wildcards are not currently supported in lookups but you do have a few options.

1. You can create a scripted input to do it.
2. You can use event types.

3. You can leverage the search language (note: the below example would need to be updated for different mask bits)

   ... | rex field=ip "(?\d{1,3}.\d{1,3}.\d{1,3}.)" | strcat cidr_test "0/24" cidr_test | eval cidr=if(cidrmatch(cidr_test,ip),cidr_test,null) | where isnotnull(cidr) | lookup cidr OUTPUT location | search location="home_office"

[gkanapathy · Accepted solution]

You can not do this at index time (except by preprocessing your data before it even gets to Splunk).

You can configure automatic lookups that execute a script that can do whatever operations you need on the incoming data however.

[bwooden · Accepted solution]

You can not automatically call an eval out of the box using transforms.conf. You could create a macro though to minimize your typing. Instead of typing

... | eval some_var=<some eval opeartion>

you would type

... `macro_name`

---

Wildcards are not currently supported in lookups but you do have a few options.

1. You can create a scripted input to do it.
2. You can use event types.

3. You can leverage the search language (note: the below example would need to be updated for different mask bits)

   ... | rex field=ip "(?\d{1,3}.\d{1,3}.\d{1,3}.)" | strcat cidr_test "0/24" cidr_test | eval cidr=if(cidrmatch(cidr_test,ip),cidr_test,null) | where isnotnull(cidr) | lookup cidr OUTPUT location | search location="home_office"