Getting Data In

Struggling with ingest-time lookup

PickleRick
SplunkTrust
SplunkTrust

Trying to do a lookup in ingest-time according to https://docs.splunk.com/Documentation/Splunk/8.1.3/Data/IngestLookups and can't get it to work.

If I do a simple transform like

[my-test-eval]
INGEST_EVAL = test=spath(_raw,"Event.System.Computer")

 

I properly get the test field propagated with a value extracted from the event. (as you probably guessed - it's a typical windows XML-formed event).

But if I want to use the value retrieved from the event to perform a lookup... sorry, won't happen. And I have no clue why.

[my-test-eval]
INGEST_EVAL = test=coalesce(json_extract(lookup("test.csv",json_object("Key",spath(_raw,"Event.System.Computer"),json_array("Value")),"Value"),"default")

There is no test field in the ingested event. Not even having the "default" value.

 I tried giving the lookup name as defined in transforms.conf as well as the csv filename itself. I tried putting the lookup in app context as well as in system/local. Nothing works.

To make things more interesting - if I did some mistake in defining the lookup (like giving wrong column name), I'd get an error in the splunkd.log so it would be obvious that something is not right.

But the problem is I don't get any errors, the transform therefore should be working but it isn't. So I'm completely stuck here.

How to debug this thing?

Labels (1)
Tags (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. I re-checked it after writing the post (it usually works this way - I struggle with something for a few hours, then ask on community, then have an epiphany 🤣). And it seems I had parentheses wrong in the eval expression. Which only shows how annoying this whole construct is because you don't get any errors in logs so it's hard to troubleshoot it.

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...