Getting Data In

Struggling with ingest-time lookup

PickleRick
SplunkTrust
SplunkTrust

Trying to do a lookup in ingest-time according to https://docs.splunk.com/Documentation/Splunk/8.1.3/Data/IngestLookups and can't get it to work.

If I do a simple transform like

[my-test-eval]
INGEST_EVAL = test=spath(_raw,"Event.System.Computer")

 

I properly get the test field propagated with a value extracted from the event. (as you probably guessed - it's a typical windows XML-formed event).

But if I want to use the value retrieved from the event to perform a lookup... sorry, won't happen. And I have no clue why.

[my-test-eval]
INGEST_EVAL = test=coalesce(json_extract(lookup("test.csv",json_object("Key",spath(_raw,"Event.System.Computer"),json_array("Value")),"Value"),"default")

There is no test field in the ingested event. Not even having the "default" value.

 I tried giving the lookup name as defined in transforms.conf as well as the csv filename itself. I tried putting the lookup in app context as well as in system/local. Nothing works.

To make things more interesting - if I did some mistake in defining the lookup (like giving wrong column name), I'd get an error in the splunkd.log so it would be obvious that something is not right.

But the problem is I don't get any errors, the transform therefore should be working but it isn't. So I'm completely stuck here.

How to debug this thing?

Labels (1)
Tags (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. I re-checked it after writing the post (it usually works this way - I struggle with something for a few hours, then ask on community, then have an epiphany 🤣). And it seems I had parentheses wrong in the eval expression. Which only shows how annoying this whole construct is because you don't get any errors in logs so it's hard to troubleshoot it.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Deep Dive: Accelerate threat investigation with Splunk’s AI Assistant in Security

AI is one of the biggest topics in the market today, and for security teams, its value goes far beyond the ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Detection Engineering Office Hours: Real-World Troubleshooting & Q&A

[REGISTER HERE] This thread is for the Community Office Hours session on Detection Engineering Office Hours: ...