Getting Data In

Struggling with ingest-time lookup

PickleRick
SplunkTrust
SplunkTrust

Trying to do a lookup in ingest-time according to https://docs.splunk.com/Documentation/Splunk/8.1.3/Data/IngestLookups and can't get it to work.

If I do a simple transform like

[my-test-eval]
INGEST_EVAL = test=spath(_raw,"Event.System.Computer")

 

I properly get the test field propagated with a value extracted from the event. (as you probably guessed - it's a typical windows XML-formed event).

But if I want to use the value retrieved from the event to perform a lookup... sorry, won't happen. And I have no clue why.

[my-test-eval]
INGEST_EVAL = test=coalesce(json_extract(lookup("test.csv",json_object("Key",spath(_raw,"Event.System.Computer"),json_array("Value")),"Value"),"default")

There is no test field in the ingested event. Not even having the "default" value.

 I tried giving the lookup name as defined in transforms.conf as well as the csv filename itself. I tried putting the lookup in app context as well as in system/local. Nothing works.

To make things more interesting - if I did some mistake in defining the lookup (like giving wrong column name), I'd get an error in the splunkd.log so it would be obvious that something is not right.

But the problem is I don't get any errors, the transform therefore should be working but it isn't. So I'm completely stuck here.

How to debug this thing?

Labels (1)
Tags (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. I re-checked it after writing the post (it usually works this way - I struggle with something for a few hours, then ask on community, then have an epiphany 🤣). And it seems I had parentheses wrong in the eval expression. Which only shows how annoying this whole construct is because you don't get any errors in logs so it's hard to troubleshoot it.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...