Splunk Search

Using eval to create custom CSV file names and then use that field in lookup as CSV file name

andra_pietraru
Path Finder

Hello,

Is it possible to create custom lookup files names and then use them in lookup command in a query?
My events have this field:
... csvfileIndex=1 ...
... csvfileIndex=2 ...
I have many CSV files like: csvfile1.csv, csvfile2.csv. ...

What I would like to achieve is based on that field I would like to lookup a different CSV file. Something like:
... | eval csvName="csvfile".csvfileIndex.".csv" | lookup csvName name AS surname OUTPUTNEW address AS homeaddress | ...
But I am getting error "Error in 'lookup' command: The lookup table 'csvName.csv' does not exist"
Anyone knows what I am doing wrong or if it's even possible to use lookup like this?

0 Karma

acharlieh
Influencer

It'd take a bit of python or scripting work, and I'm not exactly sure about what the script would need to look like to be efficient, but you could potentially write an external lookup for this. There's a generic example about how this works in the docs, and an example script that ships with Sp...

The idea is you have a single external lookup for splunk that takes csvfileIndex and surname and the script upon invocation dynamically retrieves addresses from the appropriate file(s), then your search is easy. (The script to accomplish this however...)

MichaelPriest
Communicator

What are the differences between the files, couldn't you try automatic lookup's instead?

0 Karma

andra_pietraru
Path Finder

An automatic lookup would not help, because the content of the csv files is very similar and I would need to lookup only one per event, otherwise my data would get really confusing. That is why I used the csvFileIndex in my events, to make sure I only look at the correct csv file.

0 Karma

woodcock
Esteemed Legend

I do not think you can do this but this kind of thing has come up for me several times. What Splunk needs to do is give us a syntax like !value2literal! where we can do something like this:

... | eval csvName="csvfile".csvfileIndex.".csv" | lookup !csvName! name AS surname OUTPUTNEW address AS homeaddress | ..

The problem is that there is no way that I know of to set a Splunk field's value and use it in a place inside a Splunk command's syntax which takes a stirng-literal, such as the lookup stanza name in your example.

andra_pietraru
Path Finder

Thanks for the reply! I was hoping it's somehow possible. They should add something like this because it's really inconvenient if you have many CSV files.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...