Splunk Search

Using eval to create custom CSV file names and then use that field in lookup as CSV file name

andra_pietraru
Path Finder

Hello,

Is it possible to create custom lookup files names and then use them in lookup command in a query?
My events have this field:
... csvfileIndex=1 ...
... csvfileIndex=2 ...
I have many CSV files like: csvfile1.csv, csvfile2.csv. ...

What I would like to achieve is based on that field I would like to lookup a different CSV file. Something like:
... | eval csvName="csvfile".csvfileIndex.".csv" | lookup csvName name AS surname OUTPUTNEW address AS homeaddress | ...
But I am getting error "Error in 'lookup' command: The lookup table 'csvName.csv' does not exist"
Anyone knows what I am doing wrong or if it's even possible to use lookup like this?

0 Karma

acharlieh
Influencer

It'd take a bit of python or scripting work, and I'm not exactly sure about what the script would need to look like to be efficient, but you could potentially write an external lookup for this. There's a generic example about how this works in the docs, and an example script that ships with Sp...

The idea is you have a single external lookup for splunk that takes csvfileIndex and surname and the script upon invocation dynamically retrieves addresses from the appropriate file(s), then your search is easy. (The script to accomplish this however...)

MichaelPriest
Communicator

What are the differences between the files, couldn't you try automatic lookup's instead?

0 Karma

andra_pietraru
Path Finder

An automatic lookup would not help, because the content of the csv files is very similar and I would need to lookup only one per event, otherwise my data would get really confusing. That is why I used the csvFileIndex in my events, to make sure I only look at the correct csv file.

0 Karma

woodcock
Esteemed Legend

I do not think you can do this but this kind of thing has come up for me several times. What Splunk needs to do is give us a syntax like !value2literal! where we can do something like this:

... | eval csvName="csvfile".csvfileIndex.".csv" | lookup !csvName! name AS surname OUTPUTNEW address AS homeaddress | ..

The problem is that there is no way that I know of to set a Splunk field's value and use it in a place inside a Splunk command's syntax which takes a stirng-literal, such as the lookup stanza name in your example.

andra_pietraru
Path Finder

Thanks for the reply! I was hoping it's somehow possible. They should add something like this because it's really inconvenient if you have many CSV files.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...