Getting Data In

How to upload a csv from a host that has a universal forwarder?

danielbb
Motivator

We have a universal forwarder and the customer has a csv file on this machine that he would like to ingest. The customer would like to ingest it as a lookup so I wonder whether we should ingest the csv via the UF or potentially, send it via the REST api to be uploaded as a lookup. Does the latter option make sense?

Labels (3)
Tags (1)
0 Karma

marycordova
SplunkTrust
SplunkTrust

Does the file ever change?

If so, I would index the file and then create a scheduled search to update the lookup based on the indexed data.  

If it never changes, just import the file one time with the Lookup Editor App.

@marycordova

livehybrid
Super Champion

Hi @danielbb 

There was a very similar question the other day around this, please see my answer below or check out the original question at https://community.splunk.com/t5/Splunk-Enterprise-Security/Can-Splunk-read-a-CSV-file-and-automatica...

The other option as you mentioned would be to use the REST API - There are some scripts at https://github.com/mthcht/lookup-editor_scripts#readme. which aim to achieve this if this is the route you wanted to go down.

@livehybrid wrote:

 

If you have a CSV on a forwarder that you want to become a lookup in Splunk then the best way to achieve this is probably to monitor (using monitor:// in inputs.conf) the file and send it to a specific index on your Splunk indexers.

Then, Create scheduled search which searches that index and retrieves the sent data and outputs it to a lookup (using | outputlookup command). Depending on how/when the CSV is updated may depend on exactly how the resulting search ends up, but ultimately this should be a viable solution.

There may be other solutions but would require significantly more engineering effort. 

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing


 

Get Updates on the Splunk Community!

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...