Getting Data In

How to upload a csv from a host that has a universal forwarder?

danielbb
Motivator

We have a universal forwarder and the customer has a csv file on this machine that he would like to ingest. The customer would like to ingest it as a lookup so I wonder whether we should ingest the csv via the UF or potentially, send it via the REST api to be uploaded as a lookup. Does the latter option make sense?

Labels (3)
Tags (1)
0 Karma

marycordova
SplunkTrust
SplunkTrust

Does the file ever change?

If so, I would index the file and then create a scheduled search to update the lookup based on the indexed data.  

If it never changes, just import the file one time with the Lookup Editor App.

@marycordova

livehybrid
Super Champion

Hi @danielbb 

There was a very similar question the other day around this, please see my answer below or check out the original question at https://community.splunk.com/t5/Splunk-Enterprise-Security/Can-Splunk-read-a-CSV-file-and-automatica...

The other option as you mentioned would be to use the REST API - There are some scripts at https://github.com/mthcht/lookup-editor_scripts#readme. which aim to achieve this if this is the route you wanted to go down.

@livehybrid wrote:

 

If you have a CSV on a forwarder that you want to become a lookup in Splunk then the best way to achieve this is probably to monitor (using monitor:// in inputs.conf) the file and send it to a specific index on your Splunk indexers.

Then, Create scheduled search which searches that index and retrieves the sent data and outputs it to a lookup (using | outputlookup command). Depending on how/when the CSV is updated may depend on exactly how the resulting search ends up, but ultimately this should be a viable solution.

There may be other solutions but would require significantly more engineering effort. 

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing


 

Get Updates on the Splunk Community!

Splunk App for Anomaly Detection End of Life Announcment

Q: What is happening to the Splunk App for Anomaly Detection?A: Splunk is officially announcing the ...

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...