Getting Data In

How to upload a csv from a host that has a universal forwarder?

danielbb
Motivator

We have a universal forwarder and the customer has a csv file on this machine that he would like to ingest. The customer would like to ingest it as a lookup so I wonder whether we should ingest the csv via the UF or potentially, send it via the REST api to be uploaded as a lookup. Does the latter option make sense?

Labels (3)
Tags (1)
0 Karma

marycordova
SplunkTrust
SplunkTrust

Does the file ever change?

If so, I would index the file and then create a scheduled search to update the lookup based on the indexed data.  

If it never changes, just import the file one time with the Lookup Editor App.

@marycordova

livehybrid
Super Champion

Hi @danielbb 

There was a very similar question the other day around this, please see my answer below or check out the original question at https://community.splunk.com/t5/Splunk-Enterprise-Security/Can-Splunk-read-a-CSV-file-and-automatica...

The other option as you mentioned would be to use the REST API - There are some scripts at https://github.com/mthcht/lookup-editor_scripts#readme. which aim to achieve this if this is the route you wanted to go down.

@livehybrid wrote:

 

If you have a CSV on a forwarder that you want to become a lookup in Splunk then the best way to achieve this is probably to monitor (using monitor:// in inputs.conf) the file and send it to a specific index on your Splunk indexers.

Then, Create scheduled search which searches that index and retrieves the sent data and outputs it to a lookup (using | outputlookup command). Depending on how/when the CSV is updated may depend on exactly how the resulting search ends up, but ultimately this should be a viable solution.

There may be other solutions but would require significantly more engineering effort. 

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing


 

Get Updates on the Splunk Community!

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...

SOCin’ it to you at Splunk University

Splunk University is expanding its instructor-led learning portfolio with dedicated Security tracks at .conf25 ...

Credit Card Data Protection & PCI Compliance with Splunk Edge Processor

Organizations handling credit card transactions know that PCI DSS compliance is both critical and complex. The ...