Splunk Search

Monitor a set of csv files for errors

robnewman666
Path Finder

So I have a particular number of important csv files that I need to ensure have no errors - which I can lookup using the cmd:

find . -name "bad_ips.csv" -exec 2>/dev/null echo {} \; -exec grep -n ",," {} \; | grep -B 1 "^1:"

(I run about 5 of these against the csv files I am most interested in - with 'bad_ips' as an example) 

I am after a way of automating this and being able to run each day and see it in a Splunk dashboard. - is this possible?

Labels (1)
Tags (2)
0 Karma
1 Solution

PickleRick
SplunkTrust
SplunkTrust

Then just do a cron job, write the output to a file and let splunk ingest the file contents. One caveat though. As far as I can see your command does write a "header" with a file name and then file's contents. It would be unpractical to parse such data in splunk. You'd better either write each file's grep output to a separate file so they would be easily distinguishable from one another by source path in splunk or rewrite the command so that each output line is prepended by the grepped file name (you might use paste for that, or a simple bash loop with read/echo)

View solution in original post

PickleRick
SplunkTrust
SplunkTrust

But what do you want from splunk here? You want to ingest raw files and make splunk check the contents for errors? Or do you want to run the command externally and ingest into splunk its results?

0 Karma

robnewman666
Path Finder

Run the cmd externally and Splunk to ingest results. 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Then just do a cron job, write the output to a file and let splunk ingest the file contents. One caveat though. As far as I can see your command does write a "header" with a file name and then file's contents. It would be unpractical to parse such data in splunk. You'd better either write each file's grep output to a separate file so they would be easily distinguishable from one another by source path in splunk or rewrite the command so that each output line is prepended by the grepped file name (you might use paste for that, or a simple bash loop with read/echo)

ITWhisperer
SplunkTrust
SplunkTrust

Can you output the results to a log (on a daily basis) and ingest the log as events into splunk, and build your dashboard from these events?

0 Karma
Get Updates on the Splunk Community!

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...

SOCin’ it to you at Splunk University

Splunk University is expanding its instructor-led learning portfolio with dedicated Security tracks at .conf25 ...

Credit Card Data Protection & PCI Compliance with Splunk Edge Processor

Organizations handling credit card transactions know that PCI DSS compliance is both critical and complex. The ...