Monitor a set of csv files for errors

[robnewman666]

So I have a particular number of important csv files that I need to ensure have no errors - which I can lookup using the cmd:

find . -name "bad_ips.csv" -exec 2>/dev/null echo {} \; -exec grep -n ",," {} \; | grep -B 1 "^1:"

(I run about 5 of these against the csv files I am most interested in - with 'bad_ips' as an example) 

I am after a way of automating this and being able to run each day and see it in a Splunk dashboard. - is this possible?

[PickleRick]

But what do you want from splunk here? You want to ingest raw files and make splunk check the contents for errors? Or do you want to run the command externally and ingest into splunk its results?

[robnewman666]

Run the cmd externally and Splunk to ingest results. 

[PickleRick · Accepted solution]

Then just do a cron job, write the output to a file and let splunk ingest the file contents. One caveat though. As far as I can see your command does write a "header" with a file name and then file's contents. It would be unpractical to parse such data in splunk. You'd better either write each file's grep output to a separate file so they would be easily distinguishable from one another by source path in splunk or rewrite the command so that each output line is prepended by the grepped file name (you might use paste for that, or a simple bash loop with read/echo)

[ITWhisperer]

Can you output the results to a log (on a daily basis) and ingest the log as events into splunk, and build your dashboard from these events?