Splunk Search

Monitor a set of csv files for errors

robnewman666
Path Finder

So I have a particular number of important csv files that I need to ensure have no errors - which I can lookup using the cmd:

find . -name "bad_ips.csv" -exec 2>/dev/null echo {} \; -exec grep -n ",," {} \; | grep -B 1 "^1:"

(I run about 5 of these against the csv files I am most interested in - with 'bad_ips' as an example) 

I am after a way of automating this and being able to run each day and see it in a Splunk dashboard. - is this possible?

Labels (2)
Tags (2)
0 Karma
1 Solution

PickleRick
Ultra Champion

Then just do a cron job, write the output to a file and let splunk ingest the file contents. One caveat though. As far as I can see your command does write a "header" with a file name and then file's contents. It would be unpractical to parse such data in splunk. You'd better either write each file's grep output to a separate file so they would be easily distinguishable from one another by source path in splunk or rewrite the command so that each output line is prepended by the grepped file name (you might use paste for that, or a simple bash loop with read/echo)

View solution in original post

PickleRick
Ultra Champion

But what do you want from splunk here? You want to ingest raw files and make splunk check the contents for errors? Or do you want to run the command externally and ingest into splunk its results?

0 Karma

robnewman666
Path Finder

Run the cmd externally and Splunk to ingest results. 

0 Karma

PickleRick
Ultra Champion

Then just do a cron job, write the output to a file and let splunk ingest the file contents. One caveat though. As far as I can see your command does write a "header" with a file name and then file's contents. It would be unpractical to parse such data in splunk. You'd better either write each file's grep output to a separate file so they would be easily distinguishable from one another by source path in splunk or rewrite the command so that each output line is prepended by the grepped file name (you might use paste for that, or a simple bash loop with read/echo)

ITWhisperer
SplunkTrust
SplunkTrust

Can you output the results to a log (on a daily basis) and ingest the log as events into splunk, and build your dashboard from these events?

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...