Getting Data In

Problem with creating an alert that compares a csv file with actual events

babcolee
Path Finder

I am having a problem with creating an alert that compares a csv file with actual events. I have taken an asset record, this should be the authoritative list, and put it into a csv file (1 column of 515 entries ) and used the events coming into Splunk (approx. 360 events in a 24 hour period) to do a comparison. I should be getting a discrepancy of 155 and trigger an alert listing those 155 which have not reported into Splunk. I have fork lifted the logic from an alert of a Splunk instance which is working to the other Splunk instance. I have changed the index, fields and the csv file to match fields I need but I am not getting the expected results. The following search works on another instance of Splunk but will not work on the other Splunk instance. I am not getting any errors just not an alert.

Works:
... search | stats count as reccount by collector | append [| inputlookup collectors.csv | table collector | eval reccount = 0] | eventstats sum(reccount) as count by collector | where count == 0

Doesn't work on a different Splunk instance:
...search | stats count as reccount by server_console | append [| inputlookup assets.csv | table asset | eval reccount = 0] | eventstats sum(reccount) as count by asset | where count == 0

I have also tried
...search | dedup server_console | lookup assets.csv assets OUTPUT assets AS foundInLookup | where isnull(foundInLookup) | table server_console

and...
dedup server_console | lookup assets_per_jira.csv asset OUTPUT asset AS foundInLookup | where isnull(foundInLookup) | table server_console

Thank you for any help you can provide

0 Karma

babcolee
Path Finder

Here is where the lookup tables are located and here is the permissions. The asset_test_jira.csv has less entries in an attempt to trigger an alert. However, both lookups do not trigger an alert. I have also run the search with just the lookup part of the command and it shows the table.

[splunk@soctxsplunk02:~/etc/apps/search/lookups]$ ll
total 3384
-rw-------. 1 splunk splunk 6588 Feb 16 21:43 assets_per_jira.csv
-rw-------. 1 splunk splunk 113 Feb 21 19:13 asset_test_jira.csv

alt text

0 Karma

tiagofbmm
Influencer

You are using different lookups, so if the exact same search doesn't work in a Splunk instance, then probably you won't have the lookup in place or with the correct permissions.

Can you test the search for the time period you know it should raise an alert? Can you also check the lookups you are using exist in the Splunk instance you are running the search?

0 Karma

valiquet
Contributor

Is your lookup blacklisted from the bundle? If yes, use local=true

0 Karma

babcolee
Path Finder

Thank you for your response. This is running on a Search Head / Indexer Splunk instance so there shouldn't need to be any remote searches.

0 Karma

valiquet
Contributor

Does your searches work outside the scheduler? If yes, can you give us savedsearches.conf?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...