topic Re: Splunk Enterprise Security: Look up file in correlation searches not populating in Splunk Enterprise Security

Splunk Enterprise Security: Look up file in correlation searches not populating

wgawhh5hbnht — Wed, 30 Sep 2020 01:37:44 GMT

The following 3 Correlation Searches within ES have the error "lookup file is not populated":

Detect AWS Console Login by User from New Region
Detect AWS Console Login by User from New Country
Detect AWS Console Login by User from New City

The lookup file being referenced is "previously_seen_users_console_logins.csv "

When I look at the contents of the file, here is a redacted & trimmed down version of it:
identity,
arn:aws:sts:::assumed-role//

Why is splunk stating the lookup file is not populated? Is the syntax incorrect?
If there is any documentation on how to set the file up correctly, I would greatly appreciate some guidance on where it is located.

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

jawaharas — Thu, 08 Aug 2019 01:19:01 GMT

You need to enable the scheduled report - 'ESCU - Previously seen users in CloudTrail' or run below query

sourcetype=aws:cloudtrail eventName=ConsoleLogin 
| rename userIdentity.arn as user 
| iplocation src 
| eval City=if(City LIKE "",src,City),Region=if(Region LIKE "",src,Region) 
| stats earliest(_time) as earliest latest(_time) as latest by user src City Region Country 
| outputlookup previously_seen_users_console_logins.csv 
| stats count

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

wgawhh5hbnht — Thu, 08 Aug 2019 17:07:12 GMT

Enabled search, scheduled it, let it run, & still the 3 CS show "lookup file is not populated". If I manually run the search it returns results.

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

jawaharas — Fri, 09 Aug 2019 06:01:55 GMT

Question-
1. Are you getting result if you run (manually) 'ESCU - Previously seen users in CloudTrail' report
2. If yes, are getting result when you run (manually) any of below reports

Detect AWS Console Login by User from New Region
Detect AWS Console Login by User from New Country
Detect AWS Console Login by User from New City

Also, where you notice the error 'lookup file is not populated'? Can you share screenshot?

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

wgawhh5hbnht — Fri, 09 Aug 2019 15:04:11 GMT

Yes
All 3 searches, when ran manually, return NO RESULTS.

The error comes from configure > content management: en-US/app/SplunkEnterpriseSecuritySuite/ess_content_management?textFilter=Detect%20AWS%20Console%20Login%20by%20User%20
Unfortunately I can't upload another screenshot, as the attach picture only allows for a URL...

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

jawaharas — Tue, 13 Aug 2019 03:55:25 GMT

Thanks for the details. Even I get the same error in 'Content Management' page. But, when I run below query ( of 'Detect AWS Console Login by User from New City' Correlation report) manually after removing the 'where' condition, I get results. Can you try and update here whether you get results?

| inputlookup previously_seen_users_console_logins.csv 
| stats min(earliest) as earliest max(latest) as latest by user City 
| join user type=outer 
    [| inputlookup previously_seen_users_console_logins.csv 
    | stats min(earliest) AS earliestseen by user 
    | fields earliestseen user] 
| eval userStatus=if(earliest >= relative_time(now(), "@d"), "New City","Previously Seen City") 
| eval UserData=if(earliestseen >= relative_time(now(), "@d") OR isnull(earliestseen), "New User","Old User") 
| convert ctime(earliest) ctime(latest) ctime(earliestseen) 
| table user City userStatus earliest latest earliestseen

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

wgawhh5hbnht — Tue, 13 Aug 2019 13:51:24 GMT

I also get results when removing the where clause.

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

jawaharas — Wed, 14 Aug 2019 00:47:18 GMT

Then, if you schedule the report, ideally it should work. Regarding the error - "lookup file is not populated", it's better to create a case in Splunk support.

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

jawaharas — Thu, 15 Aug 2019 00:45:09 GMT

@wgawhh5hbnht
Please accept the answer if it significantly helped resolve your query for the benefit of other forum members, who might run into a similar issue.

Re: Splunk Enterprise Security: Look up file in correlation searches not populating

wgawhh5hbnht — Thu, 15 Aug 2019 13:19:16 GMT

I guess I'll be creating a case. The issue is we're using these lookups in Incident Response and we continue to receive alerts on users/cities/regions that we've whitelisted. My assumption is because Splunk believes the lookups are empty. I'm going to leave this open however in the meantime in case someone else has an answer or for when I get an update from Splunk about the issue, then I'll update this.