topic Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge? in Splunk Enterprise Security

Why is the assets_by_cidr.csv lookup file not populating during asset merge?

panovattack — Tue, 29 Sep 2020 11:36:28 GMT

During searches in Enterprise Security, I get the following error:

Empty csv lookup file (contains only a header) for table 'asset_lookup_by_cidr': /opt/splunk/var/run/searchpeers/{hostname}-1477514321/apps/SA-IdentityManagement/lookups/assets_by_cidr.csv

We recently updated our asset information from Active Directory and our network data. When we look at the assets_by_cidr.csv lookup file, it is not populating after a merge.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

smoir_splunk — Wed, 26 Oct 2016 21:12:59 GMT

What version of ES do you have? If 4.5, check to make sure that the corresponding saved search is running.
http://docs.splunk.com/Documentation/ES/4.5.0/User/AssetandIdentityMerging

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

panovattack — Tue, 29 Sep 2020 11:36:31 GMT

4.1.1, we have tried a forced merge. Also when we search index=_internal source=*python_modular_input.log (asset OR identity) we get:

2016-10-26 16:13:41,322 INFO pid=3406 tid=MainThread file=lookup_modinput.py:collect_files:130 | status="Lookup table file found" name=asset_list category=asset path=/opt/splunk/etc/apps/SA-IdentityManagement/lookups/asset_list.csv size=##### last_updated=########

I removed the actual numbers with #'s.

I can manually search and validate the lookup file exists at that location.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

panovattack — Tue, 29 Sep 2020 11:36:34 GMT

I just did another merge and I see:

index=_internal source=*python_modular_input.log (asset OR identity)

Result:
2016-10-26 16:45:10,570 INFO pid=3702 tid=asset file=lookup_modinput.py:streaming_merge_task:310 | status="Lookup table updated" target="asset_lookup_by_cidr" file="/opt/splunk/var/run/splunk/lookup_tmp/lookup_convaowpMV.txt"

However, "| inputlookup asset_lookup_by_cidr" still returns no results.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

smoir_splunk — Tue, 29 Sep 2020 11:33:56 GMT

If you review the asset_lookup_by_cidr in the lists and lookups view in the UI, does it show content there?

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

panovattack — Wed, 26 Oct 2016 22:32:13 GMT

No, I see no data.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

panovattack — Tue, 29 Sep 2020 11:36:52 GMT

Does anyone have a list of the fields that should be in that lookup? (assets_by_cidr.csv)

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

nnmiller — Thu, 27 Oct 2016 17:47:38 GMT

Look for index=_internal sourcetype=splunkd or source=*python_modular* ERROR over the last hour or so (it runs every 5 minutes). If they aren't merging it generally means the process is erroring out on an entry in one of the CSVs being merged--header issue, problem with field contents of a line, etc.

It will be an iterative process, as the merge process bails after the first error.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

panovattack — Fri, 04 Nov 2016 13:10:35 GMT

This was resolved. No data actually had a CIDR block designation, we made the correction and everything started working.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

panovattack — Mon, 27 Feb 2017 00:33:56 GMT

I had no CIDR entries...fixed.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

hatalla — Tue, 29 Sep 2020 18:13:00 GMT

Hey Panovattack,

It looks to me that the expandiprange.py Python script that is called to populate the assets_by_cidr.csv lookup requires that the IP field in | asset_sources to be either an IP range (as in 10.250.20.01-10.250.20.255 ) or in CIDR notation (e.g. 10.0.0.0/8) if it is just an IP the lookup just doesn't populate because one of the macros regex down the | line filters on IPs in CIDR notations.

So how did you get those CIDR entries in the IP or how did you populate the assets_by_cidr.csv lookup?

Thanks.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

riqbal47010 — Mon, 10 Feb 2020 11:34:01 GMT

can you please share the steps

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

pcarlow_splunk — Wed, 30 Sep 2020 05:08:30 GMT

In ES 6.0 assets_by_cidr.csv lookup was migrated to asset_lookup_by_cidr kvstore lookup.

You can populate the KV store table with a dummy entry to remove the message.

Navigate to ES App > Configure > Data Enrichment > Asset and Identity Management
(/en-US/app/SplunkEnterpriseSecuritySuite/ess_entity_management)

On the Asset Lookup Configuration tab, ensure static_assets Status is set to Enable. If not, click the Enable link.

Click the Source simple_asset_lookup and the editor will open in a new window.
Type in 192.168.0.1/30 to the "ip" field and save it.

Re: Why is the assets_by_cidr.csv lookup file not populating during asset merge?

khalidewaidah — Mon, 10 Aug 2020 14:31:00 GMT

Hi , Could you let me know how do you solve it