A lookup table could not be created

[aelliott]

I'm using Enterprise Security and am getting the following:
Using SA-IdentityManagement

Populating identities using ldapsearch to lookup within scheduled search.. lookup then populates using Identity Management
Populating Assets into csv, feeding to lookup file with automated search.. lookup file then populates identity table with Identity Management

lookup_conversion: A lookup table could not be created (key: identity, tempfile: C:\Program Files\Splunk\var\run\splunk\lookup_tmp\lookup_convwnyv1y.txt)

Troubleshooting: checked splunkd.log, no messages
To resolve temporarily: recreated identity_expanded.csv, assets_by_str.csv manually (using the .default template), csv's will repopulate automagically with data. When it tries to do it automatically, the csv's are deleted and splunk is unable to recreate. Checked access and account has full access to csv's and directory.

Here's some more details that I found within the _internal index.

014-04-09 09:02:10,568 ERROR pid=9620 tid=asset file=writers.py:move_lookups:156 
| FAILURE: A lookup table could not be created: (key: dns, tempfile: C:\Program Files\Splunk\var\run\splunk\lookup_tmp\lookup_convelz2ua.txt)

2014-04-08 14:05:34,845 ERROR pid=8512 tid=identity file=writers.py:_move_lookup:106 
| FAILURE: A lookup table could not be created: identities_expanded.csv

2014-04-08 14:05:30,180 ERROR pid=8512 tid=asset file=writers.py:_move_lookup:106 
| FAILURE: A lookup table could not be created: assets_by_str.csv

And More Logs:

 2014-04-09 06:02:36,535 ERROR pid=4588 tid=asset file=writers.py:_move_lookup:98 | EXCEPTION: Could not rename file after multiple retries src=C:\Program Files\Splunk\var\run\splunk\lookup_tmp\lookup_convqgehyc.txt dst=C:\Program Files\Splunk\etc\apps\SA-IdentityManagement\lookups\assets_by_str.csv
    Traceback (most recent call last):
      File "C:\Program Files\Splunk\etc\apps\SA-Utils\lib\SolnCommon\lookup_conversion\writers.py", line 85, in _move_lookup
        os.unlink(dst_lookup_path)
    WindowsError: [Error 5] Access is denied: 'C:\\Program Files\\Splunk\\etc\\apps\\SA-IdentityManagement\\lookups\\assets_by_str.csv'

    2014-04-09 06:02:36,535 ERROR pid=4588 tid=asset file=writers.py:_move_lookup:106 | FAILURE: A lookup table could not be created: assets_by_str.csv

[aelliott · Accepted solution]

aha! this is a known issue:

http://docs.splunk.com/Documentation/ES/latest/RN/KnownIssues

On a Windows search head, the asset and identity center shows no results. Error messages will be displayed on the search head about missing lookup files. The python_modular_inputs.log reports errors:
ERROR pid=4040 tid=asset file=writers.py:_move_lookup:108 | FAILURE: Temporary output file was not created: C:\Program Files\Splunk\var\run\splunk\lookup_tmp\lookup_conv6jppog.txt
ERROR pid=4040 tid=asset file=writers.py:move_lookups:156 | FAILURE: A lookup table could not be created: (key: cidr, tempfile: C:\Program Files\Splunk\var\run\splunk\lookup_tmp\lookup_conv6jppog.txt)
The asset and identity lookup creation and expansion process is not working correctly due to an issue with a python script on Windows. Please contact Splunk Support for a replacement script and reference SOLNESS-4642. (SOLNESS-4642)
Once the script is obtained, follow the instructions below:
1. Replace the writers.py script in $SPLUNK_HOME\etc\apps\SA-Utils\lib\SolnCommon\lookup_conversion
2. Make sure all the *.csv's in SA-IdentityManagement\lookups are there, and if not create a new copy from the *.csv.default files.
3. Delete all the contents under $SPLUNK_HOME\var\lib\splunk\modinputs\identity_manager
4. Restart Splunk Enterprise

[aelliott]

Received writers.py and followed instructions as stated above, already working perfectly

[aelliott]

Update: Still waiting on a response from splunk support after 11 full business days.

[aelliott]

I have submitted my issue to splunk support and if the resolution in this post works, then i will mark this as answer.