Getting Data In

Why do I get an error about sid_lookup after upgrading to 4.1?

Path Finder

I've just upgraded to 4.1 and now I'm getting an error when I search saying:

The lookup table 'sid_lookup' does not exist. It is referenced by configuration 'source::WinEventLog...'.
The lookup table 'sid_lookup' does not exist. It is referenced by configuration 'source::WMI:WinEventLog...'.

What do I need to do to get these working again?

1 Solution

Splunk Employee
Splunk Employee

On older version of the Windows app we were using a LOOKUP to find the SID of event. This is no longer the case as of 4.1. To resolve this error, edit the default/props.conf file in your windows app to the following:

# Applying GUID, SID traslation to the "event_guid, event_sid" fields in WinEventLog sourcetype events
# [source::WinEventLog...]
# LOOKUP-GUID = guid_lookup guid_lookup AS guid_to_trans OUTPUT dcName AS guid_dcname
# LOOKUP-SID = sid_lookup sid_lookup AS sid_to_trans OUTPUT cn AS sid_cn dcName as sid_dcname

# Applying GUID, SID traslation to the "guid_raw, sid_raw" fields in WMI WinEventLog sourcetype events.
# By looking up the values of those two fields, two new fields are generated, guid_name, sid_name
# [source::WMI:WinEventLog...]
# LOOKUP-GUID = guid_lookup guid_lookup AS guid_to_trans OUTPUT dcName AS guid_dcname
# LOOKUP-SID = sid_lookup sid_lookup AS sid_to_trans OUTPUT cn AS sid_cn dcName as sid_dcname

View solution in original post

Splunk Employee
Splunk Employee

The first part means a search is running in a context where it can't find the sid_lookup lookup. The second part tells you why the lookup is being called in the first place.

There are a few reasons why the sid_lookup isn't found. Make sure the lookup is defined by looking in manager (Manager > Lookups > Lookup definitions) or in the configs (http://www.splunk.com/base/Documentation/latest/Knowledge/Addfieldsfromexternaldatasources).I've found the most common issue isn't a missing definition, but one that is not shared to the appropriate level. For instance, your lookup might be in available in the Windows app, but if the lookup gets called from the search app it won't be available.

0 Karma

Splunk Employee
Splunk Employee
0 Karma

Splunk Employee
Splunk Employee

This is caused because of an older version of wondows. The workaround for the index paths, where we switched / to \ isn't being done transparently at the config layer, is listed below:

Your current etc/apps/unix/default/indexes.conf looks like this:

[os] homePath = $SPLUNK_DB/os/db coldPath = $SPLUNK_DB/os/colddb thawedPath = $SPLUNK_DB/os/thaweddb

You can override this in etc/apps/unix/local/indexes.conf with settings like this:

[os] homePath = $SPLUNK_DB\os\db coldPath = $SPLUNK_DB\os\colddb thawedPath = $SPLUNK_DB\os\thaweddb"

Also, check and verify that the lookup table isn't commented out in the props.conf file.

0 Karma

Splunk Employee
Splunk Employee

The guid_lookup and sid_lookup tables were introduced with the 4.0 release. The idea was to created a mapping of guids/sids and their object names during indexing of Active Directory Monitoring events. At search time this mapping would be referenced to translate guids/sids seen in Windows event logs.

In 4.0.7 release or earlier this method of translating guid/sid objects was disabled and another more reliable method was implemented. As Windows event logs are being read in a given system, we look for guids/sids and if one found we try to translate it by using a local Active Directory that the machine where Splunk is running on belongs to.

By default this feature is only enabled for Security Windows event logs, and in apps\Windows\default\inputs.conf it looks like this:

[default]
evt_dc_name =
evt_dns_name =

[WinEventLog:Security]
disabled = 1
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5

Field evt_resolve_ad_obj = 1/0 controls if the AD object(guid/sid) translation is enabled or not.

Fields evt_dc_name, evt_dns_name are used to specify the DC and DNS servers. If you leave them empty, we will bind to the default servers.

Conclusion, if guid_lookup and sid_lookup tables are enabled an being referenced some where, they should be disabled and their reference removed.

Cheers,
Ledio

Super Champion

Alan, are you having this problem only with built-in lookup tables or with custom lookups as well? I have user defined lookups that are giving me the same error message. I was suspecting that this is a permissions issue because the problem only appears in apps other than the one where the lookup is defined in. All the permission seem to be correct (transforms, props, and lookup/ are all globally shared). The same config worked fine in 4.0.

0 Karma

Splunk Employee
Splunk Employee

On older version of the Windows app we were using a LOOKUP to find the SID of event. This is no longer the case as of 4.1. To resolve this error, edit the default/props.conf file in your windows app to the following:

# Applying GUID, SID traslation to the "event_guid, event_sid" fields in WinEventLog sourcetype events
# [source::WinEventLog...]
# LOOKUP-GUID = guid_lookup guid_lookup AS guid_to_trans OUTPUT dcName AS guid_dcname
# LOOKUP-SID = sid_lookup sid_lookup AS sid_to_trans OUTPUT cn AS sid_cn dcName as sid_dcname

# Applying GUID, SID traslation to the "guid_raw, sid_raw" fields in WMI WinEventLog sourcetype events.
# By looking up the values of those two fields, two new fields are generated, guid_name, sid_name
# [source::WMI:WinEventLog...]
# LOOKUP-GUID = guid_lookup guid_lookup AS guid_to_trans OUTPUT dcName AS guid_dcname
# LOOKUP-SID = sid_lookup sid_lookup AS sid_to_trans OUTPUT cn AS sid_cn dcName as sid_dcname

View solution in original post

Splunk Employee
Splunk Employee

How is the SID provided in 4.1? Is this setting correct in the context of 4.0 forwarders?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!