LookupOperator errors on a forwarder?

[Lowell]

I've started to see the following messages from some of my forwarding instances of splunk:

11-30-2010 16:50:02.355 ERROR LookupOperator - The lookup table 'sqlagent_jobs' does not exist. It is referenced by configuration 'mssql_processes'.

This error is somewhat correct. I do have "sqlagent_jobs" defined in transforms.conf, but the sqlagent_jobs.csv is missing on the forwarder's app. What I'm trying to figure out is why the forwarder cares.

I'm familiar with getting these LookupOperator errors on the search-head (and interactively) when I forget part of the lookup configuration or have some kind of permissions issue--but I thought that was strictly caused by running a search. On my forwarders, there are no running searches. (I use heavy-weight forwarders, so it's possible to run search on them; but there's no locally indexed data to search; and I see no entries in searches.log or scheduler.log so I don't think that's the case, but I could be missing something.)

Does anyone know what else would trigger this error?

(As a work around, I'm commenting out the LOOKUP-* entries in my deployment-apps app for the moment, but I don't want to have to maintain two versions of my app if I don't have too. I use the same app on the search-head and on my forwarders)

[kamal_jagga]

Check if there was any Windows/OS upgrade/update on the Server.

Restarting the RPC Server of dbconnect and Splunk restart should fix the issue.

[kamal_jagga]

I am also getting the same error on my dbconnect windows server. This started coming over the weekend and no changes were made to any file or config.
Kindly advise.

[steveyz]

I'm guessing there are actually searches being run on your heavyweight forwarders. By default I believe we have scheduled searches that run that populate various status dashboards and it may be these searches that are triggering the error.

[steveyz]

Hmm, do you ever hit the UI on your forwarders?

[Lowell]

I think I disabled all the of internal searches for this very reason. But supposing I missed one, can you think of any reasons why I'm not seeing anything logged in searches.log, scheduler.log (both of these are 0 bytes on the forwarder), audit.log has nothing about running searches, and there's nothing in dispatch folder. Is there anything else I should look for?