Getting Data In

Federated Search -How do I create lookup file with results?

discenzadoe
Explorer

We are working with several remote datasets that are combined to give our end user a specific result. 

Federated Search gives us an LDAP dn, which we are trying to use to pull enhancing information from another remote source via a REST API.  The following search works:

 

index=federated:remote_dataset userid="cn=" | \
      eval dn=lower(userid) | \
      dedup dn | \
      table dn

 

The idea is to use a scheduled search to populate a csv with a list of DNs at the top of every hour, then use a cron job to spawn a python script which generates a new CSV that contains the DN and the enhancing data from the REST API source. Our python script is working, however when we add "|outputlookup dn.csv append=true" to the otherwise functional SPL, we get nothing.

This fails:

 

index=federated:remote_dataset userid="cn=" | \
      eval dn=lower(userid) | \
      dedup dn | \
      table dn | \
      outputlookup dn.csv append=true

 

Is this a limitation of Federated Search?

Thank you

Labels (1)
0 Karma

somesoni2
Revered Legend

Do you see any error when running the search? (in Job dropdown you should see some message).

0 Karma

_joe
Contributor

Similar issue. There are no error logs per say.  The search log shows the the output appears to be happening on the remote SH. 

Results written to file '/opt/splunk/etc/apps/search/lookups/mylookup.csv' on serverName=',<<remoteServerName>>

In other words, if I login to my local search head and run this and get an output of 100 entries:

| federated from:my report | outputlookup mylookup.csv

Then I run this (Again on the local search head), it will be empty:

| inputlookup mylookup.csv

 

 

0 Karma

_joe
Contributor

You can use '| append [ | noop ]' as a workaround:

| from federated <> 
| append [ | noop ]
| outputlookup <>.csv

 

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...