Getting Data In

Federated Search -How do I create lookup file with results?

discenzadoe
Explorer

We are working with several remote datasets that are combined to give our end user a specific result. 

Federated Search gives us an LDAP dn, which we are trying to use to pull enhancing information from another remote source via a REST API.  The following search works:

 

index=federated:remote_dataset userid="cn=" | \
      eval dn=lower(userid) | \
      dedup dn | \
      table dn

 

The idea is to use a scheduled search to populate a csv with a list of DNs at the top of every hour, then use a cron job to spawn a python script which generates a new CSV that contains the DN and the enhancing data from the REST API source. Our python script is working, however when we add "|outputlookup dn.csv append=true" to the otherwise functional SPL, we get nothing.

This fails:

 

index=federated:remote_dataset userid="cn=" | \
      eval dn=lower(userid) | \
      dedup dn | \
      table dn | \
      outputlookup dn.csv append=true

 

Is this a limitation of Federated Search?

Thank you

Labels (1)
0 Karma

somesoni2
Revered Legend

Do you see any error when running the search? (in Job dropdown you should see some message).

0 Karma

_joe
Contributor

Similar issue. There are no error logs per say.  The search log shows the the output appears to be happening on the remote SH. 

Results written to file '/opt/splunk/etc/apps/search/lookups/mylookup.csv' on serverName=',<<remoteServerName>>

In other words, if I login to my local search head and run this and get an output of 100 entries:

| federated from:my report | outputlookup mylookup.csv

Then I run this (Again on the local search head), it will be empty:

| inputlookup mylookup.csv

 

 

0 Karma

_joe
Contributor

You can use '| append [ | noop ]' as a workaround:

| from federated <> 
| append [ | noop ]
| outputlookup <>.csv

 

0 Karma
Get Updates on the Splunk Community!

Message Parsing in SOCK

Introduction This blog post is part of an ongoing series on SOCK enablement. In this blog post, I will write ...

Exploring the OpenTelemetry Collector’s Kubernetes annotation-based discovery

We’ve already explored a few topics around observability in a Kubernetes environment -- Common Failures in a ...

Use ‘em or lose ‘em | Splunk training units do expire

Whether it’s hummus, a ham sandwich, or a human, almost everything in this world has an expiration date. And, ...