Splunk Search

after upgrade to 4.1 unable to get fields from search using python

imrago
Contributor

After upgrading to 4.1 from 4.0.10 I am unable to get fields using a search from python script. The simplified version :

from splunk import auth, search
import time
import csv
import os.path

auth.getSessionKey('admin','xxxxx')
job = search.dispatch('search host=foobar  | head 1')

while not job.isDone:
    time.sleep(1)

for event in job:
    try:
        band=event['band']
    except:
                band=""
    print band

job.cancel()

As I see from documentation for 4.1, a lot is changed in CLI search. I was able only to retrieve the statndard fields : '_time', 'host', 'index', 'source', 'sourcetype' ...

Is there a solution to get fields using a search in python script?

Tags (2)
0 Karma
1 Solution

Johnvey
Contributor

You need to tell the server to get the fields you want:

job = search.dispatch('search host=foobar  | head 1', required_field_list='my_field1 my_field2')

Or, you can instruct the server to discover all fields:

job = search.dispatch('search host=foobar  | head 1', required_field_list='*')

Be aware that asking for all fields can slow the total search time by 5x. Specifying required_field_list is not required for fields that result from transforming commands like timechart, stats, or ctable.

The default behavior was changed to minimize overhead on searches that don't need to keep field summaries.

View solution in original post

jrodman
Splunk Employee
Splunk Employee

As Johnvey points out, the pull-out of all fields by default was a considered a misfeature, since it has performance costs for things you may have no interest in. If your search requests or makes use of any of the fields explicitly, then the fields mentioned will be pulled out.

Johnvey
Contributor

You need to tell the server to get the fields you want:

job = search.dispatch('search host=foobar  | head 1', required_field_list='my_field1 my_field2')

Or, you can instruct the server to discover all fields:

job = search.dispatch('search host=foobar  | head 1', required_field_list='*')

Be aware that asking for all fields can slow the total search time by 5x. Specifying required_field_list is not required for fields that result from transforming commands like timechart, stats, or ctable.

The default behavior was changed to minimize overhead on searches that don't need to keep field summaries.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...