Splunk Search

after upgrade to 4.1 unable to get fields from search using python

imrago
Contributor

After upgrading to 4.1 from 4.0.10 I am unable to get fields using a search from python script. The simplified version :

from splunk import auth, search
import time
import csv
import os.path

auth.getSessionKey('admin','xxxxx')
job = search.dispatch('search host=foobar  | head 1')

while not job.isDone:
    time.sleep(1)

for event in job:
    try:
        band=event['band']
    except:
                band=""
    print band

job.cancel()

As I see from documentation for 4.1, a lot is changed in CLI search. I was able only to retrieve the statndard fields : '_time', 'host', 'index', 'source', 'sourcetype' ...

Is there a solution to get fields using a search in python script?

Tags (2)
0 Karma
1 Solution

Johnvey
Contributor

You need to tell the server to get the fields you want:

job = search.dispatch('search host=foobar  | head 1', required_field_list='my_field1 my_field2')

Or, you can instruct the server to discover all fields:

job = search.dispatch('search host=foobar  | head 1', required_field_list='*')

Be aware that asking for all fields can slow the total search time by 5x. Specifying required_field_list is not required for fields that result from transforming commands like timechart, stats, or ctable.

The default behavior was changed to minimize overhead on searches that don't need to keep field summaries.

View solution in original post

jrodman
Splunk Employee
Splunk Employee

As Johnvey points out, the pull-out of all fields by default was a considered a misfeature, since it has performance costs for things you may have no interest in. If your search requests or makes use of any of the fields explicitly, then the fields mentioned will be pulled out.

Johnvey
Contributor

You need to tell the server to get the fields you want:

job = search.dispatch('search host=foobar  | head 1', required_field_list='my_field1 my_field2')

Or, you can instruct the server to discover all fields:

job = search.dispatch('search host=foobar  | head 1', required_field_list='*')

Be aware that asking for all fields can slow the total search time by 5x. Specifying required_field_list is not required for fields that result from transforming commands like timechart, stats, or ctable.

The default behavior was changed to minimize overhead on searches that don't need to keep field summaries.

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

What Is Splunk? Here’s What You Can Do with Splunk

Hey Splunk Community, we know you know Splunk. You likely leverage its unparalleled ability to ingest, index, ...

Level Up Your .conf25: Splunk Arcade Comes to Boston

With .conf25 right around the corner in Boston, there’s a lot to look forward to — inspiring keynotes, ...

Manual Instrumentation with Splunk Observability Cloud: How to Instrument Frontend ...

Although it might seem daunting, as we’ve seen in this series, manual instrumentation can be straightforward ...