Splunk Search

after upgrade to 4.1 unable to get fields from search using python

imrago
Contributor

After upgrading to 4.1 from 4.0.10 I am unable to get fields using a search from python script. The simplified version :

from splunk import auth, search
import time
import csv
import os.path

auth.getSessionKey('admin','xxxxx')
job = search.dispatch('search host=foobar  | head 1')

while not job.isDone:
    time.sleep(1)

for event in job:
    try:
        band=event['band']
    except:
                band=""
    print band

job.cancel()

As I see from documentation for 4.1, a lot is changed in CLI search. I was able only to retrieve the statndard fields : '_time', 'host', 'index', 'source', 'sourcetype' ...

Is there a solution to get fields using a search in python script?

Tags (2)
0 Karma
1 Solution

Johnvey
Contributor

You need to tell the server to get the fields you want:

job = search.dispatch('search host=foobar  | head 1', required_field_list='my_field1 my_field2')

Or, you can instruct the server to discover all fields:

job = search.dispatch('search host=foobar  | head 1', required_field_list='*')

Be aware that asking for all fields can slow the total search time by 5x. Specifying required_field_list is not required for fields that result from transforming commands like timechart, stats, or ctable.

The default behavior was changed to minimize overhead on searches that don't need to keep field summaries.

View solution in original post

jrodman
Splunk Employee
Splunk Employee

As Johnvey points out, the pull-out of all fields by default was a considered a misfeature, since it has performance costs for things you may have no interest in. If your search requests or makes use of any of the fields explicitly, then the fields mentioned will be pulled out.

Johnvey
Contributor

You need to tell the server to get the fields you want:

job = search.dispatch('search host=foobar  | head 1', required_field_list='my_field1 my_field2')

Or, you can instruct the server to discover all fields:

job = search.dispatch('search host=foobar  | head 1', required_field_list='*')

Be aware that asking for all fields can slow the total search time by 5x. Specifying required_field_list is not required for fields that result from transforming commands like timechart, stats, or ctable.

The default behavior was changed to minimize overhead on searches that don't need to keep field summaries.

Get Updates on the Splunk Community!

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...