Splunk Search

Why am I unable to set new fields in a custom search streaming command?

j8lp
Explorer

I'm writing a custom search command to convert all the full path xml names to just local names. I'm also making the field names all lower case for consistency. My code is below:

    def stream(self, records):
        for record in records:
            for fieldname in record.keys():
                if "." in fieldname and len(record[fieldname]) > 0:
                         newname = fieldname.lower().split('.')[-1]
                         record[newname] = record[fieldname]
                         record[fieldname] = None
            yield record

However, setting record[newname] only seems to work half the time. So when I run the search, I don't see all of the newname fields appearing in the events list. Am I doing something wrong?

0 Karma
1 Solution

j8lp
Explorer

Figured it out. Looks like Splunk only writes the fields that are set in the first record. So removing the len(record[fieldname]) > 0 fixed the issue:

 def stream(self, records):
     for record in records:
         for fieldname in record.keys():
             if "." in fieldname:
                      newname = fieldname.lower().split('.')[-1]
                      record[newname] = record[fieldname]
                      record[fieldname] = None
         yield record

View solution in original post

0 Karma

j8lp
Explorer

Figured it out. Looks like Splunk only writes the fields that are set in the first record. So removing the len(record[fieldname]) > 0 fixed the issue:

 def stream(self, records):
     for record in records:
         for fieldname in record.keys():
             if "." in fieldname:
                      newname = fieldname.lower().split('.')[-1]
                      record[newname] = record[fieldname]
                      record[fieldname] = None
         yield record

View solution in original post

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!