Why am I unable to set new fields in a custom search streaming command?

[j8lp]

I'm writing a custom search command to convert all the full path xml names to just local names. I'm also making the field names all lower case for consistency. My code is below:

    def stream(self, records):
        for record in records:
            for fieldname in record.keys():
                if "." in fieldname and len(record[fieldname]) > 0:
                         newname = fieldname.lower().split('.')[-1]
                         record[newname] = record[fieldname]
                         record[fieldname] = None
            yield record

However, setting record[newname] only seems to work half the time. So when I run the search, I don't see all of the newname fields appearing in the events list. Am I doing something wrong?

[j8lp · Accepted solution]

Figured it out. Looks like Splunk only writes the fields that are set in the first record. So removing the len(record[fieldname]) > 0 fixed the issue:

 def stream(self, records):
     for record in records:
         for fieldname in record.keys():
             if "." in fieldname:
                      newname = fieldname.lower().split('.')[-1]
                      record[newname] = record[fieldname]
                      record[fieldname] = None
         yield record