I tried to write a search command to track object position, and compare the current position from last position, the date are like:
Obj, x, y
A, 1.1, 2.2
B, 1.1, 2.0
A, 1.2, 2.0
B, 1.3, 2.0
...
So each record of obj 'A' need to compare with the last record of 'A', except for the first record.
However, I found that Splunk push the search result in bunches to the search command, and I will have gaps in between records.
For example when a plain search return 3,148 events, I wrote a pipe search command and find the events arrived in bunches of 50, 449, 2500, 149, 0, 0, 0, the code of the pipe search command is listed below.
import sys
from splunklib.searchcommands import \
dispatch, StreamingCommand, Configuration, Option, validators
@Configuration()
class CountMatchesCommand(StreamingCommand):
def stream(self, records):
i = 0
for record in records:
i += 1
yield record
self.logger.error('CountMatchesCommand error: %d' % i)
dispatch(CountMatchesCommand, sys.argv, sys.stdin, sys.stdout, __name__)
Can anyone advise what is the best solution to handle the 'bunch arrival' problem so I can reliably compare position with last one?
... View more
