I’m running Splunk in a Linux Red Hat environment and trying to collect logs generated by the auditd service.  I could simply put a monitor on "/var/log/audit/audit.log", but the lines of that file aren’t organized such that the records from a specific event are together, so I would end up having to correlate them on the back-end.  I’d rather use ausearch in a scripted input to correlate the records on the front end and provide clear delimiters (----) separating events before reporting them to the central server. Obviously, I don’t want the entirety of all logged events being reported each time the script is run.  I just want the script to report any new event since the last run.  I found that the "--checkpoint" parameter ought to be useful for that purpose. Here’s the script that I’m using: path_to_checkpoint=$( realpath "$( dirname "$0" )/../metadata/checkpoint" ) path_to_temp_file=$( realpath "$( dirname "$0" )/../metadata/temp" ) /usr/sbin/ausearch -–input-logs --checkpoint $path_to_checkpoint > $path_to_temp_file 2>&1 output_code="$?" chmod 777 $path_to_checkpoint if [ "$output_code" -eq "0" ]; then         cat $path_to_temp_file fi echo "" >> $path_to_temp_file date >> $path_to_temp_file echo "" >> $path_to_temp_file echo $output_code >> $path_to_temp_file It works just fine in the first round, when the checkpoint doesn’t exist yet and is generated for the first time, but in the second and all subsequent rounds, I get error code 10: invalid checkpoint data found in checkpoint file. It works fine in all rounds when I run the bash script manually from command line, so there isn’t any kind of a syntax error, and I’m not using the parameters incorrectly. Based on the fact that the first round runs without error, I know that there isn’t any kind of permissions issue with running “ausearch”. It works fine in all rounds when I run the bash script as a cronjob using crontab, so the fact that Scripted Inputs run like a scheduled service isn’t the root of the problem either. I’ve confirmed that the misbehavior is occurring in the interpretation of the checkpoint (rather than the generation of the checkpoint) by doing the following. Trial 1: First round: bash script executed manually in CMD to generate the first checkpoint Second round: bash script executed manually in CMD, interpreting the old checkpoint and generating a new checkpoint Result: Code 0, no errors Trial 2: First round: bash script executed manually in CMD to generate the first checkpoint Second round: bash script executed by Splunk Forwarder as a Scripted Input, interpreting the old checkpoint and generating a new checkpoint Result: Code 10, "invalid checkpoint data found in checkpoint file" Trial 3: First round: bash script executed by Splunk Forwarder as a Scripted Input to generate the first checkpoint Second round: bash script executed manually in CMD, interpreting the old checkpoint and generating a new checkpoint Result: Code 0, no errors Trial 4: First round: bash script executed by Splunk Forwarder as a Scripted Input to generate the first checkpoint Second round: bash script executed by Splunk Forwarder as a Scripted Input, interpreting the old checkpoint and generating a new checkpoint Result: Code 10, "invalid checkpoint data found in checkpoint file" Inference: The error only occurs when the Splunk Forwarder Scripted Input is interpreting the checkpoint regardless of how the checkpoint was generated, therefore the interpretation is where the misbehavior is taking place. I’m aware that I can include the "--start checkpoint" parameter to avoid this error by causing "ausearch" to start from the timestamp in the checkpoint file rather than look for a specific record to start from.  I’d like to avoid using that option though, because it causes the script to send duplicate records.  Any records that occurred at the timestamp recorded in the checkpoint are reported when that checkpoint was generated and also in the following execution of "ausearch".  If no events are logged by auditd between executions of "ausearch", then the same events may be reported several times until a new event does get logged. I tried adding the "-i" parameter to the command hoping that it would help interpret the checkpoint file, but it didn't make any difference. For reference, here's the format of the checkpoint file that is generated: dev=0xFD00 inode=1533366 output=<hostname> 1754410692.161:65665 0x46B I'm starting to wonder if it might be a line termination issue.  Like if the Splunk Universal Forwarder is expecting each line to terminate with a CRLF the way that it would in Windows, but instead it's seeing that the lines all end in LF because it's Linux.  I can't imagine why that would be the case since the version of Splunk Universal Forwarder that I have installed is meant for Linux, but that's the only thing that comes to mind. I'm using version 9.4.1 of the Splunk Universal Forwarder.  The Forwarder is acting as a deployment-client that installs and runs apps issued to it by a separate deployment-server that runs Splunk Enterprise version 9.1.8. Any thoughts on what it is about Splunk Universal Forwarder Scripted Inputs that might be preventing ausearch from interpreting its own checkpoint files? ... View more