I had similar issue, the difference is that my TA has inputs for multiple tables. All the inputs work ok with 3000 record count except one of them. Once I adjusted the problematics input to  1000 record count, events started to be ingested.  why only experiencing issues with one of them.  Error reported for this input is: The reason for failure= {'message': 'Transaction cancelled: maximum execution time exceeded', 'detail': 'maximum execution time exceeded Check logs for error trace or enable glide.rest.debug property to verify REST request processing'}.   any suggestions will be appreciated.    ... View more

Short answer is - no. Long answer is that if you read a file-based input you can either read lines that are appended to a file if the file appears to be in a known state or re-read the complete file if splunk decides that its contents changed so it's a new file. Remember that file based inputs are meant for log-type files which are appended to during their life cycle. So you can't "diff" a file using splunk's built-in mechanisms. Also remember that after you read data on a forwarder and push it to indexer, the input on the forwarder has no knowledge of what happened to the events, which events have been filtered out, modified during indexing, which are duplicated vs. what is already in the index (even the indexer doesn't know that - if it gets the same event twice it will happily index it twice; there is no deduplication on indexers). So in order to be able to ingest only changes from somewhere in the middle of the file, your input would have to store whole contents of the file from the "previous known state" to be able to perform such analysis and find differences. There are of course further issues of "what if the same line was repeatedly added and removed" and so on. So your task may seem like an easy feat but is not. You could, however, monitor the file and reingest the file as a whole to a temporary index each time someone updates it and then do a scheduled search to find only non-existing entries and collect the results to a destination index. Ugly - yes, will incur additional license usage - yes (but with typical csv sizes that might not matter much). ... View more

Create your cron expression using Crontab guru. There are loads of examples already present to help you understand how it works. Or if you can just tell us as to how frequent do you want the input to run, we can help you make a cron expression. https://crontab.guru/ ++If this helps, please consider accepting as an answer++ ... View more