I created a saved search and a trigger upon completion to send myself an email that links to the finished job. It works mostly as intended by default, but it directs to my junk folder because the sender is "splunk" with no @domain.com. This would just be a minor inconvenience if the report was just going to go to me, but we are planning on rolling it out to several users and want to fix this. I initally tried going to the report Advanced Settings and changed the "action.email.from" from just "splunk" to "splunk@ourcompany.com".  At that point the emails stopped coming completely. I checked the log at  splunk/var/log/splunk/python.log for errors, but the only ones I have encountered are due to size limits which were resolved by removing the inline table. I have also gone to Server settings » Email settings and changed the "Send emails as" also to add my company domain. It appears the only emails that are coming through are the ones with the specific report "Advanced Settings" "action.email.from" field set to just "splunk" without the domain, but they are still going to my junk folder and are unable to be marked as a safe sender because there is no domain on the address. Could someone assist with troubleshooting this issue and how to add a domain to the sender address correctly? Thanks. ... View more

I created a data input for a local file and a new index, the same way that I had done previously for a data input that works and is searchable on the search head. However, this time I am getting the following error:   Received event for unconfigured/disabled/deleted index=cshosts with source="source::/local/splunkdata/cshosts.json" host="host::csplunk31lxv.corp.int" sourcetype="sourcetype::_json". So far received events from 4 missing index(es).   I checked and insured that the index exists and is enabled in the indexer, as well as in the the indexes.conf entry:   [cshosts] coldPath = $SPLUNK_DB/cshosts/colddb enableDataIntegrityControl = 0 enableTsidxReduction = 0 homePath = $SPLUNK_DB/cshosts/db maxTotalDataSizeMB = 512000 thawedPath = $SPLUNK_DB/cshosts/thaweddb disabled = 0   I have hit a wall and am not sure how to troubleshoot this.  Do new indexes take some time before they can begin forwarding the data to the search head? What else could be wrong here?   Thanks ... View more

I have created a data input to run a wrapper script, which executes a python script, and gather its output. It  was working as expected during my initial tests, but seems to have failed when going for the full desired effect. My test script output 1 json record and worked correctly showing the events in Splunk as expected. When I got that working, I changed the command in the wrapper script to run the desired python script instead of the testing script. On a Friday, I scheduled it through the Splunk data input dashboard for the next day, Saturday at 8 am, using the cron syntax " 0 8 * * 6". When I checked the index this Monday morning, there was no new data, only that which was output by my initial testing script. My first concern was whether or not the script actually ran/executed.  Is there a way I can check to verify this? Or a way to check for data input script errors in general? Another concern of mine was the volume of data, as the test script only output 1 record and the real script should output well over 1 million records, sometimes 10x that amount. I should also add that this script makes a large amount of API calls, and I expect it should take several hours to complete. Are there any limitations in Splunk, or perhaps the server in general, that could have caused failure just due to the huge volume of records? Or perhaps the time it takes the script to finish?   Thank you. ... View more