Hi @richgalloway  Thank you for your prompt answer.  I agree that I don't need to change my indexer cluster. But the documentation only talked about how to set that up and not about the comparison of the two. I just need more help to decide which one to pick. Also, regarding your statement "Also, SmartStore has bearing on cross-cluster searching.  S2 is transparent to the search heads." Could you please elaborate more on this? thanks! ... View more

@richgalloway  I see. So there is no parameter for Smartstore that limit data size locally by index, am I correct? maxGlobalDataSizeMB = includes remote storage. (I only want to evict bucket in my S3 by age, not size) max_cache_size = this is cache manager as a whole, not per index.   PS: It will be perfect for my situation if maxGlobalDataSizeMB does not include remote storage. ... View more

Would you mind helping me navigate where to find this built-in alert? I appreciate you. @richgalloway  ... View more

Thanks @richgalloway  But I read the following for maxGlobalDataSizeMB: * This setting includes the sum of the size of all buckets that reside on remote storage, along with any buckets that have recently rolled from hot to warm on a peer node and are awaiting upload to remote storage.  This parameter includes all warm buckets in local storage AND remote storage. Meaning that if I set this parameter to 200GB, I can only have data for maximum 200GB in BOTH local and remote storage (correct me if I'm wrong) for that index. I'm more interested in the parameter that only includes local storage, so that when that value is reached, the cache manager will begin evicting buckets to remote storage.   ... View more

@isoutamo Thank you or the response, Although I'm not sure if it's about user + password issue simply because the exact same alert is still able to send email when triggered. But only small percentage of those triggered alerts are sent, the rest have that error I mentioned above. I however, use app to put my alerts in and this is the alert_actions.conf file in system/local: [email] hostname = http://aws-prod-east-splunk.megh.thingspace.verizon.com mailserver = top-smtp-proxy.ts-prod.cloud:587 pdf.header_left = none pdf.header_right = none disabled = 0 auth_password = {encrypted} auth_username = AKIAUN3SJVAQRIOJW62G from = myemail@mail.com (whitelisted) use_tls = 1   and this is the alert_actions.conf in each app (I have about 10 app): [email] subject= |prod-us-east-1| SplunkAlert: $name$ $result.cid$     ... View more

@richgalloway Hi Rich, I have mounted a 1000GB EBS volume to my EC2 instance that hosts my standalone Splunk. Our indexing rate is around 10GB/day. However, how do I check the size of my cache manager? My goal is to make it large enough to hold data up to 30 days. Thanks ... View more

"It's a good idea to size your cache to hold at least 30 days of data.  Increase that number if you routinely search data older than 30 days." Hi @richgalloway , what data retention policy responsible for this and how can we configure them so our Cache Manager can hold data until 30 days have passed? Thanks in advance. ... View more

@isoutamo correct, I made changes of splunkd config in host level and not inside the splunk container because file /etc/systemd/system/splunkd.service only exists in the host level and not in the container. The only config that I made inside the container is the file /etc/security/limits.conf below: root soft nofile 99999 root hard nofile 99999 root soft nproc unlimited root hard nproc unlimited * soft nofile 99995 * hard nofile 99995 * soft nproc unlimited * hard nproc unlimited   ... View more

in my /etc/systemd/system/spunkd.service , I have added this stanze [Service] LimitNOFILE=88888 LimitNPROC=16000 LimitDATA=8589934592 and my /etc/security/limits.conf inside the container is below: (splunkd is run by root) root soft nofile 99997 root hard nofile 99998 root soft nproc unlimited root hard nproc unlimited then I ran: systemctl restart splunkdocker . The container then restarted. Then I ran docker exec to access the container. And then I ran:  cat /proc/<splunkd_pid>/limits , and I got these results Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes unlimited unlimited processes Max open files 65536 65536 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 124464 124464 signals Max msgqueue size 819200 819200 bytes  As you can see that "Max open files" did not change and it stayed the default value. However, it works whenever I set the nofile to be less than 65536. I checked my system can handle over 3 milions open files, so 65k cant be the cap. I'm so confused at this point. ... View more

@rabbidroid systemd is used to start the splunk container in my AWS ec2. Could you please elaborate what you meant by "you can add it in the unit file" ? what unit file are we talking about here? ... View more

@richgalloway yes I do, we have events coming in (assume index=fortinet). However, I do not know how to write specific search query that can capture an event when the VPN is down. Also if I simply search "index = fortinet", how do you narrow the search to find events that shows a down VPN or tunnel? I could not find those events. ... View more

I go to Dashboards -> Pick a Dashboard -> Edit -> Schedule PDF Delivery ... View more