You can barely run a single instance on 4 cores reliably. Definitely not two. At 10GB/day, unless you need HA, you really shouldn't need more than one instance/server but you absolutely must get more cores.
If cost is a concern, see about reducing some of that memory usage. No way you need 200GB for this small of a deployment and RAM is expensive.
... View more
Quick and dirty example of the average of both over time.
sourcetype=Perfmon* (object="Memory" counter="Committed Bytes") OR (object="LogicalDisk" counter="% Free Space")
| timechart avg(Value) as Value by counter
... View more
You can try using sendCookedData=false as in https://docs.splunk.com/Documentation/Splunk/7.2.3/Forwarding/Forwarddatatothird-partysystemsd#Forward_a_subset_of_data
... View more
Download the app yourself and unpack to etc/apps if you have this much permission. Otherwise, it seems you have splunk admin permissions, download locally and upload via manage apps.
... View more
Because there is but only a single capability to grant access to all secrets stored on the system and it traverses system wide. Splunk really needs to work on their permissions structure.
... View more
This was fixed in 7.0.8 and recent releases of 7.2. To workaround this in earlier versions, either add the indexes to your search head or go grab an authorize_roles.xml from a 6.x version and deploy it to your SHC manually in etc/apps/search/local/data/ui/manager
https://docs.splunk.com/Documentation/Splunk/7.0.8/ReleaseNotes/Fixedissues#Splunk_Web_and_interface_issues
... View more
Some inputs can return the shortname and some return the FQDN. Check to ensure your forwarder's inputs.conf hostname under [default] matches the name in server.conf and that any other inputs do not specify a host explicitly.
... View more
Try something like
index=wineventlog EventCode=521 OR EventCode=4617 | timechart span=1h count by host
or
index=wineventlog EventCode=521 OR EventCode=4617 | timechart span=1h sum(eval(EventCode=521)) as 521 sum(eval(EventCode=4617)) as 4617 by host
... View more
Does something like this help?
index=blah sourcetype=sourcetypeA OR sourcetype=sourcetypeB | stats count(my_field) as count by sourcetype
... View more
_time is actually epoch translated for you in the UI, an easy way to strip the translation is to rename _time to something else or use an alias,
| eval epoch=_time
... View more
It would seem Splunk does not have permissions to write. Confirm the user Splunk runs as has ownership of the installation and that there are no storage issues.
chown -R splunk:splunk /mnt/*
... View more
sourcetype="pfaduit" success NOT SLO NOT OIDC adapterid=* |stats sum(responsetime) as "Total transaction time" by tid | stats avg("Total transaction time") as "Average transaction time"
... View more