Hi , I looked the daily ingestion for an index i am seeing total data ingested in last 7 days to an index is 800 GB. When i am calculating the total raw data size its showing total raw data ingested 1626 GB and its compressed to 759 GB which is at 46%. I am not understanding if i ingested 800 GB in last 7 days how come the raw total size data came to 1626 GB ? Any inputs will be appreciated. Query using for compression: | dbinspect index=xyz | fields state,id,rawSize,sizeOnDiskMB | stats sum(rawSize) AS rawTotal, sum(sizeOnDiskMB) AS diskTotalinMB |eval diskTotalinGB=(diskTotalinMB/1024) | eval rawTotalinGB=(rawTotal / 1024 / 1024 / 1024) | fields - rawTotal | eval compression=tostring(round(diskTotalinGB / rawTotalinGB * 100, 2)) + "%" | table rawTotalinGB, diskTotalinGB, compression Result: rawTotalinGB diskTotalinGB compression 1626.19525605347 759.39445495605 46.70% Query used to calculate daily ingestion : index=_internal source="license_usage.log" type=Usage idx=xyz| eval yearmonthday=strftime(_time, "%Y-%m-%d") | eval yearmonth=strftime(_time, "%Y-%m-%d") | stats sum(eval(b/1024/1024/1024)) AS volume_b by idx yearmonthday yearmonth | chart sum(volume_b) over yearmonth by idx|addcoltotals. Which gives me total 862 GB ingestion in last 7 days. ... View more

Hi , We have windows event logs we are routing data to two splunk indexer sites one is in california and one is in virginia , currently we are planning to use two settings , The logs which is routing in virginia needs to have ignoreolderthan implemented for 14 days but the one we are getting in california we need ignoreolderthan for 90 days. How i can do it ? Now we are sending logs to two destination same we have outputs setup for both sites and the inpuits are now ; [WinEventLog://Application] disabled = false start_from = oldest current_only = 0 checkpointInterval = 5 index = wineventlog renderXml=false _TCP_ROUTING = primary_indexers_california, indexers_virginia in this above input i want ignoreolderthan 14 days to be implemented for indexers_virginia only and the california one should remain unchanged ? if i add that stanza in this input here it will implementing in both sites. so looking for ways to do it ? ... View more

Hi , I am currently experiencing high memory usage on my indexers when i saw the memory usage , i saw a high amount of memory caches below, total used free shared buffers cached Mem: 516761 477169 39592 0 1158 457892 -/+ buffers/cache: 18117 498644 Swap: 51199 551 50648 How safe is it to clear buffers and cache ? If i clear caches is it will create any issue like deleting data make indexers down ? I am thinking to clear it using these commands : sync; echo 1 > /proc/sys/vm/drop_caches. sync; echo 3 > /proc/sys/vm/drop_caches ... View more

Hi , i am having issues to configure the google cloud connector , i am able to setup successfully the credentials but when i am trying to add the inputs like pub/sub connections i am getting below error , does anyone can help me out ? ERROR AdminManagerExternal - Unexpected error "" from python handler: "timed out". See splunkd.log for more details. ... View more

Hi , Currently i am restoring multiple buckets from frozen directory and getting issues. I copied the identified buckets from frozen db to thawed db. Then i am using this script as mentioned in the answer https://answers.splunk.com/answers/120007/thawing-out-multiple-buckets-at-once.html. My question is where do i need to run this script is in frozen directory or in tmp directory i am not sure where to run this script , when i try to run from frozen db its creating some buckets with end name as tmp not sure what that means ? Is their any script do we have what can tell us if their is any conflicting bucket or not if their it will rename those buckets ? ... View more

When we try to restart the splunk it says its getting permission denied on these two files:/etc/sysconfig/init: Permission denied and etc/rc.d/init.d/functions. Both of these files are owned by root and Splunk is running as user, but I have another server who have same owned by root and Splunk running as user is working fine but don't know why its creating issues on this server? ... View more

My splunk universal forwarder is not reading the logs from var/log directory , even not sending the splunkd logs to my search-head , This seems to me as a permission issue in first place , can any one help me out with the command or how can i tell my forwarder to read the logs from var/log directory ? ... View more