https://community.splunk.com/t5/Splunk-Enterprise-Security/AWS-cloudtrail-logs-and-vpc-flow-logs-not-being-ingested/m-p/387555#M4102 <P>Hi All, </P> <P>For the Cloudtrail logs, this is the last logs in splunkd logfile. </P> <P>05-22-2019 08:15:02.624 +0000 INFO IndexWriter - idx=aws-cloudtrail, Initializing, params='[300,period=300.000,frozenTimePeriodInSecs=7776000.000,coldToFrozenScript=,coldToFrozenDir=,warmToColdScript=,maxHotBucketSize=786432000,optimizeEvery=5.000,syncMeta=true,maxTotalDataSizeMB=1073741,maxGlobalDataSizeMB=0,maxMemoryAllocationPerHotSliceMB=5,addressCompressBits=5,isReadOnly=false,maxMergizzles=6,maxHotSpanSecs=7776000.000,maxMetadataEntries=1000000,maxHotIdleSecs=0.000,maxHotBuckets=3,minHotIdleSecsBeforeForceRoll=0.000,quarantinePastSecs=77760000.000,quarantineFutureSecs=2592000.000,maxSliceSize=131072,serviceMetaPeriod=25.000,partialServiceMetaPeriod=0.000,throttleCheckPeriod=15.000,homePath_maxDataSizeBytes=0,coldPath_maxDataSizeBytes=0,compressionType=gzip,lz4BlockSize=65536,compressionLevel=-1,fsyncInterval=18446744073709551.615,maxBloomBackfillBucketAge_secs=2592000.000,enableOnlineBucketRepair=true,enableDataIntegrityControl=false,maxUnreplicatedMsecWithAcks=60000,maxUnreplacatedMsecNoAcks=300000,alwaysBloomBackfill=false,minStreamGroupQueueSize=2000,streamingTargetTsidxSyncPeriodMsec=5000,repFactor=4294967295,hotBucketTimeRefreshInterval=10,enableTsidxReduction=1,suspendHotRollByDeleteQuery0,tsidxReductionCheckPeriodInSec=600.000,timePeriodInSecBeforeTsidxReduction=5184000.000,remoteVolume=,remotePath=,splitByIndexKeys=,dataType=event,serviceInactiveIndexesPeriod=60]' isSlave=false<BR /> 05-22-2019 08:15:02.624 +0000 INFO IndexWriter - openDatabases complete currentId=-1 idx=aws-cloudtrail</P> <P>We are not able to search for the logs on ES. we have one HF.<BR /> We have similar log for vpc flow logs. Could someone help in clarifying as to why the data ingestion isn't working and how to fix this so that the logs become searchable</P> Wed, 30 Sep 2020 00:40:41 GMT singhvishakha29 2020-09-30T00:40:41Z https://community.splunk.com/t5/Splunk-Enterprise-Security/AWS-cloudtrail-logs-and-vpc-flow-logs-not-being-ingested/m-p/387555#M4102 <P>Hi All, </P> <P>For the Cloudtrail logs, this is the last logs in splunkd logfile. </P> <P>05-22-2019 08:15:02.624 +0000 INFO IndexWriter - idx=aws-cloudtrail, Initializing, params='[300,period=300.000,frozenTimePeriodInSecs=7776000.000,coldToFrozenScript=,coldToFrozenDir=,warmToColdScript=,maxHotBucketSize=786432000,optimizeEvery=5.000,syncMeta=true,maxTotalDataSizeMB=1073741,maxGlobalDataSizeMB=0,maxMemoryAllocationPerHotSliceMB=5,addressCompressBits=5,isReadOnly=false,maxMergizzles=6,maxHotSpanSecs=7776000.000,maxMetadataEntries=1000000,maxHotIdleSecs=0.000,maxHotBuckets=3,minHotIdleSecsBeforeForceRoll=0.000,quarantinePastSecs=77760000.000,quarantineFutureSecs=2592000.000,maxSliceSize=131072,serviceMetaPeriod=25.000,partialServiceMetaPeriod=0.000,throttleCheckPeriod=15.000,homePath_maxDataSizeBytes=0,coldPath_maxDataSizeBytes=0,compressionType=gzip,lz4BlockSize=65536,compressionLevel=-1,fsyncInterval=18446744073709551.615,maxBloomBackfillBucketAge_secs=2592000.000,enableOnlineBucketRepair=true,enableDataIntegrityControl=false,maxUnreplicatedMsecWithAcks=60000,maxUnreplacatedMsecNoAcks=300000,alwaysBloomBackfill=false,minStreamGroupQueueSize=2000,streamingTargetTsidxSyncPeriodMsec=5000,repFactor=4294967295,hotBucketTimeRefreshInterval=10,enableTsidxReduction=1,suspendHotRollByDeleteQuery0,tsidxReductionCheckPeriodInSec=600.000,timePeriodInSecBeforeTsidxReduction=5184000.000,remoteVolume=,remotePath=,splitByIndexKeys=,dataType=event,serviceInactiveIndexesPeriod=60]' isSlave=false<BR /> 05-22-2019 08:15:02.624 +0000 INFO IndexWriter - openDatabases complete currentId=-1 idx=aws-cloudtrail</P> <P>We are not able to search for the logs on ES. we have one HF.<BR /> We have similar log for vpc flow logs. Could someone help in clarifying as to why the data ingestion isn't working and how to fix this so that the logs become searchable</P> Wed, 30 Sep 2020 00:40:41 GMT https://community.splunk.com/t5/Splunk-Enterprise-Security/AWS-cloudtrail-logs-and-vpc-flow-logs-not-being-ingested/m-p/387555#M4102 singhvishakha29 2020-09-30T00:40:41Z