@ITWhisperer that's true. But for me, it will only make sense if hot and warm buckets reside in separate disks Please take a look at this         Hot: Used for high read/write operations. For this we need our best CPU/RAM nodes here, and we use SSD storage. Warm: Lighter search, read only. We can have less powerful nodes here. Warm nodes can use very large spindle drives instead of SSD storage.         the above statement refers to another software, but just like Splunk, it also follows the hot-warm-cold architecture, so I figured it would be a good point of comparison. There, it was stated that hot and warm use different disks, which makes sense to me. On the other hand, Splunk hot and warm buckets share the same directory in the same disk, so I don't understand how is that exactly gonna save us cost (if cost management is part of the reason why there is warm bucket). That brings me back to my original question: what's the point of having warm bucket when we already have the hot bucket which is also searchable and, most importantly, resides in the same directory/disk. Maybe I'm missing something here but that's what I'm hoping to find out by posting this question. ... View more

Maybe I should rephrase my question to this: Why can't hot bucket roll straight to cold bucket? I get that hot bucket is actively getting written to which is why I said in my post that that's what I'm thinking is why there has to be warm buckets in the first place, but all I've been told so far is that hot bucket is actively being updated and warm bucket is not which, I'm afraid, doesn't exactly answer the above question. ... View more

Thanks @isoutamo. This is very insightful. ... View more

I see. So it's really just about data separation. I'm wondering though since you said this: "Warm buckets store recent data on fast, expensive storage to ensure quick access for critical searches, optimizing performance for frequently accessed information." the same thing can be said for hot buckets too, right? I mean after all, hot and warm buckets share the same directory. I don't know. Maybe I just haven't quite fully grasped yet why there can't be warm buckets for other reasons beyond data separation. ... View more

Thanks for the response. Sadly though, as comprehensive as it may be, it still doesn't quite answer my question as to why there has to be warm buckets when you could just roll the hot bucket straight to cold bucket. My guess is that it has to do with the hot bucket getting actively written to. I think because of that, Splunk can't move the hot bucket straight to another disk where the cold buckets reside, otherwise, the space taken up by the hot bucket can't be reclaimed, or something like that. I could be wrong but I just need someone to confirm it for me. Other than that, I think your answer can be quite helpful too. ... View more

Hi @marcoscala sorry for the late response. I only saw your comment just now. Here's how we did it: Before anything else, make sure that the connection between your Splunk forwarder and SFMC is established and nothing is blocking it. This is were we had our problem initially. Set up HEC on your Splunk forwarder. Make sure to set the allowQueryStringAuth setting to "true". This will make your HEC act as a webhook. This is important because SMFC only allows you to input endpoint URL and nothing else. Register your callback URL in SFMC using the HEC endpoint URL and token from step 2. Your callback URL should look something like this: https://<Your HEC endpoint URL here>:8088/services/collector/event?token=<your HEC token here>​ If successful, this will return a callbackid and verification key to be used for the next step. Manually verify the callback created from step 3. Now I'm not sure if it matters where you do it but just to be sure, execute the command on the server which is running your Splunk forwarder instance. Create your ENS in SMFC. Granted that everything went well, you should now see the events coming in. I suggest temporarily removing all the filters from your ENS until you've confirmed that you're indeed receiving data from it. ... View more

Ok so I suppose HEC is out of the question then? Is there an alternative solution? ... View more

You were right. It was my config after all. It had this line before sendCookedData = false and apparently that's what was giving the "connection closed" message. I omitted that and now I'm no longer getting that message. Thanks for the help! ... View more

Here's the version of RHEL that we currently have: NAME="Red Hat Enterprise Linux Server" VERSION="7.7 (Maipo)" ... View more