Hi all,
I couldn't find any definitive answers, so I'm hoping that the forum members' real life experiences may point me in the right direction.
We have lots of logged events (over 300 million per day) from two, distinct sources for the same application. Even though the event message format is different between the two types of source data, there is one major commonality - the raw events are completely unstructured!
When we first got Splunk, we only had one data source which made it simple to set up an Index and a Custom Sourcetype.
When we started ingesting the second source, there was a huge pressure to get the data in and analysed which left me little time to appraise the actual raw data. In my rush ( and, perhaps, naivety!!) I created a second index and a second sourcetype and, as a result of this, many of our searches start with index=abc OR index=xyz etc!
My question is, given the volume of data and the similarity in that it's all unstructured data, would it be best, from an indexing and searching perspective, to move towards one of:
1 index and 2 sourcetypes
2 indexes and 1 sourcetype
1 index and 1 sourcetype
Or stick with our current
2 indexes and 2 sourcetypes
?
Thanks for any ideas,
Mark.
... View more