Are there time look back limitations when exporting data from Splunk Hadoop Connect?

[_gkollias]

Hi!

While using Splunk Hadoop Connect to export data from Splunk to Hadoop, we've observed export functionality halts once we reach 254 days ago. We have 14 months of summary data that we'd like to export, but are unable to go back any further.

Here are some references in the documentation we've been using to test:

http://docs.splunk.com/Documentation/HadoopConnect/1.2.5/DeployHadoopConnect/Configurationfilerefere...

https://docs.splunk.com/Documentation/HadoopConnect/1.2.5/DeployHadoopConnect/ExporttoHDFS#How_data_...

Also, we've also made sure to stay in line with file size limitations. Are there any known limitations to how far back we can go to export data? Any insight is greatly appreciated.

Thanks in advance!

[sloshburch · Accepted solution]

Sounds like either a bug or other symptoms we've not yet seen:

• Bug: Open a Support Case since it's not working as documented
• Error Message: Before support case, make sure there's no error messages in _internal or the related sources from the hadoop connect. It's possible Splunk is dying on something related to the content and we just didn't notice that error.

Lastly, there are some alternatives to this functionality depending on the purpose of using it:

• Longer retention: Then check out TSIDX Reduction or Hadoop Data Roll
• Data Science work within Hadoop: Splunk Machine Learning Toolkit or Hadoop Data Roll

(I thought there was more, but now my mind is blank, so we'll start with this).