topic Re: post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild in Installation

post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild

FatDragon1 — Mon, 28 Sep 2020 11:58:45 GMT

After upgrade, I am seeing lots of our indexes have processtracker errors.

i.e.
06-23-2012 14:28:10.391 -0400 ERROR ProcessTracker - (child_13__Fsck) BucketBuilder - The rawdata version in directory "/mnt/splunk-data/gw/db/db_1298332798_1297901011_0/rawdata" does not support rebuilding

and

Unable to read raw size

file="/mnt/splunk-data/icore/db/db_1326927068_1299178043_28/.rawSize": No such file or directory

I found a link ( http://splunk-base.splunk.com/answers/47462/post-43-upgrade-unable-to-read-raw-size ) for rawsize issue and it didn't help!

Re: post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild

Drainy — Sun, 24 Jun 2012 09:46:18 GMT

Also, have you seen, http://splunk-base.splunk.com/answers/46839/splunkd-down-why to see if that perhaps is linked? (system running out of file descriptors).

Actually, another worthy question is what version did you upgrade from and to? If it was 4.1.x to 4.3.x then it is recommended to do an intermediary upgrade from 4.1.x to 4.2.x and then on to 4.3.x.

EDIT: Oh and of course, if you have a support contract it is always worth getting in touch with them for a quick resolution as they may have come across this many times before. Always best to be safe rather than sorry if it is critical data.

Re: post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild

erga00 — Sun, 24 Jun 2012 20:34:03 GMT

I have the same issue. Following the instructions in ( http://splunk-base.splunk.com/answers/47462/post-43-upgrade-unable-to-read-raw-size ) didn't help me either.

I'm on Windows 2008 R2 and upgraded from 4.1.6 to 4.3.2 (x64).

Re: post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild

Drainy — Sun, 24 Jun 2012 20:47:31 GMT

Right, if memory serves me right you may need to do a 4.2 upgrade in the middle. So get back to a 4.1 install and then upgrade to 4.2 and from there upgrade to 4.3. I appreciate that the release notes suggest it should be fine to upgrade directly, there have been cases where support recommend a stepped upgrade

Re: post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild

FatDragon1 — Mon, 25 Jun 2012 03:08:59 GMT

I upgraded from 4.1.6 to 4.3.2.

Yes, Pretty much I have been all over the splunk base posts and splunk docs/wiki.

I installed splunk on splunk (sos) and reviewed most of the structures of the splunk operation.

disabled SplunkDeploymentMonitor 4.3.2 until I find time later to check on it's behavior.
disabled SplunkDeploymentMonitor_4.1.x 1.0 .

Monitored the splunkd log and other splunk logs and restarted splunk many times after every tuning changes.

some of the changs:
I increased FD to 100,000 for soft and hard for user executing splunkd and also tweaked the limits.conf in splunk sub-directories for processes per cpu and percentage of searches and then cleaned up dispatch directory of 119 stale saved searches and reports/schedules and then few other smaller tweaks here and there..

I had lots of patience. I don't like to see errors /warnings in any system/application and etc logs. INFO/notices entries are okay.

Seems like all my splunk logs are now clean of errors/warnings. We'll monitor more during busy splunk usage and update here with more information.

Re: post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild

FatDragon1 — Mon, 25 Jun 2012 03:15:53 GMT

oh yea..I also attempted to clean several of the buckets for the manifest/rawsize errors.

I think when splunk restarted it checks all the buckets and savedsearches and schedule reports and dispatches leftover and you'll get lots of errors and warnings as it tries to clean itself off and catch up with cold/hot/frozen buckets and update the rawdata and manifested rawsize and etc.

Re: post 4.3.2 upgrade, the rawdata version in directory does not support rebuilding and rawsize issue still exists when forcing rebuild

FatDragon1 — Mon, 25 Jun 2012 16:54:09 GMT

More information also on troubleshooting your buckets at bottom of the page in http://docs.splunk.com/Documentation/Splunk/4.3.2/admin/HowSplunkstoresindexes