Case file · NAS & RAID · EDR-2025-2642
RAID 5 With Two Members Down.
From a manufacturing company located near Yeovil came a six-bay shelf that failed in stages: one bay had been amber for a month
, at which point another bay dropped out during the backup on Friday. The man responsible for their equipment put a cold spare in and set it rebuilding
; however, by morning it had frozen part way
, and half of the damage was done.
Sound familiar? Talk to us.
0800 6890668
In plain English.
RAID 5's maths does not give you much wiggle room: parity absorbs exactly one loss, so two failures leave the controller with equations it can't solve. Telling it to rebuild when there are too few good data stripes to solve those equations means writing new parity over the top of the stripes needed for recovery in the exact same state as they were at the time of the crash. The other danger was the physical shelf that contained the drives. Enterprise-level drives often have unusual sector formats and/or some form of vendor-specific metadata, and desktop hardware either will refuse to read/write these areas or simply mangle them. Also quietly illustrated by this experience was the fact that the amber light had indicated something wrong with this drive for a month. During all of that time, the parity had absorbed the failing disk.
The tools this job took.
See how your case runs →| System | The part it played | Why we rate it |
|---|---|---|
| PC-3000 SAS/SCSI | Diagnosed and read the six enterprise members across their own native interface | Because SAS and SCSI hardware from server rooms is beyond ordinary desktop ports |
| Atola TaskForce 2 | Cloned members in parallel rather than one at a time, saving days | Clones multiple array members at once, shrinking a week's imaging into days |
| UFS Explorer RAID Recovery | Recovered the geometry and mounted the rebuilt volume from the images | Reads NAS volume managers as the layered systems they are, not one flat array |
On the bench.
Six disks, six images — the living included
Nothing ever made it back onto the shelf again. All six member disks came out, and once they had been labelled with their bay number they proceeded to the imagers (the casualties were assessed before being taken to the imagers) in order of survivorship. A critical aspect of getting the next steps correct was to get the SAS interface and sector format for the enclosure correct; if you mess up these first steps, everything that comes next is poisoned.
Coax the two casualties into giving up their surfaces
One of the casualties was struggling due to its weak head. The second one had a large and growing number of "bad" sectors. Neither was even close to being blank. With careful handling of the two disks, we got a large majority of the surface area from each disk into an image, so with both on file the rebuild could use whichever version of a disputed block read most cleanly, rather than just relying on the parity information.
Let the on-disk metadata dictate the rebuild
Assembly was done virtually, across the six images, with the member order, stripe width, rotation and delay all coming from data in the array itself: no default settings or assumptions were made. The filesystem walk was done over this virtual assembly, so the original disks remained untouched.
How it ended.
Once verification of the mounted volume took place, it went home on the firm's "cold spares", now demoted to plain delivery disks (the one role that suited them). The asterisk is there because honesty should always include it: where the previous night's rebuild operation wrote new stripes, part of a single project directory was essentially destroyed and beyond recovery.
More like this in the casebook.
More from RAID & NAS.
Sound like your drive?
All cases here point in the same direction: first, power off the equipment; second, take the free diagnostic test; third, decide what course of action to pursue.