Specialist work · RAID 5 arrays
A RAID 5 will continue operation even when one unit has failed, this being all it promises. Most Exeter configurations arrive at the test bench having lost that right; a first attempt at rebuilding has been made, and a second disk has fallen off mid-way. At least some small comfort can be derived from the fact that "failed" seldom means completely dead, and the remaining disks are backed up by sector-by-sector copies, so none of your original data will ever be written over.
Talk it over with an engineer
0800 6890668
| What's wrong | What it usually means | Step one |
|---|---|---|
| You lost one of your disks and now your array is running in degraded mode | Your array has no room left for failure; each hour it remains online puts you at risk | Turn off the system before the second drive fails |
| The rebuild was initiated but crashed partway through | Full surface scan stopped by an error reading from a bad sector on a second, worn-out disk: the classic end | Stop now; do not attempt again |
| Two disks shown as failed at once | Far more often you will have one true dead body (drive) and one with some "bad sector" issues, as opposed to two complete failures | Get a quote first from us before posting both drives; imaging will give the verdict |
| The drives were put into "force online" in an attempt to revive the set | Old stale copies are now living on new stripes | Take no further action |
| Swapped the controller and the array was lost | A metadata argument; the data is untouched underneath | Reconstruction is routine for us |
| Recovery software has been released on the raw disks | The originals were written to | Identify what ran, get a quote first, then post the disks, labelled by bay |
Your device will be logged under its own case number on the day it comes into our lab. The engineer who works on your device identifies the problem with it, clearly states what we think has a good chance of coming back and provides you with one fixed quote. There are no charges for this assessment; there is no obligation for you to do anything; and paid work does not begin until you authorise us to start.
Our imaging process copies all data linearly across each drive onto our storage, as well as the two failed ones that the controller marked for removal. Areas of instability are identified and mapped; the weakest points will be addressed with the final passes, and your original data remains protected from potential harm.
The order of disks, the size of stripes and how the parities are distributed across them can be found by looking at two witnesses: the metadata that controllers write to their disks (and where), and the patterns that exist within the data itself. There is no need for a "working card", and nothing is guessed.
We reconstruct the full volume using all of the image sets, placing the blocks of data from the image set which provided the cleanest reads, and filling in from parity wherever none of the members were able to provide a clean read, even in areas where only the "failed" drives still carry data. We then repair the file system on top and verify it.
You have access to a complete list of every item recovered from your device, which you approve prior to any fees being applied. The files will be returned via new media (brand new) with the return postage paid by us, and the case remains open for as long as it takes to confirm that all the files open properly on your own machine.
Run the sums once, then decide calmly: the consumer drive is spec'd for an unrecoverable read error (URE) after reading 1014 bits, i.e. one "bad" sector every 12.5TB or so. Enterprise drives are rated at 1015. So rebuilding a four-disk RAID using 16TB disks would require reading roughly 48TB from the three remaining disks; therefore, on paper, there's an actual risk of exceeding the specification. However, in practice most drives will be able to exceed that by quite a bit. Both statements can exist simultaneously, and together they provide our repeated rule: get the images first, then rebuild without worry.
Disk two was dead since June but it wasn't noticed till August, when disk four crashed while in the middle of a rebuild. Under an image check for disk four, it was found to have nineteen bad blocks instead of being completely lost. The three good disks, along with the nearly complete fourth, were able to rebuild the striped set, allowing every filing to go in on time.
Yes, in most cases. The "failed" state from a controller often equates to just some bad sectors on that one drive and not an overall failure of the drive. Because we rebuild from all our images, we can take each sector from the readable image that works the best. We don't typically have a pair completely dead; usually it's just a pair being removed from the array.
A rebuild can be done only after all members have been imaged. In a rebuild, each sector of each remaining disk must be read to recreate the data (exactly what will finish off a marginal drive), while new parity is written over the data needed for a reconstruction. If the images are created first, this same rebuild is entirely safe.
A URE represents a condition in which the damage to a sector on a disk is so severe that even the error correcting mechanism of the disk itself cannot recover the information. The numbers provided as failure rates are based on worst case conditions and reflect approximately one URE per 12.5TB of data read for retail (consumer) hard drives, and approximately one URE per 125TB of data read for enterprise class storage. Respect it, but there is no reason to panic when a URE occurs: these figures are worst case floor estimates, and modern RAID controllers sacrifice only one stripe of data at the time of occurrence, rather than the whole array.
No. The controller metadata on the member disks records the geometry, and even if it has become corrupted, the geometry can still be inferred from the data itself. All of our rebuilds here are done using images that are made at the bench. Never will they be attempted with your original disks.
Every time a failing hardware component is powered back up, there is additional lost data. First open a case with us: the inspection (look over) is always at no cost, whichever way it goes.