RAID 5 or RAID 6 Degraded Array With Second Drive Failing During Rebuild (Why Never Run CHKDSK or fsck, ddrescue Sector Cloning, and mdadm Read-Only Assembly)¶
Os números de emerxencia varían segundo o país. Esta páxina usa 911, o número dos Estados Unidos e Canadá. Moitos outros países usan 112, 999 ou outro número. Busca o número do teu país.
Replacing a failed hard drive in a degraded RAID 5 ou RAID 6 storage server, Synology DiskStation, QNAP NAS, Linux mdadm array, or Dell PERC / HP Smart Array / LSI MegaRAID hardware controller only to watch a second remaining member drive throw UNC (Uncorrectable Read Error) bad sectors, Pending Sectors, or drop offline at 42% of the parity rebuild is one of the most dangerous storage emergencies in enterprise and home-lab computing. In a single-parity RAID 5 array (N - 1 fault tolerance), the mathematical XOR reconstruction of the replacement disk requires reading every single sector (LBA 0 to the final LBA) across 100 percent of the remaining N - 1 member drives simultaneously. When modern high-capacity 8 TB a 24 TB helium drives (purchased from the same manufacturing batch and subjected to identical thermal and vibration hours) undergo a multi-terabyte sequential rebuild read, a single latent media defect (URE) or weak read head on a second drive causes the RAID controller to abort the rebuild, kick the second drive out as Faulty / Crashed, and mark the entire storage pool Offline / Crashed / Superblock Event Count Mismatch.
Do not panic, and do not force a second rebuild or run filesystem repair utilities (fsck, e2fsck, xfs_repair, or Windows CHKDSK). Even when a second drive in RAID 5 (or third drive in RAID 6) develops 50 a 5,000 bad sectors and gets kicked out of the array, 99.9999 percent of the sectors on that "failed" second drive are still 100 percent readable. By immediately halting the array, cloning every unstable member disk sector-by-sector onto healthy spare drives using GNU ddrescue, and assembling the cloned images in Read-Only / --assume-clean overlay mode (mdadm + dmsetup Copy-on-Write loop overlays), you can mount the filesystem safely and copy out virtually every file intact. For NAS appliance specifics and hardware controller failures, read our companion guides on RAID 5 or RAID 6 Synology, QNAP, or TrueNAS multiple drive drop, RAID controller card death and foreign configuration import, e NAS RAID array degraded and ZFS/Btrfs recovery. If your server failure impacts life-safety systems or medical infrastructure, contact your local emergency number (911 nos Estados Unidos e Canadá, 112 en toda a Unión Europea, 999 no Reino Unido) ou consulte o noso directorio de números de emerxencia.
Fai isto agora¶
- Abort any active parity rebuild or scrub immediately and power down the array cleanly (or disconnect SATA/SAS backplane power if a drive is making mechanical clicking or scraping noises):
- Why letting a stalling rebuild continue destroys the array: When a member drive hits a bad sector patch (
LBAread timeout) or has a degrading read/write head (media scratch), leaving the RAID controller retrying sectors for hours grinds the microscopic magnetic platter coating into dust across the entire track cylinder, turning a recoverable20-sectordefect into a catastrophic full-platter head crash. - NEVER run
CHKDSK /f,fsck -y,e2fsck,btrfs check --repair, ouxfs_repairon a degraded, out-of-sync, or half-rebuilt RAID array: - Por que
CHKDSKefsckcause 100% permanent data destruction on out-of-sync RAID: If Drive 2 dropped out on Tuesday and Drive 3 dropped out during Thursday's rebuild, themdadmor hardware RAIDEvent Counter(Update Time) nesas dúas unidades difiere en miles de transaccións. Se forzas a montar unha matriz con xeracións de unidades non coincidentes ou franxas que faltan e permitefsckouCHKDSKpara escribir na matriz, o comprobador do sistema de ficheiros ve árbores de inodos rotas causadas por franxas RAID mal aliñadas, elimina millóns de entradas de directorio válidas enlost+found, e sobrescribe boas franxas de datos con paridade danada en todos os discos da matriz. - ** NUNCA prema
"Create Array","Initialize", ou"Clear Configuration"nunha BIOS RAID de hardware e NUNCA se executemdadm --createsen--assume-cleane un Copy-on-Write (COW) superposición de bucles**: - Execución dun estándar
mdadm --create(sen--assume-cleane facendo coincidir o tamaño exacto do fragmento, o deseño e a compensación de datos) comeza inmediatamente a escribir novas franxas de paridade en todos os discos, destruíndo o superbloque do sistema de ficheiros e as árbores B se hai algún parámetro (512Kvs.64Kanaco,left-symmetricvs.left-asymmetric, ou2048svs.262144scompensación de datos) está equivocada incluso por un sector. - Etiqueta cada ranura do disco duro físico (
Bay 0 / Bay 1 / Bay 2 / Bay 3e número de serie) cun marcador permanente ou cinta ANTES de sacar unha única unidade do chasis: - Escriba o número exacto do compartimento e o número de serie na etiqueta superior de cada unidade. Coñecer a orde física da bahía e comprobar
mdadm --examinemetadatos aforra horas de reconstrución de orde de bandas máis tarde. - Clona sector por sector as unidades que fallan en unidades novas
GNU ddrescue(cun ficheiro de mapa) a través dos portos directos SATA/HBA da placa base ANTES de tentar calquera montaxe RAID: - Nunca intente montar ou montar unha matriz RAID usando a propia unidade físicamente moribunda. Conecte a unidade que falla a un SATA/AHCI ou LSI non RAID
IT-ModePorto HBA nunha estación de traballo de rescate Linux, desactive o montaxe automático do SO e executeddrescue -f -n /dev/sdX /dev/sdY rescue_driveX.mappolo que os sectores rápidos saudables evacúanse no primeiro paso antes de tocar zonas do sector defectuoso. - Ensambla as unidades clonadas
Read-Onlymodo (mdadm --assemble --readonlyou usandodmsetup/losetupRAM/superposición de ficheiros): - Colocando un Copy-on-Write desbotable (
COW) superpoñer o ficheiro encima de cada disco membro clonado, podes probar e montar a matriz, reproducir unext4ouXFSdiario, ou incluso probar unha reconstrución virtual a vez que se garante que se modifiquen cero bytes nos teus clons de disco subxacentes.
Danger
A trampa "Forza de impulsión fóra de orde" (Stale Drive Overwriting Fresh Drives): Supoña que tes unha unidade 4 RAID 5 (Disk 1, Disk 2, Disk 3, Disk 4). Disk 3 fallou hai 6 meses e nunca o substituíches (funcionamento degradado durante 6 meses). Hoxe, Disk 2 desenvolve 100 sectores malos e abandona. Nunca forzas ao neno de 6 meses Disk 3 de novo en liña para reconstruír Disk 2. Se usas Disk 1 + Disk 3 (6 months stale) + Disk 4 para reconstruír ou montar a matriz, todos os ficheiros creados ou modificados nos últimos 6 meses corromperanse en lixo binario ilexible. Debes clonar Disk 2 (a unidade que fallou hoxe) con ddrescue e montar Disk 1 + Cloned Disk 2 + Disk 4.
Árbore de decisión: rescatar unha matriz RAID 5 ou RAID 6 cando falla unha segunda unidade durante a reconstrución¶
Siga este fluxo de traballo de recuperación de almacenamento forense para recuperar a súa matriz sen perder unha soa franxa superviviente:
flowchart TD
Start["RAID 5 (2ª unidade) ou RAID 6 (3ª unidade) falla/cae sen conexión durante a reconstrución"] --> StopNow["PASO 1: Abortar a reconstrución inmediatamente e apagar a matriz (evita a puntuación do plato na unidade que falla)"]
StopNow --> SoundCheck{"A unidade fallida produciu uns clics, uns pitidos ou uns sons fortes?"}
SoundCheck -->|"SI: Choque mecánico de cabeza / Stiction"| CleanRoom["STOP: non encienda esa unidade de novo. Envíe SÓ a unidade que fallou mecánicamente (máis 1 unidade saudable para a referencia de Stripe) a un laboratorio de sala limpa para imaxe nun clon"]
SoundCheck -->|"NON: xira suavemente, fallou debido a sectores defectuosos (UNC / Pendente / Tempo de espera)"| LabelDrives["STEP 2: Label Every Drive With Its Exact Bay Number & Serial Number -> Connect to Linux Workstation via Direct SATA / IT-Mode HBA"]
LabelDrives --> InspectMeta["Executar 'mdadm --examine /dev/sd*' e 'smartctl -a /dev/sd*' en todas as unidades (tempo de actualización de rexistro, contador de eventos, función, tamaño do fragmento, deseño, compensación de datos)"]
InspectMeta --> CloneDrives["STEP 3: Clone Every Degraded / Bad-Sector Drive to a Healthy Spare Drive Using 'ddrescue -f -n' (Fast Pass First, Then Scrape Pass With Mapfile)"]
CloneDrives --> PickFreshest["Identifique as unidades N-1 (RAID 5) ou N-2 (RAID 6) co contador de 'eventos' MÁIS ALTO/MÁIS RECENTE (Nunca mesture unha unidade antiga e obsoleta con unidades novas)"]
PickFreshest --> CowOverlay["PASO 4: Cree superposicións de bucle de só lectura / 'dmsetup' Copy-on-Write (COW) sobre as unidades clonadas para que os clons orixinais non se poidan modificar"]
CowOverlay --> AssembleRO["Execute 'mdadm --assemble --force --readonly' nos dispositivos de superposición -> Montar sistema de ficheiros de só lectura ('mount -o ro,noload') -> Copiar todos os datos críticos no almacenamento externo"]
Fase 1: Comprender por que falla unha segunda unidade durante unha reconstrución RAID (XOR Parity Math & Event Counters)¶
Para entender por que a súa matriz fallou durante a reconstrución e por que os seus datos aínda se poden recuperar, examine como RAID 5 e RAID 6 actually work on disk.
1. How XOR parity reconstruction works and why 1 bad sector aborts an entire rebuild¶
En 4 unidades RAID 5 array (Drive A, Drive B, Drive C, Drive D), os datos divídense en anacos de tamaño fixo (normalmente 64 KB ou 512 KB) striped across the disks, where one chunk in each row holds the XOR Parity (P):
$\(\text{Paridade } P = A \oplus B \oplus C\)$
Se Drive C fails completely and you insert a blank Replacement Drive C_new, the RAID controller must reconstruct every stripe on C_new by calculating:
$\(C_{\text{reconstructed}} = A \oplus B \oplus P\)$
- Que pasa cando Drive B golpea un único ilexible 4 KB sector (UNC erro) en 42% of the rebuild:
- To calculate that single 4 KB block on C_new, the controller must have a valid read from Drive A, Drive B, e Drive D.
- Cando Drive B returns a hardware Read Error (Sense Key 03h / 11h Unrecovered Read Error) despois de 7 segundos (TLER / ERC tempo de espera), os controladores RAID de consumidores e empresas adoitan programarse cun comportamento predeterminado catastrófico: en lugar de saltar ese único 4 KB sector, rexistrando un bloque cero e rematando o outro 99.9999% of the rebuild, the RAID controller marks Drive B como FAILED / OFFLINE and aborts the entire rebuild.
- At that moment, your array has two drives marked "Failed" (Drive C which was only 42% rebuilt, and Drive B which actually has 99.99998% healthy sectors and only 12 sectores malos).
2. Por que Event Counters (Update Time) determina que unidades debes usar para a recuperación¶
Cada vez que a RAID 5 ou RAID 6 array escribe datos ou cambia de estado (como unha unidade que se cae), os metadatos RAID (mdadm superbloque ou hardware DDF metadatos) incrementa un 64 bits Events contador en todas as unidades actualmente activas.
Compare the mdadm --examine saída típica de 4 unidades RAID 5 desastre onde sdb fracasou onte (Events: 10450) e sdc abandonado durante a reconstrución de hoxe (Events: 10892):
| Unidade física | Papel en Array | Update Time |
Events Contador |
Saúde física e o que contén |
|---|---|---|---|---|
/dev/sda (Baía 1) |
Active device 0 |
Today 14:22:10 |
10892 |
100 % saudable e actual: contén todas as escrituras ata o momento exacto en que se estrelou hoxe a reconstrución. |
/dev/sdb (Baía 2) |
spare / faulty (Substituído por disco novo) |
Yesterday 09:15:00 |
10450 (Rancia) |
Fallo onte: faltan 29 horas de escritura da base de datos, OU substituíronse por un disco novo que só 42% reconstruído (Reshape/Rebuild Pos). |
/dev/sdc (Baía 3) |
faulty (Caeu en 42% reconstruír) |
Today 14:22:09 |
10890 (Desactivado en 2) |
99,999 % saudable e actual: só expulsouse 1 segundo antes /dev/sda e /dev/sdd porque alcanzou 8 sectores defectuosos durante a lectura de reconstrución. |
/dev/sdd (Baía 4) |
Active device 3 |
Today 14:22:10 |
10892 |
100 % saudable e actual: contén todas as escrituras ata o momento exacto en que se estrelou hoxe a reconstrución. |
- A regra de ouro da recuperación de RAID da táboa anterior:
- Observe iso
/dev/sda(10892),/dev/sdc(10890), e/dev/sdd(10892) todos teñen practicamente idénticosEventscontadores deToday 14:22. - Unha vez que clonamos
/dev/sdcnunha condución saudable usandoddrescue(recuperando o 99,9999% dos seus sectores e enchendo cero os 8 sectores ilexíbeis), podemos forzar a montaxe/dev/sda+Cloned /dev/sdc+/dev/sdd(ignorando o medio reconstruído/dev/sdbcompletely). That 3-drive combination gives us a 100 percent consistent, up-to-the-second degradedRAID 5array with zero stripe tearing.
Phase 2: Step-by-step sector cloning of the failing drive with GNU ddrescue¶
Before you touch mdadm or any RAID assembly tool, you must clone the drive(s) that threw read errors onto healthy replacement drives of equal or larger byte capacity.
Step 1: Prepare the Linux rescue environment and disable auto-assembly¶
- Boot a Linux workstation (or boot your server from a live USB stick such as SystemRescue
system-rescue.orgor Ubuntu Live Server) with the drives connected to standard motherboard SATA ports or an LSI HBA inIT Mode(Non-RAID pass-through). - Prevent Linux from automatically trying to assemble or mount your broken RAID array on boot:
- Ao iniciar SystemRescue ou Ubuntu Live, comprobe
cat /proc/mdstat. If Linux auto-assembled a partialmddevice in the background, stop it immediately in read-only safe mode before imaging: - Identify every drive by exact model, serial number, and byte size (never guess
/dev/sdXletters, because Linux reassigns drive letters across reboots): Write down which/dev/sdXcorresponds to your Failing Source Drive (for example,/dev/sdc, serialWCC4E12345) and which/dev/sdYis your Brand-New Blank Target Drive (for example,/dev/sdf, serialZGG9988776).
Step 2: Run the 2-Stage GNU ddrescue cloning protocol (Always use a Mapfile)¶
Never use standard dd (dd if=/dev/sdc of=/dev/sdf) on a failing RAID drive: standard dd either aborts on the first bad sector or shifts output offsets if sync is misconfigured, and it has no memory of which sectors succeeded. GNU ddrescue (gddrescue package) uses an intelligent multi-pass state mapfile:
- Pass 1: Fast Non-Scraping Clone (
-nflag) to Rescue 99.99% of Healthy Sectors First: -f: Allows writing directly to block device/dev/sdf(triple-check that/dev/sdcis the source and/dev/sdfis the blank target).-n(--no-scrape): Tellsddrescueto immediately jump over any slow or bad sector region without grinding the read head, rescuing all healthy gigabytes at maximum speed (150 to 250 MB/s) before the drive's health worsens./root/raid_bay3_sdc.map: Saves the exact sector progress every 30 seconds. Even if the failing drive locks up and requires a power cycle, re-running the exact same command with the.mapfile resumes at the exact sector where it stopped.- Pass 2: Targeted Reverse/Trimming Scrape (
-d -r1) to Recover Marginal Bad Sectors: - Once Pass 1 finishes (
pct rescued: 99.99%), lower the kernel SCSI command timeout so each bad sector fails fast (5 secondsinstead of waiting30 to 180 secondsper sector): - Run Pass 2 with direct disk access (
-d) and 1 retry (-r 1): - Shut down, disconnect the original failing
/dev/sdcdrive, and set it safely on a shelf: From this point forward, you will work exclusively with your healthy clone (/dev/sdf) and your other healthy member drives.
Fase 3: montaxe segura de RAID de só lectura mediante Copy-on-Write (COW) superposicións e mdadm¶
Agora que tes clons de disco saudables, non executas nunca mdadm --assemble --force directamente nas unidades en bruto sen unha protección Copy-on-Write (COW) superposición ou --readonly flag. Why? Because --force reescribe os superbloques RAID nas unidades, e se un diario do sistema de ficheiros (ext4 / XFS) replays automatically on mount using the wrong drive combination, it permanently alters your clones.
Step 1: Inspect the RAID metadata across all member drives (mdadm --examine)¶
Run mdadm --examine en cada unidade de membro (ou partición, por exemplo, /dev/sda1 ou /dev/sda3 on Synology/QNAP NAS drives) and save the output to a text file:
Array UUID: Verify all drives belong to the exact same RAID pool.
2. Device Role: Shows the exact stripe index (Active device 0, Active device 1, Active device 2, Active device 3).
3. Update Time & Events: Identify the N - 1 drives (for RAID 5) or N - 2 drives (for RAID 6) that share the highest/closest Events count.
4. Raid Level, Layout, Chunk Size, e Data Offset: Record these exact numbers (e.g., Raid Level : raid5, Layout : left-symmetric, Chunk Size : 512K, Data Offset : 262144 sectors).
Step 2: Create non-destructive Copy-on-Write (COW) Device-Mapper overlays (overlay-create)¶
To guarantee 100 percent safety so that even if you force-assemble the array and run mount, zero bytes are ever written to your physical hard drives, create loop-backed dm-snapshot Copy-on-Write overlays in /tmp (or on an external scratch drive):
# Create a small sparse COW file + loop device for each member partition (e.g., sda3, sdf3, sdd3)
mkdir -p /root/cow
for dev in sda3 sdf3 sdd3; do
truncate -s 20G /root/cow/${dev}.cow
loop_base=$(losetup --find --show --read-only /dev/${dev})
loop_cow=$(losetup --find --show /root/cow/${dev}.cow)
sz=$(blockdev --getsz /dev/${dev})
echo "0 $sz snapshot $loop_base $loop_cow P 8" | dmsetup create ovl_${dev}
done
/dev/mapper/ovl_sda3, /dev/mapper/ovl_sdf3, e /dev/mapper/ovl_sdd3. The underlying physical drives (sda3, sdf3, sdd3) are locked --read-only at the kernel block layer. Any writes made by mdadm --force, LVM, or filesystem journal replay go harmlessly into the disposable /root/cow/*.cow files. If anything goes wrong, you simply delete the overlays and recreate them in 2 seconds.
Step 3: Force-assemble the array on the overlay devices and mount Read-Only¶
- Assemble the freshest
N - 1overlay devices using--assemble --force: - Why we leave out the stale/half-rebuilt drive (
sdb3): Forcingmdadmto assemble with the 3 up-to-date drives (sda3, clonedsdf3, esdd3) brings/dev/md99online in clean degraded mode ([U_UU]) using the exact latestEventsstate, whileddrescuealready rescued the data fromsdf3. - Activate LVM (
vgchange) if your NAS or server uses LVM on top of RAID (SynologySHR, QNAP, Proxmox, Ubuntu): (If your Synology NAS usesBtrfson LVM orSynology Hybrid RAID, you will now see/dev/vg1000/lvou/dev/vg1/volume_1appear.) - Mount the filesystem in strict Read-Only mode and evacuate your critical files immediately:
mkdir -p /mnt/raid_rescue # For ext4: mount -o ro,noload /dev/md99 /mnt/raid_rescue # (Or if noload refuses due to dirty journal, mount the COW overlay normally so the journal replays into the disposable COW file) mount -o ro /dev/md99 /mnt/raid_rescue # For Btrfs (Synology NAS): mount -t btrfs -o ro,usebackuproot,nologreplay /dev/vg1000/lv /mnt/raid_rescue - Copy your most critical folders first using
rsyncto an external backup drive:
Phase 4: Rescuing Hardware RAID Controllers (Dell PERC, LSI MegaRAID, HP Smart Array, Adaptec) and ZFS / Btrfs Pools¶
What if your array was managed not by Linux mdadm, but by a Hardware RAID Controller Card (Dell PERC H730/H740/H755, Broadcom / LSI MegaRAID, ou HP Smart Array) that now reports Virtual Disk Offline ou Foreign Configuration Found after a second drive hiccupped during rebuild?
1. Hardware RAID Cards (Dell PERC, LSI MegaRAID, HP Smart Array)¶
- ** NUNCA prema
"Clear Foreign Configuration"ou"Initialize Virtual Disk"**: - In Dell
PERC/ LSIMegaRAIDBIOS (Ctrl+RouHIIUEFI Configuration Utility),Clearwipes the RAID metadata from the drives, andInitializeoverwrites the data stripes. - If a healthy drive briefly dropped offline due to a loose backplane cable or timeout,
Import Foreign Configurationattempts to bring the drive back in; however, if one drive dropped weeks ago and a second dropped today, importing both or forcing the stale drive online via the controller BIOS can corrupt the array. - The Safe Software Reconstruction Method for Hardware RAID (
DDFMetadata /dmraid/R-Studio/UFS Explorer): - Hardware RAID cards store standard
SNIA DDF(Disk Data Format) metadata in the final sectors at the end of each hard drive, and stripe data across the drives starting at sector0(or after a small header offset) using standard64 KB,128 KB,256 KB, ou512 KBchunk sizes (Left-AsymmetricouLeft-Symmetric). - Instead of gambling with a destructive hardware RAID card BIOS:
- Label all drives by slot number and connect them to a standard SATA / non-RAID
IT-ModeHBA on a recovery workstation. - Clone the bad-sector drive using
GNU ddrescueas shown in Phase 2. - Open the cloned drives in
UFS Explorer RAID RecoveryouR-Studio Technician, ou armalos de só lectura en Linux usandodmsetup/mdadm --build: these tools automatically detect theLSI/PERC/HPstripe order, chunk size (64K/512K), e rotación de paridade na memoria de só lectura ao 100 por cento sen deixar que a tarxeta RAID de hardware escriba un só byte nas súas unidades.
- Label all drives by slot number and connect them to a standard SATA / non-RAID
2. TrueNAS/ZFS (RAID-Z1 / RAID-Z2) and Synology Btrfs Scrub/Resilver Failures¶
If your pool uses OpenZFS (RAID-Z1 ou RAID-Z2) on TrueNAS or Proxmox and a second drive throws CKSUM / READ errors during a zpool replace resilver:
1. Why ZFS handles bad sectors during resilvering better than hardware RAID, and when it still locks up:
- Unlike naive hardware RAID cards that drop an entire disk on a single bad sector, ZFS continues resilvering past localized READ errors and logs only the specific damaged file paths in zpool status -v.
- However, if the second failing drive hangs the SATA/SAS bus (I/O timeout / head thrashing) or drops completely offline (UNAVAIL / FAULTED), the pool enters SUSPENDED state (pool I/O is currently suspended).
2. How to rescue a SUSPENDED ou FAULTED ZFS RAID-Z1 / RAID-Z2 pool:
- Power down cleanly, clone the failing drive onto a healthy replacement disk using GNU ddrescue -f -n (ZFS partition labels include the exact vdev GUID at the start and end of the disk, so a raw sector clone is automatically recognized by ZFS as the original vdev member).
- Disconnect the physically dying drive, leave the ddrescue clone connected, and import the ZFS pool in strict Read-Only mode (readonly=on) without starting a write transaction or resuming the resilver:
-o readonly=on prevents ZFS from writing new uberblocks or attempting background healing writes, allowing you to copy every dataset off safely via zfs send ou rsync before rebuilding the pool from scratch.
Preguntas frecuentes¶
Why did you say NEVER to run CHKDSK ou fsck when my RAID array mounts with filesystem errors after a drive drop?¶
On a single standalone hard drive, running CHKDSK ou fsck repairs minor directory inconsistencies. On a multi-drive RAID 5 or RAID 6 array that suffered a double-drive drop or interrupted rebuild, filesystem errors are almost always caused by RAID stripe misalignment or a stale member drive being forced online (que significa cada 3rd ou 4th 64 KB chunk of data is being read from the wrong drive generation). If you let CHKDSK /f ou fsck -y run on a misaligned or out-of-sync RAID array, it misinterprets millions of valid files as corrupted metadata, truncates directory trees, dumps unnamed fragments into lost+found, and writes new parity blocks across all drives, making it mathematically impossible to ever reassemble the correct drive combination afterward. Always mount via Read-Only Copy-on-Write (COW) overlays first.
What if the partially rebuilt replacement drive (42% complete) AND the second drive that failed at 42% BOTH have needed data? Can I combine them?¶
Yes. This is a classic "Split-Horizon / Hybrid Clone" RAID rescue. Suppose Drive 2 died completely on Monday, you put in New Drive 2, and the rebuild made it to 42% (LBA 3,500,000,000) before Drive 3 suffered a fatal head crash at LBA 3,500,000,000. Notice what you actually have: for the first 42% of the array (0 % a 42%), New Drive 2 has already been 100 percent reconstructed. So for the first 42% of the LBA range, you have a complete set of stripes using Drive 1 + New Drive 2 + Drive 4 (treating crashed Drive 3 as the missing disk). And if Drive 3 only had a localized bad-sector patch at 42% (and is readable from 43% a 100% vía ddrescue), you use Drive 1 + Cloned Drive 3 + Drive 4 for the remaining 58% of the array.
Why do second drives fail so frequently right in the middle of a RAID 5 rebuild?¶
Two engineering realities combine to make RAID 5 rebuilds notorious for double-drive failures: (1) Identical Batch Aging: Most users buy 4 to 8 identical hard drives from the same retailer on the same day (same firmware, same factory lot, same spindle motor batch) and run them side-by-side at the exact same temperature for 35,000 hours; when Drive 1 wears out, Drive 2 and Drive 3 are days away from the same mechanical wear threshold. (2) Latent Uncorrectable Read Errors (UREs): Under normal daily operation, 95% of a hard drive's cold archival sectors are not read for months at a time, so latent bad sectors sit undiscovered. A RAID 5 rebuild forces a continuous 100% surface read of every single sector across every surviving drive (40 to 100 terabytes of continuous reading), immediately exposing any latent bad sector that accumulated over the past year.
Como sei que ficheiros foron afectados polos poucos sectores defectuosos ilexibles que ddrescue had to zero-fill?¶
GNU ddrescue includes a built-in forensic utility called ddrescuelog e ddru_ntfsfindbad / debugfs that maps the exact unrecovered sector LBA ranges from your .map file (status = '-' bad sectors) through the RAID stripe math directly to the filesystem inode table. Once you know which 4 KB blocks were zero-filled during cloning, you can query debugfs (icheck e ncheck on ext4) ou btrfs inspect-internal logical-resolve to print out the exact list of 2 or 3 filenames that overlapped those bad sectors, confirming with 100 percent mathematical certainty that every other file on your multi-terabyte server is completely uncorrupted.
Guías relacionadas¶
- RAID 5 or RAID 6 Synology, QNAP, or TrueNAS multiple drive drop and why forcing a rebuild destroys data
- RAID controller card death, foreign configuration import, and why you must never initialize disks
- NAS RAID array degraded, multiple drive failure, and ZFS/Btrfs recovery
- Fallo do disco duro e recuperación de datos
- Computer backups: 3-2-1 strategy and disaster recovery
- Números de emerxencia por país
Fontes e orientacións oficiais¶
- Linux Raid Wiki (
raid.wiki.kernel.org/index.php/Linux_Raid&Recovering_a_failed_software_RAID): Official KernelmdadmRecovery Protocol, Device-Mapper Copy-on-Write (dm-snapshot) Overlays, andEventsCounter Reconciliation. - GNU
ddrescueOfficial Engineering Manual (gnu.org/software/ddrescue/manual/ddrescue_manual.html): Multi-Phase Non-Scraping (-n) and Direct Access (-d) Sector Recovery Algorithms for Failing Magnetic Media. - Synology Official Knowledge Base (
kb.synology.com/en-global/DSM/tutorial/How_can_I_recover_data_from_my_DiskStation_using_a_PC): Mountingmdadm+LVM2+Btrfs/ext4NAS Volumes Read-Only on Ubuntu Rescue Systems. - Storage Networking Industry Association (
SNIA): Common RAID Disk Data Format (DDF) Specification for Hardware RAID Controller Metadata Portability.