RAID 5 or RAID 6 Degraded Array With Second Drive Failing During Rebuild (Why Never Run CHKDSK or fsck, ddrescue Sector Cloning, and mdadm Read-Only Assembly)¶
Emergency numbers differ by country. This page uses 911, the number for the United States and Canada. Many other countries use 112, 999, or another number. Find the number for your country.
Suia se kiliva malo ua faaletonu i se mea leaga RAID 5 pe RAID 6 'auʻaunaga teuina, Synology DiskStation, QNAP NAS, Linux mdadm array, or Dell PERC / HP Smart Array / LSI MegaRAID hardware controller only to watch a second remaining member drive throw UNC (Uncorrectable Read Error) bad sectors, Pending Sectors, or drop offline at 42% of the parity rebuild is one of the most dangerous storage emergencies in enterprise and home-lab computing. In a single-parity RAID 5 array (N - 1 fault tolerance), the mathematical XOR reconstruction of the replacement disk requires reading every single sector (LBA 0 i le LBA mulimuli) i le 100 pasene o totoe N - 1 member drives simultaneously. When modern high-capacity 8 TB ia 24 TB helium drives (purchased from the same manufacturing batch and subjected to identical thermal and vibration hours) undergo a multi-terabyte sequential rebuild read, a single latent media defect (URE) or weak read head on a second drive causes the RAID controller to abort the rebuild, kick the second drive out as Faulty / Crashed, and mark the entire storage pool Offline / Crashed / Superblock Event Count Mismatch.
Do not panic, and do not force a second rebuild or run filesystem repair utilities (fsck, e2fsck, xfs_repair, or Windows CHKDSK). Even when a second drive in RAID 5 (or third drive in RAID 6) develops 50 ia 5,000 bad sectors and gets kicked out of the array, 99.9999 percent of the sectors on that "failed" second drive are still 100 percent readable. By immediately halting the array, cloning every unstable member disk sector-by-sector onto healthy spare drives using GNU ddrescue, and assembling the cloned images in Read-Only / --assume-clean overlay mode (mdadm + dmsetup Copy-on-Write loop overlays), you can mount the filesystem safely and copy out virtually every file intact. For NAS appliance specifics and hardware controller failures, read our companion guides on RAID 5 or RAID 6 Synology, QNAP, or TrueNAS multiple drive drop, RAID controller card death and foreign configuration import, ma NAS RAID array degraded and ZFS/Btrfs recovery. If your server failure impacts life-safety systems or medical infrastructure, contact your local emergency number (911 i le Iunaite Setete ma Kanata, 112 i totonu o le Iuni a Europa, 999 i Peretania) pe siaki la matou lisi o numera fa'afuase'i.
Fai nei loa¶
- Abort any active parity rebuild or scrub immediately and power down the array cleanly (or disconnect SATA/SAS backplane power if a drive is making mechanical clicking or scraping noises):
- Why letting a stalling rebuild continue destroys the array: When a member drive hits a bad sector patch (
LBAread timeout) or has a degrading read/write head (media scratch), leaving the RAID controller retrying sectors for hours grinds the microscopic magnetic platter coating into dust across the entire track cylinder, turning a recoverable20-sectorfa'aletonu i se fa'alavelave mata'utia o le ulu o le ipu. - ** AUA LE tamoe
CHKDSK /f,fsck -y,e2fsck,btrfs check --repair, pexfs_repairi luga ole laiga RAID ua leaga, ua le toe fa'aogaina, po'o le afa-toe fausia**: - ** Aisea
CHKDSKmafsckmafua ai le 100% fa'atama'ia fa'amaumauga tumau ile RAID e le fa'atasi**: Afai na pa'u le Drive 2 i le Aso Lua ma fa'ate'a le Drive 3 i le taimi o le toe fausia o le Aso Tofi, o lemdadmpoʻo meafaigaluega RAIDEvent Counter(Update Time) i luga o na taʻavale e lua e ese mai le faitau afe o fefaʻatauaiga. Afai e te fa'amalosia-fa'apotopoto se fa'aputuga fa'atasi ma fa'atupu ta'avale e le fetaui po'o misi ta'avale ma fa'atagafsckpeCHKDSKe tusi i le fa'asologa, e va'aia e le faila faila o la'au inode gau e mafua mai fa'aletonu RAID ta'avale, ** tapeina le faitau miliona o fa'amaumauga fa'amaonia i totonu.lost+found, ma fa'asolo fa'asolo fa'amaumauga lelei ma fa'aletonu le tutusa i so'o se tisiki i le array**. - ** AUA LE kiliki
"Create Array","Initialize", pe"Clear Configuration"i se meafaigaluega RAID BIOS, ma E LE'I tamoemdadm --createleai--assume-cleanma se Kopi-i-Tusi (COW) fa'apipi'i fa'amau**: - Fa'atinoina o se tulaga masani
mdadm --create(e aunoa ma--assume-cleanma fa'afetaui le lapo'a sa'o, fa'atulagaina, ma fa'aputu fa'amaumauga) e amata loa ona tusia fa'atasiga fou i tisiketi uma, fa'aumatia le faila faila ma le B-la'au pe'āfai e i ai se parakalafa tasi (512Kvs.64Kfasi,left-symmetricvs.left-asymmetric, pe2048svs.262144sdata offset) e sese e tusa lava pe tasi le vaega. - Faailoga so'o se kiliva malo (
Bay 0 / Bay 1 / Bay 2 / Bay 3ma le numera fa'asologa) fa'atasi ai ma se fa'ailoga tumau po'o se mea fa'apipi'i A'O LE'I toso ese se ta'avale se tasi mai le ta'avale: - Tusi le numera faga sa'o ma le numera fa'asologa i luga o le igoa o ta'avale ta'itasi. O le iloa o le faga fa'aletino ma le siakiina
mdadm --examinemetadata e fa'asaoina itula o le toe fa'atulagaina o laina i se taimi mulimuli ane. - ** Fa'a'o'o le ta'avale ta'avale fa'aletonu lea vaega ta'itasi i ta'avale fou e fa'aogaina
GNU ddrescue(fa'atasi ai ma se fa'afanua) i luga o ports SATA/HBA motherboard sa'o a'o le'i taumafai i so'o se fa'apotopotoga RAID**: - Aua ne'i taumafai e fa'apipi'i pe fa'apipi'i se laina RAID e fa'aaoga ai le ta'avale mate fa'aletino. Fa'afeso'ota'i le ta'avale fa'aletonu i se SATA/AHCI po'o le LSI e le RAID
IT-ModeHBA uafu i luga o se fale faigaluega laveai Linux, faʻamalo le faʻapipiʻiina o le OS, ma tamoeddrescue -f -n /dev/sdX /dev/sdY rescue_driveX.mapo le mea lea e aveese ai vaega vave soifua maloloina i le pasi tasi ae leʻi paʻi i sone leaga. - ** Faʻapipiʻi le masini faʻapipiʻi i totonu
Read-Onlyfaiga (mdadm --assemble --readonlypo'o le fa'aaogainadmsetup/losetupRAM/faila fa'apipi'i)**: - E ala i le tu'uina o se Kopi-i-Tusi (
COW) fa'apipi'i faila i le pito i luga o tisiki sui auai ta'itasi, e mafai ona e su'e-fa'amauina le laina, toe ta'alo seext4peXFSapi talaaga, po'o le su'eina o se virtual rebuild ** a'o fa'amautinoa e leai ni paita e suia i luga o au fa'ailoga tisiki autu**.
Danger
O le "Ofo-of-Order Drive Force" Mailei (Stale Drive Overwriting Fresh Drives): Fa'apea e iai sau ta'avale 4 RAID 5 (Disk 1, Disk 2, Disk 3, Disk 4). Disk 3 na toilalo i le 6 masina talu ai, ma e te leʻi suia lava (taufetuli faʻaleagaina mo le 6 masina). I aso nei, Disk 2 atiina ae 100 vaega leaga ma pa'u ese. Aua le faamalosia le 6-masina le matua Disk 3 toe foi i luga ole laiga e toe fausia Disk 2. Afai e te faaaogaina Disk 1 + Disk 3 (6 months stale) + Disk 4 e toe fau pe fa'atūina le fa'asologa, o faila uma na faia pe fa'aleleia i le 6 masina talu ai o le a fa'aleagaina i lapisi binary e le mafai ona faitau. E tatau ona e fa'a-clone Disk 2 (le taavale na le manuia *aso *) ma ddrescue ma faʻapotopoto Disk 1 + Cloned Disk 2 + Disk 4.
Laau fa'ai'uga: lavea'iina se RAID 5 po'o le RAID 6 laina pe a le manuia le ta'avale lona lua a'o toe fausia¶
Mulimuli i le fa'asologa o galuega toe fa'afo'isia mo le fa'afoma'i fa'afoma'i e toe fa'afo'i ai lau 'au e aunoa ma le to'esea o se ta'i e tasi o totoe:
flowchart TD
Start["RAID 5 (2nd Drive) or RAID 6 (3rd Drive) Fails / Drops Offline During Rebuild"] --> StopNow["STEP 1: Abort Rebuild Immediately & Power Down Array (Prevent Platter Scoring on Failing Drive)"]
StopNow --> SoundCheck{"Did the Failing Drive Make Loud Clicking, Beeping, or Grinding Sounds?"}
SoundCheck -->|"YES: Mechanical Head Crash / Stiction"| CleanRoom["STOP: Do Not Power That Drive On Again. Send ONLY the Mechanically Failed Drive (Plus 1 Healthy Drive for Stripe Reference) to a Cleanroom Lab to Image Onto a Clone"]
SoundCheck -->|"NO: Spins Smoothly, Failed Due to Bad Sectors (UNC / Pending / Timeout)"| LabelDrives["Laasaga 2: Fa'aigoa Ta'avale Ta'avale Ta'itasi Ma Lona Numera Faga Sa'o & Numera Fa'asologa -> Fa'afeso'ota'i ile Linux Workstation e ala ile Direct SATA / IT-Mode HBA"]
LabelDrives --> InspectMeta["Fa'asolo 'mdadm --examine /dev/sd*' ma le 'smartctl -a /dev/sd*' i luga o ta'avale uma (Fa'amaumau Taimi Fa'afouga, Fa'asologa o Mea na tutupu, Matafaioi, Lapo'a, Layout, Fa'amaumauga Fa'amaumauga)"]
InspectMeta --> CloneDrives["LA'A'AGA 3: Fa'a'a'a'i Ta'avale Ta'avale Ta'avale Ta'avale Fa'aleaga / Leaga i se Ta'avale Fa'asao Soifua Maloloina Fa'aaogā le 'ddrescue -f -n' (Fa'aanapogi Mua'i, Ona Si'i lea Fa'atasi ma Mapfile)"]
CloneDrives --> PickFreshest["Fa'ailoa le N-1 (RAID 5) po'o le N-2 (RAID 6) Ta'avale Fa'atasi ma Fa'ata'ita'i 'Events' FA'ATA'O LELEI / TALA'I ('Aua ne'i fa'afefiloi se ta'avale tuai ma ta'avale fou)"]
PickFreshest --> CowOverlay["Laasaga 4: Fausia le Faitau-Na'o Loop / 'dmsetup' Copy-on-Write (COW) fa'alava i luga o ta'avale fa'a'ole'ole ina ia le mafai ona fa'asuia ulua'i fa'alava."]
CowOverlay --> AssembleRO["Tafe le 'mdadm --assemble --force --readonly' i luga o le Overlay Devices -> Mount Filesystem Read-Na'o ('mount -o ro,noload') -> Kopi uma Faʻamatalaga Mataʻutia i le Teuteu i fafo"]
Vaega 1: Malamalama pe aisea ua le manuia ai le taavale lona lua i le taimi o le toe fausia o le RAID (XOR Parity Math & Event Counters)¶
Ina ia malamalama pe aisea na pa'u ai lau laina i le ogatotonu o le toe fausia ma pe aisea e mafai ai ona toe maua au faʻamatalaga, suʻesuʻe pe faʻafefea RAID 5 ma RAID 6 galue moni i luga o le tisiki.
1. E faapefea XOR galuega toe fausia tutusa ma pe aisea 1 leaga vaega e fa'amuta se toe fausia atoa¶
I totonu o le 4-drive RAID 5 array (Drive A, Drive B, Drive C, Drive D), o fa'amaumauga e vaelua i le lapopo'a tele chunks (e masani lava 64 KB pe 512 KB) fa'ailoga i luga o tisiki, lea e tasi le pusa i laina ta'itasi o lo'o i ai le XOR Parity (P):
$\(\text{Parity } P = A \oplus B \oplus C\)$
Afai Drive C le manuia atoa ma e fa'aofi se avanoa Replacement Drive C_new, e tatau i le pule RAID ona toe fa'aleleia ta'i uma C_new e ala ile fa'atatauina:
$\(C_{\text{reconstructed}} = A \oplus B \oplus P\)$
- ** O le a le mea e tupu pe a Drive B taia se tasi e le mafai ona faitau 4 KB vaega (UNC sese) i 42% o le toe fausia**:
- Ina ia fa'atatauina lena tasi 4 KB poloka i luga C_new, o le pule *e tatau ona i ai se faitau aoga mai Drive A, Drive B, ma Drive D.
- O afea Drive B toe faafoi se meafaigaluega Read Error (Sense Key 03h / 11h Unrecovered Read Error) ina ua uma le 7 sekone (TLER / ERC taimi), tagata fa'atau ma atina'e fa'atonu RAID e masani ona fa'apolokalameina i se fa'alavelave fa'aletonu amio: nai lo le fa'amisi lea tasi 4 KB vaega, taina se poloka poloka, ma faauma le isi 99.9999% o le toe fausia, o le RAID controller faailoga Drive B pei FAILED / OFFLINE ma fa'ato'ilaloina le toe fausia atoa.
- I lena taimi, e lua ta'avale ua fa'ailogaina "Failed" (Drive C lea sa na o 42% toe fausia, ma Drive B lea e iai moni lava 99.99998% vaega soifua maloloina ma na'o 12 vaega leaga).
2. Aisea Event Counters (Update Time) fuafua po'o fea ta'avale e tatau ona e fa'aogaina mo le toe fa'aleleia¶
O taimi uma lava a RAID 5 pe RAID 6 fa'asologa e tusia fa'amaumauga po'o suiga o le setete (e pei o se ta'avale pa'u i fafo), le RAID metadata (mdadm superblock po'o meafaigaluega DDF metadata) faʻaopoopo i le 64-bit Events counter i ta'avale uma o lo'o ola nei.
Faatusatusa le mdadm --examine galuega faatino o se 4-drive masani RAID 5 mala i fea sdb toilalo ananafi (Events: 10450) ma sdc na pa'u i fafo i le taimi o le toe fausia o aso nei (Events: 10892):
| Avega Faaletino | Matafaioi i Array | Update Time |
Events Counter |
Soifua Maloloina Faaletino & Mea o Loo I Ai |
|---|---|---|---|---|
/dev/sda (Faga 1) |
Active device 0 |
Today 14:22:10 |
10892 |
100% Soifua Maloloina ma le taimi nei: O lo'o iai tusitusiga uma e o'o i le lona lua tonu na pa'u ai le toe fau i le aso nei. |
/dev/sdb (Faga 2) |
spare / faulty (Suia i le tisiki fou) |
Yesterday 09:15:00 |
10450 (Sole) |
Ua le manuia Ananafi: Ua misi le 29 itula o fa'amaumauga tusitusia, PO'O le suia i se tisiki fou e na'o 42% toe fausia (Reshape/Rebuild Pos). |
/dev/sdc (Faga 3) |
faulty (Pau i 42% toe fausia) |
Today 14:22:09 |
10890 (Tape e 2) |
99.999% Soifua Maloloina ma le taimi nei: Na'o le kiki i fafo ile 1 sekone muamua /dev/sda ma /dev/sdd aua na lavea 8 vaega leaga i le taimi o le toe fausia faitau. |
/dev/sdd (Faga 4) |
Active device 3 |
Today 14:22:10 |
10892 |
100% Soifua Maloloina ma le taimi nei: O lo'o iai tusitusiga uma e o'o i le lona lua tonu na pa'u ai le toe fau i le aso nei. |
- O le Tulafono Faaauro o le Toe Faʻaleleia o le RAID mai le laulau i luga:
- Matauina lena mea
/dev/sda(10892),/dev/sdc(10890), ma/dev/sdd(10892) e toetoe lava tutusa umaEventsfa'atau maiToday 14:22. - O le taimi lava matou te fa'akolone ai
/dev/sdci luga o se ta'avale maloloina fa'aaogaddrescue(toe faʻaleleia le 99.9999% o ona vaega ma leai se faʻatumuina o vaega e 8 e le mafai ona faitau), e mafai ona matou faʻamalosia faʻatasi/dev/sda+Cloned /dev/sdc+/dev/sdd(le amanaiaina le afa-toe fausia/dev/sdbatoatoa). O lena tu'ufa'atasiga e 3-ta'avale e maua ai le 100 pasene faifaipea, fa'aleaga i luga-i-le-luaRAID 5fa'asologa fa'atasi ai ma le saeia e leai se laina.
Vaega 2: La'asaga ta'i la'asaga fa'apipi'i vaega ole ta'avale fa'aletonu ma GNU ddrescue¶
Ae e te le'i pa'i atu mdadm po'o so'o se mea faigaluega fa'apipi'i RAID, e tatau ona e ** fa'aputuina le ta'avale (s) na lafo fa'aletonu faitau i ta'avale fa'alelei lelei e tutusa pe lapo'a le malosi paita.
Laasaga 1: Saunia le siosiomaga laveai Linux ma tape le auto-assembly¶
- Fa'atūina se fale faigaluega Linux (po'o le fa'aulu lau 'au'aunaga mai se la'au USB ola e pei o le SystemRescue
system-rescue.orgpoʻo le Ubuntu Live Server) faʻatasi ai ma taʻavale e fesoʻotaʻi i tulaga masani ** motherboard SATA ports ** poʻo se LSI HBA i totonu.IT Mode(E le o se RAID pass-through). - ** Taofi Linux mai le otometi ona taumafai e faʻapipiʻi pe faʻapipiʻi lau laina RAID malepe i luga o le taʻavale**:
- A faʻapipiʻi SystemRescue poʻo le Ubuntu Live, siaki
cat /proc/mdstat. Afai e faʻapipiʻi faʻatasi e Linux se vaegamdmasini i tua, taofi vave i le faitau-na'o le saogalemu a'o le'i faia ata: - Faailoa ta'avale ta'itasi i le fa'ata'ita'iga sa'o, numera fa'asologa, ma le tele o le byte ('aua ne'i mate
/dev/sdXmataitusi, aua e toe tofia e Linux tusi taʻavale i luga o le toe faʻafouina): Tusi i lalo po o fea/dev/sdXe fetaui ma lau Fail Source Drive (mo se faataitaiga,/dev/sdc, fa'asologaWCC4E12345) ma o fea/dev/sdYo lau Brand-New Blank Target Drive (mo se faataitaiga,/dev/sdf, fa'asologaZGG9988776).
Laasaga 2: Faʻasolo le 2-Stage GNU ddrescue fa'a-cloning protocol (Fa'aoga i taimi uma se Mapfile)¶
Aua ne'i fa'aaogaina tulaga masani dd (dd if=/dev/sdc of=/dev/sdf) i luga o se ta'avale RAID vaivai: tulaga masani dd pe fa'amuta i le vaega muamua leaga po'o le suia o fa'aputuga fa'asoa pe a sync ua sese, ma e leai se manatua po o fea vaega na manuia. GNU ddrescue(gddrescue package) fa'aaogaina se fa'afanua fa'afanua fa'afanua tele-pass:
- Pase 1: Fa'ato'aga Anapogi e le'i Si'iina (
-nfu'a) ia Laveai 99.99% o Vaega Soifua Maloloina Muamua: -f: Fa'ataga le tusitusi sa'o e poloka ai masini/dev/sdf(faatolu-siaki lena/dev/sdco le puna ma/dev/sdfo le taulaiga avanoa).-n(--no-scrape): Fai maiddrescueia vave oso i luga o soʻo se itulagi lemu pe leaga e aunoa ma le oloina o le ulu faitau, laveai uma gigabytes maloloina i le saoasaoa maualuga (150 to 250 MB/s) a o leʻi faʻaleagaina le soifua maloloina o le taʻavale./root/raid_bay3_sdc.map: Faasaoina le alualu i luma o vaega uma i le 30 sekone. E tusa lava pe loka le ta'avale vaivai ma mana'omia se ta'amilosaga malosi, toe fa'asolo le fa'atonuga tutusa ma le.maptoe amata faila i le vaega tonu lea na taofi ai.- Pase 2: Fa'ata'ita'i Fa'asaga i tua/Otioti (
-d -r1) e Toe Fa'afo'i Vaega Leaga Fa'apito: - A mae'a le Pasi 1 (
pct rescued: 99.99%), fa'aititia le kernel SCSI le taimi e fa'atonu ai ina ia vave ona pa'u vaega leaga ta'itasi (5 secondsnai lo le faatalitali30 to 180 secondsta'i vāega): - Fa'asolo le Pass 2 fa'atasi ai ma le avanoa tu'u sa'o (
-d) ma le 1 toe taumafai (-r 1): - ** Tapuni i lalo, motusia le mea muamua na faaletonu
/dev/sdcave taavale, ma tuu saogalemu i luga o se fata**: Mai le taimi lenei i luma, o le a e galue faʻatasi ma lau clone maloloina (/dev/sdf) ma isi au ta'avale a le tagata soifua maloloina.
Vaega 3: Saogalemu Faitau-Na'o le fa'apotopotoga RAID fa'aoga Kopi-i-Tusi (COW) fa'apipi'i ma mdadm¶
O lea la o lo'o i ai au fa'aputu tisiki maloloina, 'aua ne'i tamo'e mdadm --assemble --force sa'o i luga o ta'avale mata e aunoa ma se puipuiga Kopi-i-Tusi (COW) fa'apipi'i po'o --readonly fu'a. Aisea? Aua --force toe tusia le RAID superblocks i luga o taʻavale, ma afai o se faila faila (ext4 / XFS) toe taʻalo otometi i luga o le mauga e faʻaaoga ai le tuʻufaʻatasiga taʻavale sese, e suia tumau ai au clones.
Laasaga 1: Siaki le RAID metadata i ta'avale uma a sui (mdadm --examine)¶
Taufetuli mdadm --examine i luga o taavale a tagata uma (poʻo le vaeluaga, faʻataʻitaʻiga, /dev/sda1 pe /dev/sda3 i luga ole Synology/QNAP NAS drives) ma sefe le gaioiga i se faila faila:
Array UUID: Faʻamautinoa o taʻavale uma e iai i le vaitaele RAID tutusa.
2. Device Role: Fa'aali le fa'ailoga sa'o fa'ailoga (Active device 0, Active device 1, Active device 2, Active device 3).
3. Update Time & Events: Faailoa le N - 1 taavale (mo RAID 5) po o N - 2 taavale (mo RAID 6) lea fa'asoa maualuga/latalata Events faitau.
4. Raid Level, Layout, Chunk Size, ma Data Offset: Faamaumau numera sa'o nei (fa'ata'ita'iga, Raid Level : raid5, Layout : left-symmetric, Chunk Size : 512K, Data Offset : 262144 sectors).
Laasaga 2: Fausia Kopi-i-Tusi e le fa'aleagaina (COW) fa'apipi'i masini-Mapper (overlay-create)¶
Ia mautinoa le 100 pasene le saogalemu e tusa lava pe e te faʻamalosia-faʻapipiʻi le laina ma tamoe mount, zero bytes o lo'o tusia i au ta'avale mālosi fa'aletino, fa'atupu fa'asolo-fa'asolo dm-snapshot Kopi-i-Tusi fa'alava i totonu /tmp (po'o luga o se ta'avale mai fafo):
# Create a small sparse COW file + loop device for each member partition (e.g., sda3, sdf3, sdd3)
mkdir -p /root/cow
for dev in sda3 sdf3 sdd3; do
truncate -s 20G /root/cow/${dev}.cow
loop_base=$(losetup --find --show --read-only /dev/${dev})
loop_cow=$(losetup --find --show /root/cow/${dev}.cow)
sz=$(blockdev --getsz /dev/${dev})
echo "0 $sz snapshot $loop_base $loop_cow P 8" | dmsetup create ovl_${dev}
done
/dev/mapper/ovl_sda3, /dev/mapper/ovl_sdf3, ma /dev/mapper/ovl_sdd3. O ta'avale fa'aletino autu (sda3, sdf3, sdd3) ua lokaina --read-only i le vaega poloka poloka fatu. So'o se tusitusiga na faia e mdadm --force, LVM, po'o le toe fa'afo'i fa'asologa o faila faila e alu ma le le afaina i le tu'uina atu /root/cow/*.cow faila. Afai e iai se mea e faaletonu, na'o lou tapeina o mea fa'apipi'i ma toe fai ile 2 sekone.
Laasaga 3: Fa'amalosi-fa'apotopoto le laina i luga o masini fa'apipi'i ma fa'apipi'i le Faitau-Na'o¶
- ** Faʻapotopoto le mea fou
N - 1fa'aogaina masini fa'aoga--assemble --force**: - Aisea tatou te tu'u ai le ta'avale tuai/afa toe fausia (
sdb3): Fa'amalosimdadme faʻatasi ma le 3 faʻaonaponei taʻavale (sda3, fa'aclonesdf3, masdd3) aumai/dev/md99luga ole laiga ile tulaga mama mama ([U_UU]) fa'aaoga le sa'o lata maiEventssetete, a oddrescueua uma ona laveaʻiina faʻamatalaga maisdf3. - ** Fa'agaoioia LVM (
vgchange) pe a faʻaaogaina e lau NAS poʻo le server le LVM i luga ole RAID (SynologySHR, QNAP, Proxmox, Ubuntu)**: - (Afai e faʻaaogaina e lau Synology NAS
Btrfsi luga o le LVM poʻoSynology Hybrid RAID, o le ae vaai nei/dev/vg1000/lvpe/dev/vg1/volume_1aliali mai.)* - ** Fa'amauina le faila faila i le fa'atonu Faitau-Na'o ma aveese vave au faila taua**:
mkdir -p /mnt/raid_rescue # For ext4: mount -o ro,noload /dev/md99 /mnt/raid_rescue # (Or if noload refuses due to dirty journal, mount the COW overlay normally so the journal replays into the disposable COW file) mount -o ro /dev/md99 /mnt/raid_rescue # For Btrfs (Synology NAS): mount -t btrfs -o ro,usebackuproot,nologreplay /dev/vg1000/lv /mnt/raid_rescue - ** Kopi muamua au faila pito sili ona taua
rsynci se ta'avale faaleoleo fafo**:
Vaega 4: Lavea'i Mea Fa'atonu RAID Pule (Dell PERC, LSI MegaRAID, HP Smart Array, Adaptec) ma ZFS / Btrfs Pools¶
Ae faʻapefea pe a fai o lau faʻasologa e leʻo pulea e Linux mdadm, ae e ala i le Kata Pule RAID Meafaigaluega (Dell PERC H730/H740/H755, Broadcom / LSI MegaRAID, pe HP Smart Array) lea ua lipotia nei Virtual Disk Offline po o Foreign Configuration Found ina ua mae'a le ta'avale lona lua na mapu i le taimi o le toe fausia?
1. Meafaigaluega RAID Kata (Dell PERC, LSI MegaRAID, HP Smart Array)¶
- ** AUA LE kiliki
"Clear Foreign Configuration"pe"Initialize Virtual Disk"**: - I Dell
PERC/ LSIMegaRAIDBIOS (Ctrl+RpeHIIUEFI Configuration Utility),Clearsolo le RAID metadata mai taʻavale, maInitializefa'asolo fa'ailoga fa'amaumauga. - Afai na pa'u i luga ole laiga mo se taimi puupuu se taavale maloloina ona o se uaea i tua o le vaalele po o le taimi e malolo ai,
Import Foreign Configurationtaumafai e toe aumai le taavale i totonu; peita'i, afai na pa'u le taavale e tasi i vaiaso ua mavae ae pa'u le lua i le asō, o le faaulufaleina mai o mea uma e lua po o le faamalosia o le taavale tuai i luga o le initaneti e ala i le BIOS controller e mafai ona faaleagaina le array. - ** Le Safe Software Toe Fausiaina Metotia mo Hardware RAID (
DDFMetadata /dmraid/R-Studio/UFS Explorer)**: - Meafaigaluega RAID kata teuina tulaga masani
SNIA DDF(Disk Data Format) metadata i vaega mulimuli i le fa'ai'uga o ta'avale malo ta'itasi, ma fa'asolo fa'amaumauga i luga o ta'avale amata i le vaega0(po'o le mae'a o se fa'aulu la'ititi fa'apipi'i) fa'aaoga tulaga masani64 KB,128 KB,256 KB, pe512 KBvaega tetele (Left-AsymmetricpeLeft-Symmetric). - Nai lo le faitupe fa'atasi ma se meafaigaluega fa'aleagaina RAID card BIOS:
- Fa'ailoga ta'avale uma ile numera slot ma fa'afeso'ota'i i se ** SATA masani / non-RAID
IT-ModeHBA** i luga o se fale faigaluega toe faaleleia. - Fa'asolo le ta'avale leaga e fa'aaoga
GNU ddrescuepei ona faʻaalia i le Vaega 2. - Tatala le ta'avale cloned i
UFS Explorer RAID Recoverypo oR-Studio Technician, poʻo le faʻapotopotoina naʻo le faitau i le Linux faʻaaogadmsetup/mdadm --build: e otometi lava ona iloa e nei meafaigaluega leLSI/PERC/HPfa'asologa laina, lapo'a fasi (64K/512K), ma le suiga tutusa i le 100 pasene faitau-naʻo manatua e aunoa ma le tuʻuina atu o le RAID card meafaigaluega e tusi se paita tasi i au taʻavale.
- Fa'ailoga ta'avale uma ile numera slot ma fa'afeso'ota'i i se ** SATA masani / non-RAID
2. TrueNAS / ZFS (RAID-Z1 / RAID-Z2) ma le Synology Btrfs La'u/Ta'u mea ua le manuia¶
Afai e fa'aaoga e lau vaitaele OpenZFS (RAID-Z1 pe RAID-Z2) i le TrueNAS poʻo le Proxmox ma le togi lona lua CKSUM / READ mea sese i le taimi a zpool replace tupe siliva:
1. **Aisea e taulimaina ai e le ZFS vaega leaga i le taimi e toe faʻaleleia ai e sili atu nai lo meafaigaluega RAID, ma pe a loka pea :
- E le pei o kata RAID meafaigaluega faʻapitoa e paʻu ai se tisiki atoa i luga o se vaega leaga e tasi, o le ZFS o loʻo faʻaauau pea ona toe faʻaleleia i tua atu o le faʻaogaina. READ mea sese ma ogalaau naʻo ala faila faʻaleagaina i zpool status -v.
- Ae peitaʻi, afai o le taʻavale lona lua e le mafai ona tautau le pasi SATA / SAS (I/O timeout / ulu sasa'a) po'o le pa'u atoa i luga ole laiga (UNAVAIL / FAULTED), ua ulufale le vaitaele SUSPENDED setete (pool I/O is currently suspended).
2. E faapefea ona laveai a SUSPENDED pe FAULTED ZFS RAID-Z1 / RAID-Z2 vaita'ele:
- Fa'amama lelei i lalo, fa'a'o'o le ta'avale vaivai i luga o se tisiki sui lelei e fa'aaoga GNU ddrescue -f -n (ZFS vaelua igoa e aofia ai le saʻo vdev GUID i le amataga ma le fa'ai'uga o le tisiki, o lea e otometi lava ona iloa e le ZFS se fa'ailoga vaega mata'utia e pei o le ulua'i. vdev sui usufono).
- Aveese le ta'avale ua mate fa'aletino, tu'u le ddrescue feso'ota'i clone, ma fa'aulufale mai le vaitaele o le ZFS ile ** fa'atonu Faitau-Na'o (readonly=on) e aunoa ma le amataina o se fefa'ataua'iga tusitusia po'o le toe fa'aauauina o le tupe fa'aola:
-o readonly=on taofia le ZFS mai le tusiaina o uberblocks fou poʻo le taumafai e faʻamalolo le talaaga tusitusi, faʻatagaina oe e kopiina faʻamaumauga uma ma le saogalemu e ala i zfs send pe rsync a o lei toe fausia le vaitaele mai le sasa.
Fesili e fai soo¶
Aisea na e fai mai ai E LE'I tamoe CHKDSK pe fsck pe a faʻapipiʻi laʻu RAID faʻasologa i faila faila pe a maeʻa le pa'ū o le taavale?¶
I luga o se kiliva malo e tasi, tamoe CHKDSK pe fsck toe fa'afo'i fa'aletonu laiti fa'atonuga. I luga o a multi-drive RAID 5 poʻo le RAID 6 array na mafatia i se pa'ū faʻalua-taʻavale poʻo le toe faʻalavelaveina, fa'aletonu faiga faila e toetoe lava a mafua mai i RAID stripe misalignment po'o se ta'avale a le tagata ua fa'amalosi i luga ole laiga (o lona uiga uma lona 3 pe lona 4 64 KB vaega o faʻamatalaga o loʻo faitau mai le faʻatupuina o le taʻavale sese). Afai e te faatagaina CHKDSK /f pe fsck -y tamomoe i luga ole laiga RAID e le fetaui, e faauigaseseina ai le faitau miliona o faila aoga e pei o metadata leaga, tipi laau tusi, lafoa'i vaega e le o ta'ua i totonu. ua leiloa+ua maua, ma tusia ni poloka parite fou i ta'avale uma, ma fa'amatematika e le mafai ona toe fa'aopoopo le tu'ufa'atasiga sa'o o le ta'avale mulimuli ane. Fa'amau i taimi uma e ala Faitau-Na'o Kopi-i-Tusi (POPU) fa'apipi'i tulaga tasi.
Ae fa'afefea pe a fai o le ta'avale ua toe fa'aleleia fa'atasi (42% mae'a) MA le ta'avale lona lua na le manuia i 42% E LAUA e manaʻomia faʻamatalaga? E mafai ona ou tuufaatasia?¶
Ioe. Ole mea masani lea "Split-Horizon / Hybrid Clone" RAID laveai. Faapea Avetaavale 2 maliu atoa i le Aso Gafua, na e tuu i totonu Ta'avale Fou 2, ma o le toe fausia na taunuu i 42% (LBA 3,500,000,000) muamua Avetaavale 3 sa lavea ai le ulu i LBA 3,500,000,000. Matau mea o loʻo ia te oe: mo le muamua 42% o le faasologa (0% ia 42%), Ta'avale Fou 2 ua uma ona 100 pasene toe fausia. O lea mo le muamua 42% o le laina LBA, o lo'o ia te oe se seti atoa o ta'aloga e fa'aaoga Avea 1 + Avega Fou 2 + Avega 4 (togafitiga pa'ū Avetaavale 3 e pei o le tisiki misi). Ma afai Avetaavale 3 sa na'o se vaega leaga fa'apitonu'u i 42% (ma e mafai ona faitau mai 43% ia 100% auala ddrescue), e te faʻaaogaina Aveta'i 1 + Aveta'i 3 + Avetaavale 4 mo le vaega o totoe 58% o le fa'asologa.
Aisea e le manuia ai ta'avale lona lua i le ogatotonu tonu o le RAID 5 toe fausia?¶
E lua fa'ainisinia mea moni e tu'ufa'atasia e fai RAID 5 toe fausia ta'uta'ua mo fa'aletonu ta'avale lua: (1) Tulaga Tulaga Matua: O le tele o tagata fa'aoga e fa'atauina le 4 i le 8 ta'avale malosi tutusa mai le fa'atau oloa i le aso e tasi (e tasi le firmware, tutusa falegaosimea, tutusa foʻi afi afi) ma fa'ata'ita'i fa'atasi i le vevela tutusa mo le 35,000 itula; pe a vaivai le Drive 1, o le Drive 2 ma le Drive 3 o loʻo mamao ese mai aso e tasi mai le faʻaogaina o masini. (2) Sese Faitau e le mafai ona fa'asa'oina natia (UREs): I lalo o galuega masani i aso taitasi, 95% e le faitau masina i le taimi, vaega leaga o lo'o natia e le'i maua. A RAID 5 toe fausia malosiaga a faifai pea 100% faitau i luga ole vaega ta'itasi i ta'avale ola uma (40 i le 100 terabytes o le faitau fa'aauau), fa'aalia loa so'o se vaega leaga natia na fa'aputuina i le tausaga talu ai.
E fa'afefea ona ou iloa poo fea faila na afaina i nai vaega leaga e le mafai ona faitau ddrescue sa tatau ona leai-faatumu?¶
GNU ddrescue e aofia ai se fa'aoga mo su'esu'ega fa'apitoa e ta'ua ddrescuelog ma ddru_ntfsfindbad / debugfs o lo'o fa'afanua le vaega tonu LBA e le'i toe fa'aleleia mai lau .map faila (tulaga = '-' leaga vaega) e ala i le RAID stripe numera saʻo i le faila faila inode laulau. A e iloa loa po o fea 4 KB blocks were zero-filled during cloning, you can query debugfs (icheck ma ncheck luga ext4) po o btrfs inspect-internal logical-resolve to print out the exact list of 2 or 3 filenames that overlapped those bad sectors, confirming with 100 percent mathematical certainty that every other file on your multi-terabyte server is completely uncorrupted.
Taiala fa'atatau¶
- RAID 5 or RAID 6 Synology, QNAP, or TrueNAS multiple drive drop and why forcing a rebuild destroys data
- RAID controller card death, foreign configuration import, and why you must never initialize disks
- NAS RAID array degraded, multiple drive failure, and ZFS/Btrfs recovery
- Fa'aletonu le ta'avale malo ma le toe fa'aleleia o fa'amaumauga
- Fa'amaumauga fa'akomepiuta: 3-2-1 fuafuaga ma fa'alavelave fa'afuase'i
- Numera o fa'alavelave fa'afuase'i ile atunu'u
Punavai ma ta'iala aloa'ia¶
- Linux Raid Wiki (
raid.wiki.kernel.org/index.php/Linux_Raid&Recovering_a_failed_software_RAID): Official KernelmdadmRecovery Protocol, Device-Mapper Copy-on-Write (dm-snapshot) Overlays, andEventsCounter Reconciliation. - GNU
ddrescueOfficial Engineering Manual (gnu.org/software/ddrescue/manual/ddrescue_manual.html): Multi-Phase Non-Scraping (-n) ma Avanoa Tu'usa'o (-d) Algorithms Toe Fa'aleleia a le Vaega mo Ala o Fa'asalalauga maneta ua le manuia. - Synology Official Knowledge Base (
kb.synology.com/en-global/DSM/tutorial/How_can_I_recover_data_from_my_DiskStation_using_a_PC): * Faʻamauinamdadm+LVM2+Btrfs/ext4NAS Volumes Faitau-Na'o luga ole Ubuntu Rescue Systems*. - Fa'alapotopotoga o Alamanuia o Feso'otaiga Teuina (
SNIA): * Fa'asologa masani RAID Disk Fa'amaumauga (DDF) Fa'amatalaga mo Meafaigaluega RAID Pule Metadata feavea'i*.