Data recovery from RAM bit error

[nakano-sora]

I used ZFS, but many data is corrupted because of RAM bit error.
The ZFS dataset is still importable.
Only a single bit is corrupted, and I know the offset.
Can I fix it?
I read the code, but I don't know where raidz repairs from checksum.
How and where does raidz detect data error vdev to fix? Thanks.

[amotin]

Depending how and where the corruption has happened, ZFS may not have correct redundant data to recover. Or it may be opposite, when data are correct, but checksum is corrupted,and ZFS will still block the data access. That is why ECC memory is recommended -- to avoid this kind of uncertainty.

[amotin]

Checksums can not repair any errors, they are not a correction codes. Combination of checksums and RAIDZ though can recover up to one (or 2/3 for RAIDZ2/3) completely lost or corrupted drive. ZFS will retry reads in different combinations of drives trying to find one that matches the checksum, and if found it will issue recovery write. But if RAM can produce random new errors while ZFS tries to solve this puzzle, then result is unpredictable.

[nakano-sora]

I know checksum can't repair error, but I want to test to insert bit-fliped checksum puzzle.
Corrupt RAM is already replaced, so it will not produce new errors. I know the RAM error is only a single bit, so I want to try this.
I believe false positive rate is very low in such single bit case.
Where ZFS tries to solve the puzzle?

[nakano-sora]

I think raidz_reconstruct runs recovery, but I don't believe it checks all patterns.
Does raidz_reconstruct call itself indirectly?

[amotin]

See vdev_raidz_combrec() calling raidz_reconstruct().

[Germano0]

also worth reading
https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/