May 4, 2008
MD5 Checksums bring peace of mind
Filed under: data storage, archival practices — Richard L. Hess @ 7:35 pm

If you ever worry about a bit error happening to your files and not finding out about it, you should use MD5 checksums (or some similar method) to be able to verify that the file has not changed.

The general theory behind a checksum (or “Message Digest” = “MD”) is that it provides a unique 128-bit number for each and every file, based on its content. If one bit changes, the MD5 checksum (sometimes called “hash”) changes. The checksum is repeatable, does not permit discovery of two different files that produce the same checksum, and is non-reversible (i.e. you can’t create the content from the checksum).

Common practice in audio archiving is to use 128-bit checksums, although 160-bit checksums are available. The other common practice is to take the entire filename, including extension, and add a second “MD5? extension to that. The *.*.md5 file contains the MD5 checksum plus generally the filename.

While there are many options for creating MD5 checksums, for Windows, FastSum is a convenient and low-cost package.

At any point in the future, to verify your files, you create a new MD5 checksum and compare it to the ones that were created when the file was created. If they match, you know your file has not changed. The likelihood of both the original file and the checksum file changing so that they still appear valid is essentially nil.

Some applications, like D-Space, check the integrity of all the files in the system on a regular basis using MD5 checksums.

If you receive WAV files from my, more than likely, you will also receive MD5 checksums in the same package, be that on a DVD, hard drive, or via the Internet.

It only takes a minute to create these and it truly does offer peace of mind.

==================

This article is online at:
http://richardhess.com/notes/2008/05/04/md5-checksums-bring-peace-of-mind/