Can intelligence agencies read overwritten data

Daniel Feenberg concludes that, "Gutmann's claim [that intelligence agencies can read overwritten data] belongs in the category of urban legend. Another fact to ponder is the failure of anyone to read the "18 minute gap" Rosemary Woods created on the tape of Nixon discussing the Watergate break-in.

In spite of the fact that the data density on an analog recorder of in the s was approximately one million times less than current drive technology, and that audio recovery would not require a high degree of accuracy, not one phoneme has been recovered.

If the zeroing procedure skipped remapped blocks, it may be possible to recover a few blocks hundreds, maybe a few thousand. In practice, this would amount to very small byte pieces of random files. With a thousand such blocks, you'd only recover about k of random data. I inadvertently stumbled upon the link 40Hz gives in the opening post before actually seeing the post. What I unearthed was some further, corroborating material to the OP.

Craig Wright, a forensics expert, claims to have put this legend finally to rest. He and his colleagues ran a scientific study to take a close look at hard disks of various makes and different ages, overwriting their data under controlled conditions and then examining the magnetic surfaces with a magnetic-force microscope. They concluded that, after a single overwrite of the data on a drive, whether it be an old 1-gigabyte disk or a current model at the time of the study , the likelihood of still being able to reconstruct anything is practically zero.

Well, OK, not quite: a single bit whose precise location is known can in fact be correctly reconstructed with 56 per cent probability in one of the quoted examples.

To recover a byte, however, correct head positioning would have to be precisely repeated eight times, and the probability of that is only 0. Recovering anything beyond a single byte is even less likely. Nevertheless, that doesn't stop the vendors of data-wiping programs offering software that overwrites data up to 35 times, based on decades-old security standards that were developed for diskettes.

Although this may give a data wiper the psychological satisfaction of having done a thorough job, it's a pure waste of time. Something much more important, from a security point of view, is actually to overwrite all copies of the data that are to be deleted. If a sensitive document has been edited on a PC, overwriting the file is far from sufficient because, during editing, the data have been saved countless times to temporary files, back-ups, shadow copies, swap files Really, to ensure that nothing more can be recovered from a hard disk, it has to be overwritten completely, sector by sector.

Although this takes time, it costs nothing: the dd command in any Linux distribution will do the job perfectly. It seems quite a thorough coverage. Skepticism may be advisable. Claims that intelligence agencies can read overwritten data on disk drives have been commonplace for many years now.

The most commonly cited source of evidence for this supposed fact is a paper Secure Deletion of Data from Magnetic and Solid-State Memory by Peter Gutmann presented at a Usenix conference. I found this an extraordinary claim, and therefore deserving of extraordinary proof. Of course, modern operating systems can leave copies of " deleted" files scattered in unallocated sectors, temporary directories, swap files,remapped bad blocks, etc, but Gutmann believes that an overwritten sector can be recovered under examination by a sophisticated microscope and this claim has been accepted uncritically by numerous observers.

I don't think these observers have followed up on the references in Gutmann's paper, however. Gutmann explains that when a 1 bit is written over a zero bit, the "actual effect is closer to obtaining a. Given that, and a read head 20 times as sensitive as the one in a production disk drive, and also given the pattern of overwrite bits, one could recover the under-data.

The references Gutmann provides suggest that his piece is much overwrought. None of the references lead to examples of sensitive information being disclosed. Rather, they refer to experiments where STM microscopy was used to examine individual bits, and some evidence of previously written bits was found.

Gutmann mentions that after a simple setup of the MFM device, that bits start flowing within minutes. This may be true, but the bits he refers to are not from disk files, but pixels in the pictures of the disk surface. Charles Sobey [ A single write is sufficient if the overwrite is truly random, even given an STM microscope with far greater powers than those in the references.

In fact, data written to the disk prior to the data whose recovery is sought will interfere with recovery just as must as data written after - the STM microscope can't tell the order in which magnetic moments are created.

It isn't like ink, where later applications are physically on top of earlier markings. Recently I was sent a fascinating piece by Wright, Kleiman and Sundhar who show actual data on the accuracy of recovered image data. While the images include some information about underlying bits, the error rate is so high that it is difficult to imagine any use for the result. While the occasional word might be recovered out of thousands, the vast majority of apparently recovered words would be spurious.

Another fact to ponder is the failure of anyone to read the "18 minute gap" Rosemary Woods created on the tape of Nixon discussing the Watergate break-in.

In spite of the fact that the data density on an analog recorder of in the s was approximately one million times less than current drive technology, and that audio recovery would not require a high degree of accuracy, not one phoneme has been recovered. Feenberg links to a paper by Charles Sobey: " Recovering unrecoverable data ".

The link he uses is stale; this one is live at the moment. However Sobey's paper is about recovery of data from failed drives, not overwritten ones:. If the disk is not physically damaged, the user's data is still there, unless it has been overwritten. I expect the real explanation is far more prosaic. How would you confirm that the overwrite was not pseudo-random? Smashing the drive with a sledgehammer is easy to do, easy to confirm, and very hard to get wrong.

Or it may be in the category of marketing hype. Additional information may be sent to feenberg at nber dot org. Rugar, H. Mamin, P. Guenther, S. Lambert, J. Stern, I. McFadyen, and T. Yogi, Journal of Applied Physics, Vol. Wright, C. RSS - Posts. RSS - Comments. Never Ending Security It starts all here.