Finds and removes duplicate files
Dupseek groups files by size, then reads and compares small chunks of the files of the same size. It creates smaller groups depending on these comparisons. It proceeds with bigger and bigger chunks (of size up to a hard-coded limit). It stops reading from files as soon as they form a single-element group or they are read completely (which only happens when they have a very high probability of having duplicates). The program does remove files, but it asks first. Dupseek aims for maximum efficiency by keeping file reads to a minimum and is much better than other similar programs when dealing with groups of large files of the same size. It can be interrupted at any moment. The user is then presented with partial results and can either intervene manually or go on with the reading and computation, on a group-by-group basis.
released on 13 March 2005
19 December 2013
Leaders and contributors
Resources and communication
|Required to use||Perl|
|Required to use||File_Find|
This entry (in part or in whole) was last reviewed on 19 December 2013.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the page “GNU Free Documentation License”.
The copyright and license notices on this page only apply to the text on this page. Any software or copyright-licenses or other similar notices described in this text has its own copyright notice and license, which can usually be found in the distribution or license text itself.