WINtools and Duplicate File Management
Here are some useful windows tools that I have recently written:
- EDT - This is a port of my EDT editor to windows
Not written lately, but ported.
- VLT - View Large Text files
A port of my VLT utility, can look at very large text files (8m lines)
- FDF - Find Duplicate Filess
My first attempt at finding duplicates.
FDF takes a very large directory listing produced by windows:
DIR/s directory >list and examines all the files listed to report
- DFF - Duplicate File Finder
Improves on FDF in several ways:
- Generates it's own directory list file (smaller and easier to change)
- Organiezes output by directory
- Can include "where the dups are" information in the output.
- BDA/EDA - Tools to manage "Daves Archive"
Dave's Archives contain the smallest possible representation of a
complete directory tree:
- Only one copy of the data for duplicate files is stored.
- Duplicate filenames are stored only once.
- Path information is stored only once per directory, and
only additions to a path are stored (adding to last path).
- CMPTREE - Compare two directory trees
Performs a fast and complete comparison of two directory trees,
giving you details on whats added or removed in either tree, and
what files are different.
- SDIR - Small DIR command
Does the same as DIRexcept it's output is very small and concise, good
for text scanning large directoy listing to locate files.
(much faster than scanning a directory tree)
WHY? Over the course of my career I have collected a LOT of files. Backups,
useful software, documentation and a lot more. Years ago, I pretty much
gave up keeping track of it all, and began just "filling drives". I was
thinking that it would be OK as long as I kept it, because one day I could
That day has finally arrived... but I have DOZENs (possibly 100's of drives
which over the years have been copied/replaced with larger drives (mostly
500G to over 1TB units) and as it turns out, there is a LOT of dupliction.
These are some tools I wrote to help finding and manage duplicates in a very
large file collection. They have proven so useful that I tidied them up a
little and am publishing them here. Hopefully you will find them as useful.
NOTE: These are WIN32 console programs. They run from the command line and
read/write information as text files. This imposes the least restriction on
how you can control what they look at and interpret the information produced.
If you are uncomfortable with using the command line, don't know what a text
file is, prefer pretty (and proprietery) output, then these are probably NOT
tools you would find useful.
Also note: These were compiled with LCCWIN32 a windows port of GCC. Not having
written the compiler, I cannpt speek to quality and reliability of the
executables. LCC uses a "replica" of the old TURBO-C console video functions
which are not nearly as capable as the text windowing library in my MICRO-C/PC.
There are also some issues which may affect these programs:
- It is incomplete, with much of the documentation and header files saying
"does nothing" and/or "for compatability only", things I have noticed:
- The right CONTROL key does not work correctly, many keys reporting the same
as a non-control keypress. The LEFT control key seems to work ok.
- Seems to only work (mostly) correctly on a 25x80 format screen.
- Many functions like "clear to end of window" and "clear to end of line"
will happily proceed further. To deal with this I wrote my own layer over
the video library. Not always the fastest, but seems to work OK.
- I have not found a way to display in the lower-right corner of the screen
without scrolling it. For this reason my video layer treats the lower line as
79 characters wide.
Back to Dave's Personal Page