Efficiently delete large directory containing thousands of files

kernel_kate

We have an issue with a folder becoming unwieldy with hundreds of thousands of tiny files. There are so many files that performing rm -rf returns an error and instead what we need to do is something like: find /path/to/folder -name "filenamestart*" -type f -exec rm -f {} \; This works but is very slow and constantly fails from running out of...

Originally asked by Toby on Stack Exchange (unix) (CC BY-SA 3.0).

uphilltech

Good Linux question — answering as the UphillTech team:

Using rsync is surprising fast and simple. mkdir empty_dir rsync -a --delete empty_dir/ yourdirectory/ @sarath's answer mentioned another fast choice: Perl! Its benchmarks are faster than rsync -a --delete . cd yourdirectory perl -e 'for(<>){((stat)[9]<(unlink))}' or, without the stat (it's debatable whether it is needed; some say that may be faster with it, and others say it's faster without it): cd yourdirectory perl -e 'for(<>){unlink}' Sources: https://stackoverflow.com/questions/1795370/unix-fast-remove-directory-for-cleaning-up-daily-builds...

Full answer by stevendaniels: https://unix.stackexchange.com/a/79656 (CC BY-SA 3.0).

Need Linux help for your business? UphillTech.