CeWL is a Custom Word List generator which spiders a given site to create a word list of all words it finds on that site. It can also grab email addresses and usernames found in the HTML and in some document types including Office and PDF.
Useful for targeted penetration testing which involves brute force password cracking.
We first wrote about CeWL way back in 2009 not long after it first came out – it’s been updated plenty since then and is now at version 5.1.
There are also a bunch of other similar tools out there (some older some newer):
– Crunch – Password Cracking Wordlist Generator
– The Associative Word List Generator (AWLG) – Create Related Wordlists
– Wyd – Automated Password Profiling Tool
– CUPP – Common User Passwords Profiler – Automated Password Profiling Tool
– RSMangler – Keyword Based Wordlist Generator For Bruteforcing
If you combine the wordlists from the above tools with the commonly found standard password cracking wordlists, you should have a pretty comprehensive, targeted set of lists for bruteforcing with something like John the Ripper, thc-hydra or hashcat.
cewl [OPTION] ... URL
--depth x, -d x
The depth to spider to, default 2
The minimum word length, this strips out all words under the specified length, default 3
By default, the spider will only visit the site specified. With this option it will also visit external sites
--write, -w file
Write the ouput to the file rather than to stdout
--ua, -u user-agent
Change the user agent
Verbose, show debug and extra output
Don't output the wordlist
--meta, -a file
Include meta data, optional output file
--email, -e file
Include email addresses, optional output file
Filename for metadata output
Filename for email output
The directory used used by exiftool when parsing files, the default is /tmp
Show the count for each of the words found
Digest or basic
Proxy port, default 8080
Username for proxy, if required
Password for proxy, if required
The site to spider.
You can download CeWL v5.1 here:
Or read more here.