Password Cracking Tips: A CrackTheCon Roundup

“It is common sense to take a method and try it. If it fails, admit it frankly and try another. But above all, try something.”― Franklin D. Roosevelt

CrackTheCon, a password cracking contest run by CynoSurePrime, just finished. I competed as a Street team and I was really impressed. This was a well run contest, and I felt was very friendly to new and experienced password crackers alike. At least from a player's perspective, the infrastructure was rock solid, there was a great variety of challenges, and the difficulty level had a good gradient. Thanks to everyone who helped put this contest together!

My computer setup for this challenge was limited. I performed all my cracking on one laptop with no GPU support. You read that right, I was rolling old school with a pure CPU cracking session. Because of that, my primary password cracking program was John the Ripper, which has a ton of features that I prefer when I can't just let HashCat burn through some GPUs. While my operating system was Windows, I used Windows Subsystem for Linux to run John the Ripper and perform analysis on the cracked passwords. You can read about how to configure JtR and WSL here.

This lead to a modest performance of 9th place:

GPUs are nice, and this certainly shows it! If you have some GPUs available I highly recommend using them along with HashCat. As some backstory, I still have my main password cracker set up to run medical security capture the flag events, and I was too lazy to get it reconfigured for this contest.

Therefore you should probably take everything I say with a healthy degree of skepticism. Based on the chat on Discord afterwards though I realized there's a few password cracking tips that might be helpful to share. One important point I want to stress is that anyone can make use of these tips. You don't need a fancy GPU hash-cracking monster to crack passwords. In fact, most of all my attacks were "semi-automated" with very little manual analysis of the cracked passwords. So you can apply all of these techniques yourself regardless of your past level of experience.

Tip #1: Make sure your John the Ripper build is based off Bleeding-Jumbo, and update it regularly!

Even if you normally use Hashcat, JtR is a very powerful password cracking tool that has a lot of nice "research friendly" features. This makes it an extremely useful tool to have in your toolbox. As a general rule of thumb, if I'm cracking passwords with a GPU I use Hashcat. If I'm leveraging my CPU I use JtR. The key to JtR is you need to use the Bleeding-Jumbo version of it. The "main" branch prioritizes compatibility with different architectures, but the Bleeding-Jumbo branch goes all in on features. As an example, over the last couple of months they added "duplication detection" to the early portions of a password cracking session (to help with slow or salted hashes), and performed a complete rework of the included rulesets. What I do is use Git to clone JtR from its github repo at: https://github.com/openwall/john, check out the "bleeding-jumbo" branch, and then periodically pull down updates and rebuild it, (roughly once a month). This makes a huge difference!

As to the deeper question of "Why would you ever crack passwords on a CPU and not GPU", that gets more complicated... At  a high level, I do a lot of password cracking research from a researcher and hobbyist viewpoint, so a CPU based approach makes it easier to tailor attacks. The real reason though is I don't own a massive cracking setup. From a training perspective, this means even if you only have a Raspberry Pi, you can pretty much recreate all of the techniques described here. That being said, sometimes the features of John the Ripper still outweigh the speed that a GPU provides, and at the very least it's a good tool to run on parallel on a VM or research computer while running longer GPU sessions with Hashcat on your main cracking box.

Tip #2: Use the '--loopback' option to leverage previously cracked passwords in your rules

The clickbait title was going to be: "This one simple trick is like a cheat code for password cracking competitions!" That's not much of an exaggeration. Full disclaimer, this technique probably resulted in around 50% of my successful password cracks in the CrackTheCon competition. I'd periodically wander over to my laptop, and feel l33t by hitting the enter key to kick off a new loopback session. So if you only follow one of these tips, this is the one to pay attention to.

As to the actual technique itself, John the Ripper's '--loopback' option tells JtR to use previously cracked passwords as a wordlist in a cracking session. Hashcat also supports loopback attacks as well. There's a million different names for this approach, which by itself should tell you how powerful it can be. You can further optimize this attack by specifying a different .pot file from your main one such as '--loopback=Challenge1.pot'. This can be helpful if you are keeping your .pot files separate for different challenges, (I don't actually do this, but some people might). Once you are using --loopback to generate your base words, you can then apply mangling rules to them like a normal wordlist. Aka by also adding: '--rules=hashcat'. 

What this means was that my typical cracking session would start by running fairly basic attacks to generate an initial set of cracked passwords. For example, I'd run '--incremental' to brute force shorter passwords. I'd run a quick cracking session using the wordlist 'dic-0294' and hashcat + single rules to get slightly more complicated passwords. And I'd run a quick PCFG guessing session as well. After that initial set of passwords were cracked, loopback became one of my main attacks. And as you can see from the results, it was very effective.

Now in the real world, loopback attacks while still powerful, aren't nearly as game braking as it is in a password cracking competition. Real users don't exclusively pick their passwords from a list of fungi names. But even then, loopback can still be useful to help augment your other cracking sessions. 

Tip #3: John the Ripper supports dynamic hash formants on the command line. No need to modify a kernel or look though lots of documentation!

This being a CynoSurePrime cracking competition, there was bound to be weird hashtypes to crack. This problem also pops up time in real life cracking situations where some vendor decides to roll their own password hashing function. This can be a challenge since writing your own Hashcat kernel is not a lot of fun. That's one area where JtR really shines is with their extensive "Dynamic" hash type support. You can see the main formats that JtR supports by specifying '--list=formats' on the command line. That only shows the "mainstream" formats though. If you really want to see all the various formats supported by "Dynamic" mode you can specify '--list=subformats' on the command line.

There's a lot of them included, and sometimes even that is a pain to look through and remember. One feature of JtR most people don't know about though is you can specify the hash details directly on the command line. For example, Challenge2 of the CtC contest was five rounds of MD5. To crack this with John the Ripper I simply needed to specify the following command:

./john '--format=dynamic=md5(md5(md5(md5(md5($p)))))'

The single quote around format is important so that your shell command doesn't misinterpret the parenthesis (). Basically though, you can specify the hash type, and how the password ($p) is applied, along with any salt ($s) as well. Dynamic mode supports multiple types of hash primitives, so for example, with Challenge4 which was a sha256 of a md5 hash I was able to use the following command:

./john '--format=dynamic=sha256(md5($p))'

Long story short, if you ever find yourself needing to crack a weird hash type, don't forget about John the Ripper's Dynamic formats. 

Tip #4: Leverage MDXFind to identify unknown hash types

I'll be up-front. I did not follow this tip and I'm really kicking myself over it. To guess the hash types, I relied on trying the suggestions provided by John the Ripper, and when that failed, I manually tried different hashing functions using the command line dynamic mode (Tip #3). Don't be like me. If you are dealing with an unknown hash, the tool you want to use is MDXFind. You can obtain it here: https://www.techsolvency.com/pub/bin/mdxfind/. If I had followed this advice, I probably would have ranked higher as I never figured out that Challenge #5 was:

 --format=dynamic=sha256(sha1($p))

 To get MDXFind running on an Ubuntu image running on Windows Subsystem for Linux (WSL2):

1. Download mdxfind.1.116.bin
2. sudo apt-get install libjudy-dev
3. sudo apt-get install libmhash-dev
4. sudo apt-get install librhash-dev

Here is an example leveraging MDXFind to identify the hash type for Challenge #5. The passwords look like SHA256, so the command I'd start with would be:

./mdxfind.1.116.bin -h 'SHA256' -f Challenge5.txt wordlist.txt

• -h 'SHA256': is the base hash type to use
• -f Challenge5.txt: is the hashlist
• wordlist.txt: is the wordlist 

And the results...

It quickly identified SHA256(SHA1($p)) in 4 seconds... Yeah that would have been nice to use.

Tip #5: John the Ripper supports mangling rules on the command line

In password cracking competitions one of the keys is to try and identify mangling techniques and create rulesets to target them. Now, I'll admit that for this competition, I mostly relied on the included rulesets (Tip #6), and using the PCFG Toolset to autodetect and create rulesets (Tip #7). A hands on approach is more effective though, but it quickly becomes annoying to have to constantly open up your ruleset file to modify it. This may sound like a minor nitpick, but your analysis time is valuable. One hidden feature John the Ripper supports is creating rulesets right on the command line. This is a huge timesaver, and in my opinion one of the killer features of John the Ripper. For example, let's say you want to duplicate a word and then add two digits to the end of it. Your JtR command might look like:

./john --wordlist=somelist.txt '--rules=:d$[0-9]$[0-]' hashlist

Key points:

• You need to include --rules in single quotes. Aka '--rules...'
• Your rule needs to start with ':' which is JtR's "no-op"
• You can include multiple rules separated by a ';'. For example: '--rules=:d$[0-9]$[0-];:$[a-z]' 

This is also very useful to test the output of your rules. To do this you can feed in an single word via stdin, and then you can apply rules to it using JtR's --pipe command. So for example:

echo test | ./john --stdout --pipe '--rules=:d$[0-9]$[0-9]'

It may seem weird, but this is one of those tricks that makes me smile every time I use it.

Tip #6: Making use John the Ripper's mangling rulesets

John the Ripper includes a ton of wordlist mangling rules. Given this contest was run by CynoSurePrime, I figured there would be heavy hashcat users on the hash creation side, so I primarily used the 'hashcat' ruleset. Aka:

./john --wordlist=somelist.txt --rules=hashcat

This runs through some of the main rules included in Hashcat such as:

[List.Rules:hashcat]

.include [List.Rules:best64]

.include [List.Rules:d3ad0ne]

.include [List.Rules:dive]

.include [List.Rules:InsidePro]

.include [List.Rules:T0XlC]

.include [List.Rules:rockyou-30000]

.include [List.Rules:specific]

Other useful rulesets (though not as useful for this particular competition)

• --rules=phrase: Great for attacking passphrases
• --rules=l33t: Good for attacking l33tsp33k passwords
• --rules=ShiftToggle: Good for attacking weird capitalization
• --rules=by-score: A good set of rules to use for fast hashes
• --rules=by-rate: A good set of rules to use for slower hashes

Tip #7: Using the Pretty Cool Fuzzy Guesser (PCFG) Toolset

Of course I was going to mention the PCFG toolset! I just recently released version 4.3 and it has a ton of expanded documentation, plus better support for cracking Russian passwords. You can get it here:

https://github.com/lakiw/pcfg_cracker

The default PCFG ruleset is usually decent, but not great when it comes to password cracking competitions. This is because context passwords don't resemble RockYou passwords which the default ruleset was trained on. The real value is using the PCFG trainer to learn new rules and create new wordlists based on cracked passwords. The trainer does a lot of cool stuff in the backend such as multiword detection, keyboard walk identification, and other mangling rule generation. It can also be fairly effective even if you only have a couple of hundred cracked passwords.

To train a PCFG ruleset:

1. You first need to create the training list. Adding support for JtR pot files has been on my todo list forever, but currently you need to strip the hash information off of your cracked passwords. For example, if I wanted to create a training list for Challenge #2 which was five rounds of md5 I ran: "cat john.pot | grep 'md5(md5(md5(md5(' | awk -F':' '{print$2$3$4$5$6}' > plains_2.txt" Yes this is a horribly inefficient way to do this, but it printed out all of the hashes, then only printed ones with the correct hashtype, then stripped off the hash, and then saved the results to plains_2.txt.
2. Run the PCFG trainer on the set. For example: "python3 trainer.py -r Challenge2 -t plains_2.txt

Once you have the training set you can do a couple of things:

1. You can run a PCFG attack against the challenge using the new ruleset. For this I recommend disabling Markov generation using the --skip_brute option. For example: python3 pcfg_guesser.py --skip_brute -r Challenge2 | ../JohnTheRipper/run/john --stdin '--format=dynamic=md5(md5(md5(md5(md5($p)))))' Challenge2.txt
2. Another good option is to use princeling to generate a wordlist optimized for PRINCE attacks. You can also use this as a normal wordlist as well. For example, this will create a 50k word dictionary: python3 prince_ling.py -r Challenge2 --size 50000 -o new_wordlist.txt
3. You can manually go through the generate rules file to identify mangling rules. A good option to open up is: Rules/<RULENAME>/Grammar/grammar.txt

Summing all of this up, 99% of my cracking sessions for this contest were:

• Identify the correct hashtype
• Run a default attack against it using the dict0294 wordlist and the hashcat rules
• At the same time run a JTR bruteforce Incremental attack "./john --incremental=All". That's the nice thing about CPU cracking. I have enough cores I can run around three sessions at the same time on my laptop before things get really slow.
• Run a couple of loopback attacks using the hashcat rules
• Train a PCFG ruleset and run a PCFG cracking session until it gets to around 95% coverage. You can see the coverage by hitting enter while running it.
• Run a couple more loopback attacks
• Re-Train the PCFG ruleset
• Create a wordlist using the PCFG prince_ling
• Run a PRINCE cracking session using the wordlist and JtR
• Run a normal cracking session using the prince wordlist and the hashcat ruleset
• Re-Train the PCFG ruleset and run a PCFG cracking session
• Repeat. Maybe run a longer incremental session, or try another input dictionary.

Following these steps, you too can get 9th place in a password cracking competition!