Project Description

In order to fulfill the programmatic content of one of the course subjects a distributed application had to be created from scratch.
The purpose was to split a certain task or process across several computers, proving the concept of distributed computing.

This project was done with a collaboration of a colleague, that worked along side me from the software / hardware design & concept to the final presentation.

The Idea

We came across the idea of creating a Master / Slave application in C++ that could find MD5 hash keys on a given string, using brute force techniques to accomplish this.

Hardware

In order to accommodate hardware & software requirements, we used:

• 1 Regular PC PSU
• 1 Raspberry PI 2 (Master)
• 3 Raspberry PI 3 (Slaves)
• 1 Switch 100/1000

To accommodate all the required equipment a structure in wood was built with 3 distinct levels, Power, Networking & Processing, this way we could easily transport the unit around.

Power Supply (PSU)

We consider using a universal powered USB HUB in order to provide power but we have found out that each Raspberry uses around 2 to 3 Amps of voltage to operate properly, to power 4 PI's in proper conditions the HUB was not going to cut it so we used a regular power supply unit from a standard desktop pc to power the PI's, the main connector of the PSU was modded in order to get the voltage required to power all the machines, and it could also be easily escalated.

Operating Systems

Application was fully developed in C++, naturally a linux OS was the choice, in this case we went for Ubuntu version 16, with GUI in the Master PI and console envr. in the Slaves.

Modules

The mechanism architecture is based in a Master / Slave (please see project report & images below) application.

The Master Module - This application is responsible for finding & auth any slaves on the network prior to the process, it queries the network subnet and finds any "peers" waiting for a connection. Next the application log's in to the slave via socket's and provides several commands. The master is then responsible to coordinate processing task between the slaves.

The Slave Module -  This application holds all the algorithms required to process and store MD5 hash's. Its responsible to process a certain sequence  and look up for possible matches and provide information in real time about the time that the process takes and percentage completed of the sequence.

Modules architecture

In order to accomplish parallel processing tasks such as communication, processing & management (Master), threads were naturally used.
Subdiving each process in their own class and provide common methods to interlink them (please see project report & images below).

Sequence Generator Algorithm

Easily the "Sequence Generator Algorithm" was the bit where we have spent more dev. time, essentially because of its complexity in logic terms due to the fact that we could not use a recursive method to generate sequences, as this would easily eat any available RAM very quickly.

In order to understand the nature of problem I strongly suggest having a quick read to the project report, but essentially the problem resided in the now simple but not so obvious way to compute the source string to run all the possible dictionary entries.

In the end we have managed to come up with a simple and effective solution that can easily be interrupted and restarted as it finds the starting point of the sequence by itself and keeps on incrementing it...

Here is the source code:

Final Notes

This was without any doubt a very ambitious project that took quite a long time to develop, but in the end the effort paid off, we he could see the magic happening and the planning and hours spent on the design and concept gaining life in front of us, that by itself in my humble opinion is any programmer holy grail.

Sources

I'm here making available to free download the sources (Master and Slave Apps) and reports of the project, feel free to use & modify the code, i simply ask for a referral to this page if you do.

Project Information

Categories: Application Development

Video Url: https://www.youtube.com/watch?v=EKb6fXhunes

Project Timeline

Start: Fev, 2016

End: Mar, 2016