/masterzorag/log

OpenCL + PS3: Improvements!

2012-01-25T12:21:00.000+01:00

To anyone follow this story, here some things to see!
I'm doing Pairwise Master Keys with all 7 compute units on a PS3 via OpenCL, check out my three kernel's output:

This is always pyrit related, but it works alone (for now), porting to a pyrit plugin will take the last step of my interests.
Was January 14, 2011 when I've told: "Correctly improve that is possible, we have tools."I was right, I've wrote the fastest implementation ever!
Now I need to improve more, there's again!

blackhole: added sha1_process variants

2011-12-27T18:09:00.000+01:00

some of blackhole in action:

. removing old hashfile
. counting records in database
. listing computed pmk grepping the correct one
. exporting to a new cowpatty_hashfile
. run cowpatty attack over test libpcap capture file

cutted out cl computation, I've implemented a different vectorized sha1_process (I've two now).
results of this variant are poorly the same as first one on a 8400GS (need to check on the spus: they're the vector units).
this one needs more _private space to compute (need to stage some on _local), but I can set a larger NDRange.
thinking to write something about on Google Docs (need time).
good computation, good output.

"This is not hacking, it's crypto!"

2011-12-05T17:36:00.001+01:00

I've wrote a single program called blackhole, it:
. Stores words,
. Computes pmk against an essid,
. Stores back results into a simple sqlite database,
. Exports computed pmks to a compatty hashfile to perform the attack later with aircrack-ng or cowpatty.

And here everybody'll say: "So why you should use this? Just use pyrit!"
Pyrit is NOT optimized to run on the Cell/BE , I'm doing this to improve the speed of computing pmks with all 7 Spe in a PS3.

Just to let you know that I'm back to work over the OpenCL code now!
I'm thinking to elaborate some of my idea, writing down to let you, interested reader, understand and maybe found better solutions;
Was Feb this year when I've got some speed improvements without knowing C, how can do the job without C basis?
Months spended reading and learning a lot of stuff, sources of aircrack-ng, stuff as genpmk.c, sha1-sse2.S, pyrit, threading, the sqlite library...
This is the power of open source, I was able to realize something I've in my mind, for free!
Don't miss my next new video "blackhole's evidence".

What's up next?

2011-11-10T15:44:00.000+01:00

Busy at the moment, but let me know you out there what's going on:
bruteforge will be updated adding the 'Shifted variant' mode set, Niclas already knows what it does!
blackhole is importing, computing pmk and exporting to cowpatty fie as pyrit/airolib-ng/genpmk does!
About computing, I'm using the CPU in one single thread; about exporting, results are the same: diff tells me that: binary files are identical!
Now I've to implement the OpenCL code, my vectorized one(!); I can skip the stock pyrit cl implementation now, I've got error checking with SSE2 or quering blackhole_sqlite database also!
Time to give power to the Cell, targeting my 'too long time idle' 7 SPU!

What (da fuck) I'm doing now?!

2011-09-19T14:15:00.001+02:00

bruteforge does words, where store those? I'm writing blackhole.
blackhole is a database engine with import/export capability to/from an sqlite database, the third piece of code does pmk computation via OpenCL (I'm targeting the PS3 on all 7 SPE!), an optional fourth piece will export to a cowpatty table to externally perform the attack.
All is written in C, linking gmp, sqlite, openssl; OpenSSL? Yes, blackhole starts doing pmk hash using the CPU with SSE2, I'm using for my tests the SHA-1 SSE2 implementation (C) 2008 by Alvaro Salmador, (easy, I'm not spreading anything, same code was implemented in pyrit too...)
I've melt that code to get result checking, was a new stuff, as sqlite. Many useful thing to know, as ever.

So what? One single thread export from sqlite db, compute pmk and store back result into sqlite db doing a little more of 500 pmk/s.
Sounds good, now I'm thinking how store the pmk: a uchar[32], or better: 8 uint32, 32byte, let me better explain:
After doing the math the result is 8 uint32, 8*4byte = 32byte, by the way the hash output is obtained by reading as uchar (byte after byte) the same 32byte, so simply I'm thinking to store those 8 uint32 into 8 sqlite coloums, in this way sqlite will stores max 64byte per record using INTEGER type.
The fouth piece will then read computed INTEGER results with atoi(), stores into a 32byte before get written to a cowpatty table.
Why? uchar[32] can be full of NULL and friend... How to store an uchar[32] into sqlite coloumn using BLOB type?
Storing as hexadecimal string will be the same (64byte per record), but every uchar must be hexcoded importing and then hexdecoded while exported...
Storing as binary? New stuff to learn, anyone can point examples?

bruteforge 0.6r18, here comes a new rule!

2011-09-12T13:38:00.000+02:00

my friend don't want more than N couple of same char in word, ex:
Valid:
02AE7AABF97
2BEFFA29151
Not valid:
00EE7ABBF97
we can now append -c 1 to let bruteforge compose with max 1 couple.
grab the latest binary here.

1UP

2011-06-22T23:32:00.002+02:00

Base system is less than 800M, I've got one more spu.
In meantime I've wrote another tool to interface with bruteforge, more to come.

missing ln -s

2011-05-05T20:06:00.002+02:00

upgrading to IBM OpenCL Development Kit for Linux on Power 0.3, restarting F12 troubles...
Specs says tested on:
Red Hat Enterprise Linux 5.5, 5.6 and 6.0;
SUSE® Linux Enterprise Server 11 SP 1;
I test Fedora12, ppc ends at 12.

Install logs tells you that all went fine, but xlcl start script fails!
To update succesfully invoke this missing parts:
cd /opt/ibmcmp/xlc/opencl/0.3/etc/
ln -s xlc.cfg.rhel.gcc444 xlc.cfg.rhel12.gcc444
ln -s /usr/lib/libIBMOpenCL.so /usr/lib/libOpenCL.so

A voice in a loop: "... upgrade, patch your firmware, free another spe, unlock all... "!

pmk, dissected

2011-02-17T13:30:00.001+01:00

As seen, pyrit opencl does about 1200... i've wroted a kernel that grab data as seen here to get processed by kernel as uint4 vectors, theorically this should does x4, and it does!
What's the big news? Output data are verified with tested vectors against 8192 sha1 rounds, or better;
First 20 of 32 byte of pmk are good with mine implementation, I've now enough C/OpenCL basis to target the differents cl address spaces provided, made use of _local and async_worg_group_copy and maybe redesign the algo to do 20+12 in the same time!
I've gotta go now, this fucking work...

pause, read, start

2011-01-14T16:57:00.002+01:00

Open the door to gameos means that some developers will start to know the Cell/BE and its architecture, we all know its potentials.
Of curse, we are seeking the return of linux, with one more spu for us.
Now that I've gain some C basis and finally nvidia-toolkit 3.2 can compile fine (the stock pyrit opencl kernel back to run on my cl_developing mobile 8400gs, I can't develop on PS3 only everytime) I'm back focusing to improve performance of OpenCL implementation of PMK hash on the Cell/BE, existing implementation computes 1200 pmk/s;
Correctly improve that is possible, we have tools.

Some stuff
WPA password cracking
fedora12-ps3 setup
Cell Architecture Explained
Hints from IBM forum
CUDA/OpenCL differences
OpenCL implementation of Sha1 hashing algo

...bruteforge has been C revamped!

2010-12-10T01:45:00.012+01:00

I've completed the first C version, time to release;
All features of bruteforge 0.56 are implemented, with less skips, I've added full support to Extended ASCII Codes; (you can use your favourite 'èòàìù...')
bruteforge 0.6x, linux/elf386 here
documentation will be posted here

I was focused on a new function, the 'Filter Mode', from now you can pass your favourite wordlist to trim down over your favourite criteria mix...
So, not only generate wordlist in conditional way, but we can start filtering huge wordlists! (...this project seems moving like a squid...)

A diff about performance versions, I've done a simpe test, generating 100.000.000 words in a lowercasealpha, starting from 'a':
0.56 generates those in real 18m25.013s, doing 90.496 words/sec;
0.612 do the same job in real 0m24.451s, doing 4.089.812 words/sec;
In this test performance gain is 45.19x, I have no reason to maintain the python version,
anyway, script is open...

...quietly...

2010-11-15T19:52:00.002+01:00

First part is over, with something added like full support to Extended ASCII Codes, less skips and more speed, (almost x16 faster of python code when prints to terminal), code is running;
user have to define a 'template' text file like previously for advanced, program reads first 4 lines as:
charset
maxchars
starting word
ending word
it summarizes criteria to understand those combinations that will be skipped;
you can have multiple 'settingfiles' to run bruteforge over, it prints out generated words;
I've to made -lp -n -v -q customizables from commandline, add the fixed mode and more...
here a snapshot, follow a short demo from the cold North! Thanks!

Meanwhile, at lower levels...

2010-10-21T14:26:00.002+02:00

Writing OpenCL code needs some C knowledges, in time I've thinked about PMK algo, Cell/BE, python, and C...
How can C code run faster, pointers, arrays and structs, ints, memory allocation/alignment and more...
Time to see what can be done in C, so how easy can be porting mine bruteforge python script other than code written by others?
I've wrote this time ago, code and algorithms are in my mind! I want to see the diffs, and learning C for my needs, of course.
As expected, C uses less memory (68kb over 1990Kb) and runs moOore faster, it generates 10.000.000 words in 3.384s, the python version computes the same numbers of words in 1m46.062s, so I've got a x31 speedup in front of 1/31 in memory!
My new code isn't well optimized, I'll spend some more time with it, but actual results are impressive; so I'm working to complete the C port of bruteforge with renewed logics; all will be in function of the -so called- 'advanced mode', so users can sets a custom charset (including special and spaces characters) and assign a maximum number of use for every char to compose a word!
Patterns eventually excluded like 'abc', 'stuv'... due to consecutiveness will be ever skipped by setting pattern's length, this also for consecutive repetition of same char like 'aaa', 'rrrr'... but I'm re-evaluating others skips like inversed consecutives that in case can skip too often probable words!
When completed, maybe I'll learned C better (a little bit) to expand functionalities, like transform words to compute some kind of hash...
Mine PS3 don't play games, but is hungry of ints!

"Cops can't stop me!", but I need fuel!

2010-05-20T14:10:00.002+02:00

Can't release anything yet, I've tested, but sometimes got wrong results!
Too busy at the moment, I'm not a real programmer, I've to spend some time working now...
I'm not surrendered, but working changes needs study.
By the way, IBM has released an upgrade to OpenCL, with big changes about device control and more, but I haven't test yet.
I've to end up with a simple host code to run the cpyrit_opencl alone and setup timing to see if any changes are good!

Mine PS3 hits 7000 PMKs/s!

2010-04-30T22:01:00.006+02:00

Small changes in the code about feeding and organizing the WorkSizes, but there are other improvements we can do about; personally I have two of those, one is make use of uint4 power, the other regards the use of couples of WorkItem per WorkGroup...
I'm passed from 1200 to 7000 PMKs without changes in the kernel!
There is now a place in the performance-graph? Eh eh

pyrit: OpenCL + PS3

2010-03-19T22:41:00.004+01:00

Less than one month to see the results, setting up gentoo with a broken xlcl compiler, than wiping all for F12, now OpenCL are running as core in pyrit, and computing.
Thanks to joker that points me in a right direction, now lukas can update again his performance graph!

Now what?! I've got a PS3

2010-02-25T13:45:00.001+01:00

With bruteforge 0.52 you can do any wordlist!
In a spare time will try to approaching the ppc world, to setup a gentoo system on my 60gb PS3
First milestone will be run python 2.6 and how bruteforge performs on the cell, than interfacing pyrit
I've read here about the OpenCL on cell, I'll try.

0.52 Stable, Fast and Colorful

2010-02-17T16:56:00.009+01:00

Gots 'acqua' in 5.334s with 511443 attempts = 95883 words/s, main faster;
Cosmetics changes in debug/verbose mode to be more read-friendly;
Fixed possible fixed mode related exceptions/issues.

Here comes the advanced mode!

2010-02-08T19:50:00.008+01:00

Added advanced mode
You can set your custom charset + max same per-char use times
bruteforge expect for a file called 'advanced' that contains only 2 lines:
first for charset, second for max same per-char use times, that obiouvsly must be filled by digits, one example:
cat advanced
abc?ef
213111
Import adding -c 9, you have set composing with max 2 a, max 1 b, max 3 c, only 1 ? and so on, that's so... how can tell... ummm
This mode locks also the charset order!
Added to the main logic, advanced overrides the -s switch, but bruteforge ever skips based on -p [ when importing custom patterns with -i, when charset can compose patterns ]
Remember that fixing with this mode gives you a fixed number of combinations, add -d to understand the concept
Set your options, and generate!

Now at 0.45, more options!

2010-02-04T13:26:00.011+01:00

Added special charset selection;
Added custom charset selection via user input after -c 7;
Added the '##fixed##' implementation, changing only '#' in word;
Not only will do '##fixed##', but also '##fi#ed##', and '##f#x#d##', and what you want, must use '#'.
You can fix 3 'a' and set -s 3 to not use more 'a' when changing the rest of the word, es. '#a#a#a########' change # in all putting no more 'a'.
When fixing, resulting word lenght not increase!
To show a summary of your criteria, add -d -n 1, so program stops in verbose/debug mode after first attempt.

user# wget http://mz03.netfirms.com/py/bruteforge
user# chmod +x ./bruteforge
user# ./bruteforge -h
for help, without arguments shows a short description and rules.

Crunch some bytes,
I'll made a new video in a next time!

Power up!

2010-01-22T00:00:00.006+01:00

Fourth version, from scratch, again.

The code is now splitted into ten small functions that can be also imported from the interpreter, the main script got 2 new features:
Unique char composing via the -s 1 switch, and a very simple Custom patterns importing options.
Importing works reading one patterns for line from a custom file and appends ALL, so put everything you want to exclude.

My first Python version gots the word 'acqua' at 512099 in 20.328s, this version scores 511443 in 6.898s!
More than three times faster with more skips; if you haven't see any of my video, this is mine testing algorithm to compare versions, ever with 26 char in set, starting from 'a'.

So, switching to Python, brings to mine code ability to run everywhere, from linux to windows, also on an iphone; it's a simple script, you need only Python => 2.6
If interested, enter a terminal, and:
# wget http://mz03.netfirms.com/py/bruteforge
# bruteforge -h
Run without arguments for a small descriptions about conditions and rules, or put an eye at the code at same url

bruteforge 0.10_rc1, codename 512512

2009-12-02T13:26:00.008+01:00

Almost rewritten to go faster, I've also changed his name into 'bruteforge', becouse now is smart enough to choose which char must be changed first and much more. Now it skip fast forward without restart the changing char to the first char if there are just n of that char.
Keep in mind that bruteforge doesn't break or crack nothing! Is a wordlist generator, so only prints out words!
Words can be acqua, acqua can be a password, like aababbcc can be a password, so aababbcc is a word.
Now, a simple bruteforce compute all combinations to reach the word acqua, 521743; bruteforge reach acqua in 512512.
There are rules that permits skips, and decrease the number of words printed out, useful when the target is having a less-weight of words to compute in your way.
Speed is not a problem, prints out about 13141 w/s on a Core2 Duo T7250; hosted on mz03.netfirms.com, ELF 32-bit binary

UPDATE:This was the old, no more mantained python version! Now bruteforge is entirely coded in C, check this one or here at right the blog archive to know more!

bruteforce is evolving

2009-10-26T14:28:00.007+01:00

Added alphabet selection from lowercase alpha, numeric only, lowercase alpha + numeric, esadecimal, the 32 character set and custom. Now the main loop analyze and change the selected char(s) until it founds a valid one and prints it out, then return the loop.
Added a new algorithm based on some discussion on forums about sequences of four numbers in group of six, but the third algorithm is the good stuff. Looks like 'if there are n equal chars, don't try again that char, until one of those change', and n is a new customizable option!

p4-clockmod: 'too long latency for HW'

2009-10-09T13:28:00.004+02:00

if you get the 'too long latency for HW' message and it fallback into performance governor when you want to switch into ondemand or others this helps you. developers has intenctionally disabled this, but if you can compile kernel by yourself just change the value of latency 10000001 to 1000000 in arch/x86/kernel/cpu/cpufreq/p4-clockmod.c and recompile. here the snip

- policy->cpuinfo.transition_latency = 1000000; /* assumed */
+
+ /* the transition latency is set to be 1 higher than the maximum
+ * transition latency of the ondemand governor */
+ policy->cpuinfo.transition_latency = 10000001;

lowercharset proof of concept

2009-10-06T12:00:00.004+02:00

A normal bruteforce like crunch compute all 521743 combinations to found the word 'acqua', mine uses one algorithm to skip unprobable combinations and do the job with 502118. Based on bruteforce 0.01 alpha written by Tony Bhimani.