Decrypting encrypted archives

by Eric
Courtesy of fravia's pages of reverse engineering
June 1999

Nice simple c code by Eric to have your own console application (win32 console) that loads a DLL (mwsetup.dll in this case), attaches a pointer to the DLL's function and calls it with your own parameters. Enjoy!

Decrypting Matlab's Encrypted *.Z files

Decrypting encoded files can be done in 2 ways:
1) Figure out the algorithm.
2) Use the function supplied by the packer itself.

The first method can be very hard and sometimes impossible, so let's stick to the easiest one...
This example will decrypt matlab 5.2 *.Z files using it own DLLs.

1st step: Obtaining the DLLs.

Browse to /windos/temp dir and delete everything.
Run matlab's setup, and switch to explorer.
Browse to /windos/temp dir and see the new files there (sometimes they'll be under a subdir).
Copy all the files to somewhere safe, as setup will delete them after aborting it.
Exit setup.
Note:
Sometimes the wanted DLLs will have different names, viewing the file will usually reveal their real name.

2nd step: Write a simple program that calls the decrypt function.

A useful tool knowing what functions a DLL can export is borland's tdump utility (ships with almost all borland compilers).
type:
tdump filename.dll > filename.txt
and view the results.
What's we're looking for is the EXPORT section - all the functions that can be imported by external applications.
In mwsetup.dll it looks like:

Exports from mwsetup.dll
30 exported name(s), 30 export addresse(s). Ordinal base is 1.
    Ordinal RVA       Name
    ------- -------- ----
    0000    000010a5 AdaMenuPatch
    0001    000010aa CheckExpDate
    0002    00001064 CreateComponentList
    0003    00001037 CreateStatusWin
    0004    00001041 DecryptArchive :-)
    0005    000010d2 GetChangeInfo
    0006    00001014 GetClusterSize
    0007    00001046 GetExpirationDate
    0008    00001127 GetHelpInfo
    0009    000010e1 GetLicenseNo
    0010    00001023 GetProductIndex
    0011    0000107d GetProductInfo
    0012    000010c8 GetVolSerno
    0013    000010d7 InstallFlexService
    0014    00001032 InstallSSIService
    0015    0000101e IntMakeIt
    0016    00001113 IsClasskit
    0017    0000100f IsDemo
    0018    000010a0 IsDongle
    0019    0000108c MakeClassKitBanner
    0020    000010f5 MakeIt
    0021    00001109 ModExe
    0022    00001082 ModV4Exe
    0023    000010be RemoveControlFromDialog
    0024    00001055 ReplaceFileText
    0025    00001104 ReplaceStringText
    0026    000010af RunningAsAdministrator
    0027    00001005 SetComponentList
    0028    000010dc SetLicenseNo
    0029    0000105a SetStatusWinText

The 5th function is called DecryptArchive , we'll want to use this function to decrypt the files for us.
Next, write a simple console application (win32 console) that loads the DLL (mwsetup.dll in this case), attaches a pointer to the DLL's function and calls it with your parameters.
It looks something like this :

//decrypt.cpp
#include<stdio.h>
#include<windows.h>

int main (int argc, char **argv)
{
if (argc != 3) {
printf("Syntax: Decrypt <infile> <outfile>");
return 1;
}
//declare function pointer
int (__cdecl *DecryptZ)(char*, char*);
//load DLL
HINSTANCE libh=LoadLibrary("mwsetup.dll");
if (libh==NULL)
{
printf("dll not found !");
return 1;
}
//assign decrypt function address to pointer
DecryptZ=(int(__cdecl *)(char*, char*)) GetProcAddress(libh,"DecryptArchive");
if (DecryptZ==NULL) {
printf("Error attaching decrypt function!");
return 1;
}
printf("Decrypting %s\n",argv[1]);
DecryptZ(argv[1],argv[2]); //decrypt the file
//unload DLL
FreeLibrary(libh);
return 0;
}

The code is so simple, it will take seconds to translate it to pascal or whatever you use.
Put the needed dlls (mwsetup.dll and lmgr325c.dll) in the same dir as decrypt.exe.
The microsoft api functions can be complicated, so i'll explain them in detail:
The following section is for programmers who are not trained in C/C++.

int (__cdecl *DecryptZ)(char*, char*);

Here I declared a function pointer of a function that looks like:
int DecryptZ (char *infile, char *outfile);
It gets 2 strings for input and returns an integer.
the __cdecl tells the compiler that the function uses standard C function call (right parameter gets pushed into the stack first).
If __cdecl fails you can try the rest (__fastcall, __stdcall ...).

HINSTANCE libh=LoadLibrary("mwsetup.dll");

Here I declared a handle for the DLL ( a handle is usually an integer).
LoadLibrary is called with the DLL name and returns a handle.

DecryptZ=(int(__cdecl *)(char*, char*)) GetProcAddress(libh,"DecryptArchive");

All i do here is get the function address from the DLL using GetProcAddress function.
It gets a handle for the DLL, and the function name.
It returns the address of that function.
A type casting is used so the function types will fit (int(__cdecl *)(char*, char*)).

After all that DecryptZ is actually a function and is called normally.

For comments and questions write me.

Eric

P.S. Please post here any amelioration / suggestion, thanks.