Comprehensive NWT Comparison Project (calling all technically skilled members) (page 11)

Simon

Also, it could be UTF-16 so a variable number of pairs per character (i.e. some have 1 hex pair, some 2, some 3 ...)

Perhaps I'll try to find an encoding/decoding forum to post it on. You are right, it really looks like an encoding and not an encryption. I've tried a lot of different encodings, in SQLite and outside (.NET and java), attempting to convert the hex string into characters. And like I said, I've even tried to see if endian-ness has anything to do with it. SQLite has encoding support for utf-8 and utf-16. So one of the first things I tried was to cast the binary data to a text field with different encodings.

(I highlighted the SQL query in red to make it easier on your eyes to parse through the prompt)

As you can see, nothing came out.

MMM

MeanMrMustard

OK, 2:39 AM - I thought I would give everyone an update on what I found. After trying various encodings, I decided to re-visit the entire .ZIP package that castthefirststone provided to me. The tool that he used to unpack the android APK file, as it turns out, can take the compiled java and turn it into a decompiled *.smali file. Now, I have decompiled java in the past, and for the most part, you can bring *.class files back to somewhat understandable java code. However, as it turns out, *.smali files are about as far back toward java as you can get because a lot of information about the original code is lost during the compile process. Yes, I did find some people out there trying to do a further decompile, but that process is in its infancy.

Anyway, the *.smali files are very much like a java assembly code. They are not as readable as java... but if you want to, you can find what you need. So what I did was seach the entire JW app for the occurance of the string "bible_verse". My goal was to find a SQL query and trace back to as far as I could until I found how the DB data is interpreted. The search did locate many occurances. I chose a simple one. You can see in the assembly-like language the SQL query is created and the verseIndex is appended to the end of the SQL query as a string, using a StringBuilder. Then the ";" is added. The final highlighted line means that the StringBuilder.toString() method is called and the result is stored in a variable "v3"...

"v3" holds the SQL query string. That string gets submitted to the rawQuery() function, part of the SQLite library. It returns a Cursor. The first record is peeled off the top, finally the field is selected, and the getString() is called, followed by valueOf(). This value is placed in variable "v2" and then passed to.... drum roll!!!

A class called PublicationDef.Deobfuscator...

A quick peek at that class shows that the process looks to be an AES decryption (the AES cypher is not shown in the following shot) followed by an unzip. So the data we see in the content field of the WT database is most likely a compressed text, followed by an AES encryption. And to get it back we would have to do the opposite (as shown in the picture, AES followed by an inflate). Simon, that is why you don't see the ZIP header there.

Now on to a little sleep... I'll have to decide whether or not this is worth pursuing later...

MMM

Apognophos

Ha, wow. I was thinking before of suggesting trying to look for the app's function for deciphering the text, but I didn't know if it was reasonable to suggest REing the app (I don't know much about Java), so I decided to just wait quietly on the sidelines. It's strange if they really zipped each verse individually, since as Simon said, that's probably not even saving any space.

I assume the encryption key is somewhere in the app package?

MeanMrMustard

Apognophos,

Yes, the key is 20 bytes - its in there, but you have to parse through assembly to get it. Perhaps I'll take the hex for one of the scriptures, convert it into the byte array, then try to AES decrypt with the key ... then try to unzip... good lord.

You are right the ZIP isn't buying them much space, but it is another layer in their obfuscation - even though I have no idea why they would want to obfuscate that. Just odd.

MMM

Apognophos

I wouldn't be surprised if the format is the same one that was used for the WT Library app for Windows. Some years ago I briefly investigated whether I could read the literature text out of the Society's files with a self-made program, but I gave up when I saw how obfuscated it was. Not being good at reverse-engineering, I didn't get as far as you have with the JW Library files, so I can't say if it's the same method.

But if it is, it makes a bit more sense because files are easier to get to on a desktop app and I don't think the Society wanted people "ripping" their literature and then printing it online, or making competing Library apps.

This seems somewhat futile, as I understand that there are actually a number of unauthorized mobile apps for JWs out there. Does anyone know more about this? I was shown one that displays the daily text, but you can also tap on the scripture and Watchtower citations and the linked text from the NWT or WT actually pops up! If this isn't an app from the Society, it implies that someone is already ripping Watch Tower content successfully. I just learned about this yesterday, and don't have a smart phone to check it out myself, so I could be wrong.

stapler99

In the past the Society has made a big thing of using one English word to translate each Greek or Hebrew word in the NWT. I remember hearing that "soul" was one of these words, but apparently this has changed in the new version. It might be possible to see if there are any others by checking where one word in the old version has been replaced with several different words in the new version.

MeanMrMustard

Apognophos,

If that is what the WT is thinking, then after I figure it out (if, in fact I do figure it out), I'll post sample code to use their format both in java and .NET :)

But I think it is utterly stupid, since they have all the content online (just not the NWTx2 yet). When its online, you can link to it... you can just download it whenever you want. The content is there for anyone. It took me 3 hours to create an app to peel off the entire bible from their website, all parsed into separate records. Seems silly to obfuscate it, then just put it out there for anyone to link to.

MMM

MeanMrMustard

@stapler99: when this is all done we should know each and every difference between the two versions.

MMM

Apognophos

MMM: I agree it's odd that they would try to protect something that could simply be copy-pasted into Word, and that was always the case with the desktop WT Library, but I think they still considered there to be a dividing line between being able to copy-paste individual works, and getting the whole kit and kaboodle directly saved into another format. Who knows, the text obfuscation could have been the decision of a lone brother with a funny idea in his head. I think the original WT Library app was written by one person at Bethel in his spare time.

stapler99: Yes, that's a good idea for a category. It's correct that "soul" is now translated variously according to 'what makes the most sense', e.g. "person", "life", etc.

Comprehensive NWT Comparison Project (calling all technically skilled members)

Share this