Short message compression

Community General Discussion
1

I´ve writen a programm for a Wemos S2 mini to to send messages to a friends which use the same setup to read it. The message are written by hand and transfered after they are encrypted into a text string. Like a prepaid code for a smartphone. The messages can be shared any way we like, and the devices always stays offline, no Bluetooth and so on. But because it is required to enter the encrypted message via hand to read it, it should be as short as possible. So I have written an algorithm to compress pure and relatively short strings of text as good as possible. Here is how it looks like:

trackless-text-compression
trackless-text-compression1920×1080 177 KB

In general, it works, and onto what you write it compress the text quite good. It uses the 80 different characters I use (a-z, A-Z, 0-9, plus special characters) to shortcut most common letter combinations, numbers, and it has 2000 common words it can shortcut as well. The message length should be around 10-80 characters. So like as short SMS. Of course the entire code can be fixed and improved, and the fun thing is anybody can build, just need some microelectronic parts like the OLED, and an Arduino or Wemos. So if somebody like to play around with encryption, and text compression, let me know.
We want to make a real application out of that called TRACKLESS with as mentioned the biggest advantages of having a 100% corruption safe messaging system.

What do you think about, interested to try it? Found some left over OLED screen for a test build?

Best wishes
Markus

2

Unlikely. Post the code and someone will find a way around that.

3

I think it's not THAT important to compress text messages. :stuck_out_tongue:

Let's say a page of text is about 5000 characters (5k).

One second of CD audio is 176K.
A high-quality MP3 is about 1/5th of that (about 35k).
One second of DVD video (which is MPEG-2 compressed) is about 750k.

...I'm not an encryption expert but it helps if you are using an unknown algorithm. And I think it's more secure if you expand the data, sending extraneous useless information or obfuscating the message by hiding in an image, audio, or video file, etc.

One of the techniques used for decoding decrypted messages is finding patterns and commonly used words & letters which is "similar" to compression.

On the other hand, compression itself is a kind of encryption because it has to be decompressed and that's very difficult if you don't know what compression was used,

4

Hello DVDdoug,
yes it is important, I can explain you why.
If you use software to compress or and encrypt data somehow to send it over the internet, it means you using a smartphone, or a computer, something connected to the internet. That's already where the things getting complex and non-controllable.
If somebody wants by force or randomly hack your messages, there are multiple weak spots in a system too complex to be 100% sure you have all under control. How sure you can be there's no corrupted CPU in your device, there are rumors secret services did or tried to implement such, did you ever left your computer unattended in your life, get some keylogger, malware, or used a safe wireless network which wasn't as safe as you thought? It the software you are using is 100% safe or did somebody knock at their door and forced them to implement backdoors? Or hackers just found a way to hack it?
The idea to solve it is to have a simple device NOT connected to anything, without WLAN, Bluetooth and Internet at all. Its a little autarc shielded device and you enter the message by hand via the buttons of the device. Shown in the photo above. If you send it you can just make a photo of that screen and send it to your friend, or you write it with a edding marker at the door of your schools toilet lol whatever, or graffity it on a train driving to your friends city. He enter the code by hand in his device, and is the only one who can read the device. Even if you use email or your smartphone to send the message, it safe, after its encrypted by the Trackless device nobody except the target device is able to decrypt it. EVEN if the source code of Trackless is known. Thats why it is much more comfortable to compress the messages, you will only send messages from 20-80 characters, and your friend need to input the encrypted string to read it. Entering 50 characters takes way longer then enter 22 characters or so.
Thats why in this case compression is important. Of course one day we maybe have a way to safely export and import strings like photo recognition, or whatever else, but it must be 100% safe so for now we only enter it by hand. I now just set the compression on, and then I can show here some sample messages, in different states, and also unsolved messages, so you can try hacking. So give me a few moments please.

5

I will! Thats why I´m here. The code can be known, including the encryption and compression algorithm, that won´t help to hack it. You will see why. I now integrate the compression and then export messages of different states to show how it work in detail, and also gives you a message you can try to hack.

6

Hi, I added the compression to the rest of the code, and now it works all together, so I created a sample for you, and a message to solve:

I toke two devices, and booted them with the same master key which then gets erased so, only the hardware shielded devices know afterward. This master key gets created by entering characters into a matrix of 20x4 characters. So up to 80 characters. We only enter 8 characters there, somewhere. Possible variants based on the fact we have 80 different characters and 80 slots for result in:
80 chars x 80 free positions x 80 chars in 79 free positions, 80 chars in 78 and so fort, that's 1,96E+30 possible combinations. Of course, I suggest using 10+ characters but already 8 are damn strong, and should be enough for this reason. Entered at the boot up process of each device, it looks like that:

pic1

This key get then encrypted using the device's encryption function, which uses a dummy master key. Afterward it looks like this:

pic2

The master key has now 80 characters, and we give the device a unique serial number.
For the examples below which I show you, so you can try hacking it. I of course use another master key, also one defined with 8 different characters somewhere placed in the 80 possible spaces.

Now we go for the "GENERATED ANONYM-ID" function to generate an encryped and always changing string containing secretly our device ID
I setup two devices, with ID 1 and ID 2, normally you would never know or find out just by the ANONYM-ID.

Just for fun I create two different IDs with each device:
Device ID1: "Niy, fJO/" and "-SFk CJ[J"
Device ID2: "8YSY 3V8N" and "MpFY %sKq"

Note: Use Consolas or similar font to see a clear difference between O and 0 and so on. In this board the letters maybe looks somehow similar.

Then we take Device 2, and go to SEND MESSAGE. We enter one of the Ids of Device 1 as target: "Niy, fJO/"
We enter the message: "safe and sound encryption"
The device encodes it to:",yb+&@Snv,AFeN"
We do the same again, sending the same message, and recognize the result is complete different: "CTGuVNAKBfBQoN"

This is because the device has an inbuilt counter for sending messages, it will that way alter the message even if you write the same, and the encryption engine will take care that result in a totally changed message. Altering each letter also influence and change the entire encrypted string.

Now we take Device 1, go to receive message, enter the encrypted string, no matter if first or second message, and the device translate it into the clear message: "safe and sound encryption"

Even the fact that you now knows the message in two variants, in clear AND encrypted form, shouldn't help you to hack the device and get the real master key.
You can try, now I encrypt another message, which I don't tell you, using Device 1, send it to Device 2. So same master key, same device ID. I just don't tell you the clear message.

And I even send it twice, so we get two encrypted strings of the same text message:

String 1: "r(,+!=xac9qfinYsqc)"
String 2: "7%OkKg8K/5IS!ne)ohI"

Good luck hacking it! Can you? I guess nobody can't but let's proof that.

Of course, it's easier but also impossible if you have the code of how the Encryption engine work in detail, as well as the compression function. I don't know if it's a good idea to open post the code right here. We developed the entire thing and of course want to keep the copyrights, but it is also no secret, so it can be seen and proofed.

You can email me to markus@trackless.info or leave me a message here.

Best wishes, and I wish you good luck hacking the message!

Markus

7

Why are you bothering to post a commercial project on an open source code forum?

This thread appears to be marketing spam.

8

If I want to make advertisement, I do not try to find some intelligent people want to have fun with trying to hack my code, then I or we search people who want to order it, consumers whatever.
The code for the simple device can be free, so people can use it and send messages if they like to build one with an oled and some buttons.
Here it's just about the code, and having fun with encryption, programming and hacking.

And we do something good with it, what message service´s code you can read and understand and be sure you can use it and are not risk to get hacked.

9

Surely you are joking! Well, do have fun!

10

And I thought you believe the code will show a weakness to hack it.
What I don´t believe, and which will be fun to find out.
I just not directly post it right here because I HAVE NO IDEA about the conditions in detail of open source, this board etc. It's ok if anybody see, or build an own test, but of course we want to use it for our device as well.

11

Hello Jremington, I don't know what you are able to and like to do. Can you entirely check something like Telegram or WhatsApp safe and sound? Or Open Source Projects as complex as Vera Crypt? For probably, you could check our little project, I guess. At least, it's way more simple / non-complex. But I would say not less safe, or the exact opposite. Probably you can blast it in pieces, who knows. But I don't want to bother you. If you like to take that challenge, just let me know.
And don't wonder I'm now off for the night. It's late here. Cheers!