DataSleeve, the Worst Best Wearable PC Ever

A while back, I saw this picture of Limor Fried (Ladyada) at MIT wearing a really awesome wearable computer. It was taken a long time ago, but I kept looking into the wearable PC subculture.

And I loved it.

So I threw together a concept called DataSleeve, a wearable with so many compromises it's cool.

HUDs and AR glasses are expensive, so the screen is a resistive TFT on your arm. I didn't want to buy a Pi 3, so the display is soldered to a Pi 0 W. Using gesture control for the keyboard would require a lot of work, so I plonked a 12-key keyboard on the main PCB. 12 keys isn't enough for the entire alphabet though, so there's a rotary encoder in charge of what each key does. The gesture control element I did keep is just for clicks (get it?), and bending my index finger triggers a left click, and bending my middle sends a right click. There's also gestures for space.

The final three features are the most interesting: The sensors, what the thing does when you connect it to another thing using BT, and the two Arduinos driving everything.

I'll start with the sensors. An MPU6050 tells me where my hand is and how it's moving, a DHT11 breakout senses temperature and humidity, and an APDS9960 detects the temperature and how I'm waving my hand under my arm. Most sensor data is displayed on a tiny OLED.

As for the driving Arduinos, there's an RP2040 Connect and a Micro. The RP2040 Connect does pretty much everything, from taking gesture inputs to updating the OLED to taking the keyboard presses. Some of the information is relayed to the Micro, which is plugged into the Pi and sends keyboard/mouse inputs. The Micro also does a few things itself, operating the MPU, the APDS, and an emergency buzzer for some reason.

Now for the Bluetooth thing. The wearable can actually be connected to another computer and act as a Wii-style airmouse, and also send inputs from the keyboard.

For now, DataSleeve is still in prototyping. I will update the thread periodically as I make R&D and build progress.

UPDATE | 8:11 | 1/12/22
I'm changing the RP2040 Connect for a Seeeduino XIAO RP2040 clone with more pins. I'm changing the original power controller I was using to something from Seeed called the LiPo rider.

UPDATE | 13:33 | 1/12/22
So turns out the Adafruit thing I was going to add requires code to be uploaded to two boards at once, but only one board can be reprogrammed. So I guess no sensor bracket.

EDIT - Actually I might keep it, but with no native reprogramming support.

UPDATE | 14:47 | 1/12/22
I found a tiny USB hub that I might be able to epoxy somewhere. If I can make it work, the Adafruit thing might be on track after all.

UPDATE | 8:45 | 1/13/22
I had an idea in the car this morning. The Adafruit sensor bracket doesn't need a reprogramming interface. Just a lot of flash memory. Each sensor in the sensor bracket system should have a different I2C address. If I sniff the address, I can do different things based on which sensor is connected. So the Arduino Micro, where the sensor bracket is connected, would sniff the address, pick the appropriate library, and then read the sensor. Next, there's two concepts I had in mind. The first saves memory but takes longer, and the second is the opposite. The two onboard microcontrollers are connected to different things, and the Micro, which takes the sensor readings, is not directly connected to the status display. The RP2040 is. The first concept requires a USB to be connected from the sleeve's main computer to the RP2040, and what it does is simple. Each sensor has a key combination. When that sensor is put in, the Micro sends the corresponding keycombo to the computer, which uploads a new sketch to the RP2040, compensating for the new sensor. The problem with this is that flashing the RP2040 takes a minute, and that the Pi 0 only has one USB output, which is being taken by the Micro for keystrokes. As I said previously, I could epoxy a tiny USB hub to one of the PCBs, but I'd rather not. So that's the first idea. The second idea uses the software serial communication (fight me) between the Micro and RP2040 to update the status display. Along with the data normally being sent across serial, the Micro also sends a small int. Almost like a note. It tells the RP2040 what sensor is connected. Depending on this, the RP2040 can change what data is being displayed. The problem with this is that it uses a lot of precious flash and RAM, because it has to change both the amount of data points that the OLED can display and the actual data. I'll have to see how much of the 2MB of flash and 256KB of RAM I've got left after programming.

UPDATE | 13:50 | 1/13/22
Another change to the RP2040 MCU, and this time it's permanent. I found the perfect board: the Adafruit ItsyBitsy RP2040. It's got 3rd party IDE support and has 8MB of flash. I also decided to change over from software serial to hardware serial. Sacrificing easy debugging for easy communication. Maybe I'll find another way to debug.

EDIT - I might add debug through a blinking LED. Obscure, but it might work.

UPDATE | 10:14 | 1/14/22
It wasn't permanent. Adafruit's KB2040 board fits a Pro Micro footprint and still has a reasonable amount of IO.

EDIT - Also, pink/purple PCBs are awesome.

UPDATE | 10:00 | 1/20/22
I did some PCB layout. Still loads of space on the bottom PCB. Maybe I should stop looking at sensors to attach and think about making everything smaller and less bulky.

Would take a lot of work but be nice to have.

UPDATE | 9:13 | 1/21/22
I just realized that modifying the PCB design would require me to design and print out new mounts. The CAD model timeline looks like fettuccini and the 3D printer is like 15mm out of alignment on the Z axis, so I think I'll live with it.

EDIT - Also I tested the airmouse thingy on a breadboard and it works remarkably well with some modifications.

UPDATE | 12:33 | 1/24/22
Welcome back to my strange topic. The 3D printer is working again, just needs some tuning. I ordered the KB2040, 12 red key switches, and some header strip last night from Adafruit. Some Aliexpress stuff is also on the way. For now, the project is idle. I'll update when the parts arrive. Or maybe before then, I don't know.

UPDATE | 10:26 | 1/28/22
Hello. The keyboard prototype is done and details can be found here if you're interested. It was too much detail to put into an update.

UPDATE | 10:03 | 2/1/22
The Adafruit stuff arrived. The KB2040 didn't go up in smoke, but it's not uploading. The keyswitches don't fit my keycaps so I have to make new ones.

UPDATE | 9:17 | 2/2/22
The KB2040 is not going to work for this project. I think I'm going to have to use the only other high-memory MCU I have around, a WEMOS D1 Mini V2. I've already changed the PCBs over, and I just need to put some headers on it.

You might be wondering if that means I'm going to add some special IoT or WiFi function. The answer is no. Mostly because I don't want to.

It's going to need some serious code optimization, because it has 1/8 of the RP2040's RAM, but I think I'll be okay if I don't do anything crazy. Oh, and it also only has 1 analog pin, so I'll need to add an external ADC/DAC.

UPDATE |7:28 | 2/7/22
I did bit of an oopsie on the PCBs. There's identical pin headers for the same device on two of the boards. I only meant for there to be one. I guess I'll delete one. I also didn't include a header for the MPU6050, which I did add last night.

That aside, I am now in the breadboarding phase! Most of the parts I'll be using have been pressed into the 'board, and I'm using copious amounts of jumpers to connect everything. Here's to hoping I don't plug a 3.3V part into the 5V side.

UPDATE | 18:13 | 2/7/22
Decided to change back to software serial after some head-scratching.

UPDATE | 19:07 | 2/7/22
Soft serial is hard. Going to try SPI. I2C is already being used by multiple sensors.

UPDATE | 10:39 | 2/8/22
Back to software serial. Couldn't get SPI working.

UPDATE |12:57 | 2/9/22
I got software serial working through a secondary library. Data was sent back and forth with few errors.

But one of the microcontrollers was a Nano, not the WeMos I'll be using, so I still have things to try.

Except I'm not using a WeMos anymore, or at least not really. I decided to try the Bluetooth keyboard software on the Pi last night and it didn't install. So the workaround was to do it on the microcontroller side. I think I'm going to try a third-party board with the same footprint as the WeMos but an ESP32.

UPDATE | 9:03 | 2/10/22
I ordered the "ESP32 mini" as it's supposedly called. I have no idea when it will arrive. I have absolutely no idea what to do with the project until it arrives other than work on the CAD models for the few parts I haven't printed yet.

UPDATE | 9:42 | 2/11/22
I'm looking at AR glasses as a companion to the DataSleeve but they're all either not well documented or very expensive.

UPDATE | 8:21 | 2/14/22
I'm going to build the AR glasses.

"Wow, that went from 'this seems like a terrible idea' to 'lets do this' in less than 4 days."

Let me explain. The optics don't look that complex. And building it would be cheap. All it requires is an OLED, a mirror, a fresnel lens, and a piece of glass. And some Adafruit parts. As some frequent forum users may know, this isn't the first time I've played with optics...

That enough reason for you? Alright. On to the cool stuff.

The glove... works I guess. I ruined it and my hands while testing due to some bad design, but I did get one finger to read a bend through my crap 8-bit ADC, so that's nice. I also ordered a different screen. Desoldering the pins from the old screen was driving me nuts, and this new one just goes right on.

EDIT - Here's a screenshot of the AR glasses. It's a separate CAD document to the DataSleeve, but I plan to connect them via serial over a USB cable, virtually making them part of the same thing. 1 is perspective off, and 2 is perspective on.

UPDATE | 8:56 |2/17/22
Remember the KB2040? I got it working. Shame I'm not using it anymore.

Oh well, I'll find a use for it. It's a good MCU.

If anyone's wondering, the way I got it working is programming it on a Pi instead of my Linux machine, which has always kind of handled serial ports and drives sort of weirdly.

Also, the bracket for the new screen is done and it fits PERFECTLY. My rough estimates were actually really good.

UPDATE | 8:58 |2/22/22
I have the battery and screen mounts done. I'm working on the tracking glove. I tried sewing the sensors and magnets on, but it's a massive pain and kind of screwed up the glove, so I'm going to move over to contact cement.

UPDATE | 12:55 |2/23/22
I don't think I mentioned this. I'm now splitting up the DataSleeve. The Pi and battery are on my left arm, and the main electronics will be on the right. The problem is that now, I'm running into I2C cable distance issues...

UPDATE | 10:46|2/28/22
I'm doing a major PCB and 3D printed part redesign. The new stuff is all angular and mean-looking and is absolutely fantastic. Also, the contact cement is here so I can actually start the gesture sensing modules. Problem is, my experiments ruined the right glove. So I'm moving the electronics to the left and the Pi to the right to compensate.

UPDATE | 14:02|4/6/22
Abandoning the project. Going to work on other stuff instead. This too rapidly exceeded my goals and during small scale tests the impracticality of an arm computer became obvious. I will leave the thread open to inform others wanting to do similar projects.

I really love the detail and attention to updates in this post. Thank you for documenting for the community!