Gesture Recognition

Projects Programming
1

Hi,
Need a bit of guidance about gesture recognition.

What I’m trying to do is read coordinates from a touch screen and turn them into gestures i.e. Tap, Double Tap, Swipe Left|Right, Swipe Down|Up, etc…

I am already reading the Touch Events and putting into a structure like

struct _coordinates 
{
	uint32_t x;
	uint32_t y;
	uint8_t finger;
};

struct touch_screen_event
{
	uint8_t  number_of_fingers;
	struct _coordinates coordinates [5];
};

Now I’m thinking I need to pass this onto the recognition engine and it would store and compare and then return the gesture.

This is where I’m drawing a mind blank, does anyone have any ideas on how I can achieve this?

Thanks for the time and help.

2

Now I'm thinking I need to pass this onto the recognition engine and it would store and compare and then return the gesture.

Sounds like a plan to me.

This is where I'm drawing a mind blank, does anyone have any ideas on how I can achieve this?

Apple does. Samsung does. It is not a trivial task. One challenge is recognizing when a gesture starts and ends. When pinching to shrink an image, for instance, it is impossible to get both fingers to touch the screen at exactly the same time. So there is a period of time (and distance) where a pinch gesture looks like a drag gesture.

I suspect that 5 coordinates is going to prove too few.

3

Thanks for tacking the time to replay Paul.

Some miss understandings the 5 coordinates are one for each finger, i.e. the panel can track five fingers at any one time.

also it's the recognition engine part I'm drawing a blank on, don't know where to start.

pushing forwards I've start to put together a test project (as I'm leaning this as I go the following code could be completely wrong, I'm open to all suggestions for improvements)

Waring: the follow compiles but is untested
GestureRecognition.h

/*
* GestureRecognition.h Gesture Recognition Class Header File
*/
#ifndef GestureRecognition_h
	#define GestureRecognition_h

	/* includes */
	#include <Stdint.h>
	#include "Arduino.h"

	struct _coordinates  
	{ 
		uint8_t finger;
		uint32_t x;
		uint32_t y;
	};

	struct ts_event
	{
		uint8_t  number_of_fingers;
		uint32_t time_stamp;
		struct _coordinates  coordinates [5];
	};

	class GestureRecognition{
		public:
		/* Members */

		/* Members Functions */
		GestureRecognition();
		void GestureEvent(const ts_event &event);   
		private:
		/* Members Functions */
		void _Recognise();

		/* Members */
		bool _touchBegin;
		bool _touchEnd;
		uint32_t _xDelta;
		uint32_t _yDelta;
		uint32_t _timeDelta;
	};

#endif

GestureRecognition.cpp

/*
* GestureRecognition.cpp Gesture Recognition Class File
*/

/* includes */
#include "GestureRecognition.h"

GestureRecognition::GestureRecognition()
{

}

void GestureRecognition::GestureEvent(const ts_event &event)
{
	// TODO: Code(Add Gesture Event to Stack)

	// Set touch begin and end.
	if (event.number_of_fingers > 0 && !_touchBegin)
	{
		_touchBegin = 1;
	};

	if (event.number_of_fingers == 0 && _touchBegin && !_touchEnd)
	{
		_touchEnd = 1;
	};


	// Try to Recognise Gesture
	_Recognise(); 
};		

void GestureRecognition::_Recognise()
{
	// TODO: Examine Stack and See if we can Recognise a Gesture
};

GR_Test.ino

#include "GestureRecognition.h"
#include "testTouch.h"

GestureRecognition GR = GestureRecognition();

void setup() {
	Serial.begin(115200);
	delay(500);

	for (int i = 0; i < 13; i++)
	{
		GR.GestureEvent(slowTap[i]);
	}

}

void loop() {

}

testTouch

#ifndef _testTouch_h
	#define _testTouch_h

	#include "GestureRecognition.h"

	const ts_event quickTap[5] = {
		{1, 144940, 1, 405, 311, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 144960, 1, 405, 311, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 144979, 1, 405, 311, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 144998, 1, 405, 311, 0, 0, 0, 0, 0, 0, 0, 0 },
		{0, 144998, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }
	};

	const ts_event slowTap[13] = {
		{1, 145977, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 145996, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146016, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146036, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146055, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146075, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146094, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146114, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146133, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146153, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146172, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{1, 146192, 1, 390, 304, 0, 0, 0, 0, 0, 0, 0, 0 },
		{0, 146192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }
	};

#endif

From the above test project you can probability guess I'm not a programmer only a hobbyist.

Anyways how can I build a stack in the class and then how can I examine the stack?

a bit of a brakedown on the test data a quick tap

Finger Count Time Stamp Finger Finger 1 X Finger 1 Y Finger 2 X Finger 2 Y Finger 3 X Finger 3 Y Finger 4 X Finger 4 Y Finger 5 X Finger 5 Y
1 144940 1 405 311 0 0 0 0 0 0 0 0
1 144960 1 405 311 0 0 0 0 0 0 0 0
1 144979 1 405 311 0 0 0 0 0 0 0 0
1 144998 1 405 311 0 0 0 0 0 0 0 0
0 144998 0 0 0 0 0 0 0 0 0 0 0

in this we touched on and off in less than 250ms and the delta X = 0 and delta Y = 0. So it should be a 'Single Tap' and or a possible 'Double Tap' in progress.

I'm thinking a stack for each fingers X and Y shorted by time (again can I do stacks on the Arduino?) then I would want to get the min and max and so on.

4

What is your hardware? I think, most of arduino touch shields use cheap resistive sensor, that is unable to distinguish 1 or 2 finger. Tablets use capacitive sensors, more accurate, faster etc.

5

I'm using a 5 inch Multi Touch Screen Panel with Controller GSL1680 for 800x480 display (Link)

I have this up and running and am reading up to 5 simultaneous touches happily but it's a bit hard to get 5 fingers onto a 5 inch panel tho.

6

Some miss understandings the 5 coordinates are one for each finger, i.e. the panel can track five fingers at any one time.

So, coordinates is a dumb name in that it causes confusion. fingerPositions would be a better choice. And, somewhere, obviously, you need an array (or stack or list) of events.

7

Ok think I may have something workable still adding gestures and debugging/improving

Still looking for ideas, comments, code improvements, etc..

GR_Test.ino

#include "GestureRecognition.h"
#include "testTouch.h"

GestureRecognition GR = GestureRecognition();

void setup() {
 Serial.begin(115200);
 delay(1000);
 Serial.println("Go...");
 printCommands();
}

void loop() {
 char command = getCommand();
 switch (command)
 {
 case '0':
 Serial.println("Testing quick Tap...");
 for (int i = 0; i < 5; i++)
 {
 GR.GestureEvent(quickTap[i]);
 }
 printCommands();
 break;
 case '1':
 Serial.println("Testing Slow Tap...");
 for (int i = 0; i < 13; i++)
 {
 GR.GestureEvent(slowTap[i]);
 }
 printCommands();
 break;
 case '2':
 Serial.println("Testing Double Tap...");
 for (int i = 0; i < 10; i++)
 {
 GR.GestureEvent(doubleTap[i]);
 }
 printCommands();
 break;
 case '3':
 Serial.println("Testing One Finger Swipe Down...");
 for (int i = 0; i < 22; i++)
 {
 GR.GestureEvent(oneFingerSwipeDown[i]);
 }
 printCommands();
 break;
 case '4':
 Serial.println("Testing One Finger Swipe Up...");
 for (int i = 0; i < 31; i++)
 {
 GR.GestureEvent(oneFingerSwipeUp[i]);
 }
 printCommands();
 break;
 case '5':
 Serial.println("Testing Two Finger Swipe Down...");
 for (int i = 0; i < 49; i++)
 {
 GR.GestureEvent(twoFingerSwipeDown[i]);
 }
 printCommands();
 break;
 case '6':
 Serial.println("Testing Two Finger Swipe Up...");
 for (int i = 0; i < 33; i++)
 {
 GR.GestureEvent(twoFingerSwipeUp[i]);
 }
 printCommands();
 break;
 case '7':
 Serial.println("Testing Adjacent Corner Touch...");
 for (int i = 0; i < 40; i++)
 {
 GR.GestureEvent(adjacentCornerTouch[i]);
 }
 printCommands();
 break;
 default:
 break;
 }
}

char getCommand()
{
 char c = '\0';
 if (Serial.available())
 {
 c = Serial.read();
 }
 return c;
}

void printCommands()
{
 Serial.println("Send:");
 Serial.println("  0 to run Quick Tap test");
 Serial.println("  1 to run Slow Tap test");
 Serial.println("  2 to run Double Tap test");
 Serial.println("  3 to run One Finger Swipe Down test");
 Serial.println("  4 to run One Finger Swipe Up test");
 Serial.println("  5 to run Two Finger Swipe Down test");
 Serial.println("  6 to run Two Finger Swipe Up test");
 Serial.println("  7 to run Adjacent Corner Touch test");
}

GestureRecognition.h

/*
* GestureRecognition.h Gesture Recognition Class Header File
*/
#ifndef GestureRecognition_h
 #define GestureRecognition_h
 #define SAMPLES 50

 /* includes */
 #include <Stdint.h>
 #include "Arduino.h"

 struct _fingerPosition  
 { 
 uint8_t finger;
 uint32_t x;
 uint32_t y;
 };

 struct ts_event
 {
 uint8_t  number_of_fingers;
 uint32_t time_stamp;
 struct _fingerPosition  fingerPositions[5];
 };

 class GestureRecognition{
 public:
 /* Members */
 uint8_t down;
 uint16_t downSpeed;
 uint8_t up;
 uint16_t upSpeed;
 uint8_t right;
 uint16_t rightSpeed;
 uint8_t left;
 uint16_t leftSpeed;

 /* Members Functions */
 GestureRecognition();
 void GestureEvent(const ts_event &event);   
 private:
 /* Members */
 struct _finger
 {
 int32_t x;
 int32_t xLast;
 int32_t y;
 int32_t yLast;
 int16_t xSpeed;
 int16_t xSpeedLast;
 int16_t ySpeed;
 int16_t ySpeedLast;
 int32_t timeStamp;
 int32_t timeStampLast;
 int8_t up;
 int8_t down;
 int8_t right;
 int8_t left;
 int8_t hold;
 };
 bool _touchBegin;
 bool _touchEnd;
 uint8_t _fingerCount;
 uint8_t _fingerCountLast;
 struct _finger _finger1 = _finger();
 struct _finger _finger2 = _finger();
 struct _finger _finger3 = _finger();
 struct _finger _finger4 = _finger();
 struct _finger _finger5 = _finger();

 /* Members Functions */
 void _recognise();
 void _evaluateFinger(struct _finger &finger);
 void _clearFinger(struct _finger &finger);
 };

#endif
8

GestureRecognition.cpp

/*
* GestureRecognition.cpp Gesture Recognition Class File
*/

/* includes */
#include "GestureRecognition.h"

GestureRecognition::GestureRecognition()
{

}

void GestureRecognition::GestureEvent(const ts_event &event)
{
 // Serial.println("New Event...");
 _fingerCount = event.number_of_fingers;
 if (_fingerCount != _fingerCountLast)
 {
 _clearFinger(_finger1);
 _clearFinger(_finger2);
 _clearFinger(_finger3);
 _clearFinger(_finger4);
 _clearFinger(_finger5);
 _fingerCountLast = _fingerCount;
 }
 // Add Gesture Event to Stack
 Serial.print("Finger Count "), Serial.print(_fingerCount);
 switch (_fingerCount){
 case 0:
 _clearFinger(_finger1);
 _clearFinger(_finger2);
 _clearFinger(_finger3);
 _clearFinger(_finger4);
 _clearFinger(_finger5);
 break;
 case 1:
 _finger1.x = event.fingerPositions[0].x;
 _finger1.y = event.fingerPositions[0].y;
 _finger1.timeStamp = event.time_stamp;
 _evaluateFinger(_finger1);
 break;
 case 2:
 _finger1.x = event.fingerPositions[0].x;
 _finger1.y = event.fingerPositions[0].y;
 _finger1.timeStamp = event.time_stamp;
 _evaluateFinger(_finger1);
 _finger2.x = event.fingerPositions[1].x;
 _finger2.y = event.fingerPositions[1].y;
 _finger2.timeStamp = event.time_stamp;
 _evaluateFinger(_finger2);
 break;
 case 3:
 case 4:
 case 5:
 // Don't care for now
 
 default:
 break;
 }

 // Set touch begin and end.
 // TODO: Rework this
 if (event.number_of_fingers > 0 && !_touchBegin)
 {
 _touchBegin = 1;
 };

 if (event.number_of_fingers == 0 && _touchBegin && !_touchEnd)
 {
 _touchEnd = 1;
 };

 // Try to Recognise Gesture
 _recognise(); 
}; 

void GestureRecognition::_evaluateFinger(struct _finger &finger)
{
 // Serial.println("Evaluating a Finger...");
 int32_t xDelta;
 int32_t yDelta; 
 int32_t timeDelta;

 /* Serial.print(", x: "), Serial.print(finger.x);
 Serial.print(", y: "), Serial.print(finger.y);
 Serial.print(", time: "), Serial.print(finger.timeStamp);*/

 xDelta = finger.x - finger.xLast;
 yDelta = finger.y - finger.yLast;
 timeDelta = finger.timeStamp - finger.timeStampLast;

 Serial.print(", xDelta: "), Serial.print(xDelta);
 Serial.print(", yDelta: "), Serial.print(yDelta);
 Serial.print(", timeDelta: "), Serial.print(timeDelta);

 // Evaluate the X directions
 if (xDelta != 0)
 {
 if (xDelta > 0)
 {
 // We are moving down
 finger.down++;
 // Hold & Up decay
 finger.hold--;
 finger.up--;
 }else{
 // We are moving up
 finger.up++;
 // Hold & down decay
 finger.hold--;
 finger.down--;
 }
 }else{
 // Decay X Directions
 finger.down--;
 finger.up--; 
 }
 // Set upper and lower limits 0-10
 if (finger.down > 10) {finger.down = 10;}
 if (finger.up > 10) {finger.up = 10;}
 if (finger.hold > 10) {finger.hold = 10;}
 if (finger.down < 0) {finger.down = 0;}
 if (finger.up < 0) {finger.up = 0;}
 if (finger.hold < 0) {finger.hold = 0;}

 // Calculate speed of the X direction pixels/s
 if (timeDelta !=0){
 finger.xSpeed = abs(xDelta *1000 / timeDelta);
 }

 // Evaluate the Y directions
 if (yDelta != 0)
 {
 if (yDelta > 0)
 {
 // We are moving right
 finger.right++;
 // Hold & left decay
 finger.hold--;
 finger.left--;
 }else{
 // We are moving left
 finger.left++;
 // Hold & right decay
 finger.hold--;
 finger.right--;
 }
 }else{
 // Decay Y Directions
 finger.right--;
 finger.left--;
 }
 // Set upper and lower limits 0-10
 if (finger.right > 10) {finger.right = 10;}
 if (finger.right < 0) {finger.right = 0;}
 if (finger.left > 10) {finger.left = 10;}
 if (finger.left < 0) {finger.left = 0;}

 // Calculate speed of the Y direction pixels/s
 if (timeDelta !=0){
 finger.ySpeed = abs(yDelta * 1000 / timeDelta);
 }

 //TODO: Hold, need to acount for wobble
 /* if (xDelta == 0 && yDelta == 0)
 {

 }*/

 // Update Last
 finger.xLast = finger.x;
 finger.yLast = finger.y;
 finger.xSpeedLast = finger.xSpeed;
 finger.ySpeedLast = finger.ySpeed;
 finger.timeStampLast = finger.timeStamp;
}

void GestureRecognition::_clearFinger(struct _finger &finger)
{
 finger.x = 0;
 finger.xLast = 0;
 finger.y = 0;
 finger.yLast = 0;
 finger.xSpeed = 0;
 finger.xSpeedLast = 0;
 finger.ySpeed = 0;
 finger.ySpeedLast = 0;
 finger.timeStamp = 0;
 finger.timeStampLast = 0;
 finger.up = 0;
 finger.down = 0;
 finger.right = 0;
 finger.left = 0;
 finger.hold = 0;
}

void GestureRecognition::_recognise()
{
 // TODO:
 // flick left or right
 // Tap
 // Double Tap

 // How many finger gesture is this
 switch (_fingerCount)
 {
 case 0:
 // TODO:
 // What just ended? Check Flags and see if we justed Tap and could be ging for a double tap
 break;
 case 1:
 Serial.print(" Up Count: "), Serial.print(_finger1.up);
 Serial.print(", Down Count: "), Serial.print(_finger1.down);
 Serial.print(", Right Count: "), Serial.print(_finger1.right);
 Serial.print(", Left Count: "), Serial.print(_finger1.left);
 // Are we swiping down
 if (_finger1.down > 7)
 {
 down = 1;
 downSpeed = _finger1.xSpeed;
 Serial.print(", Swiping Down at ");
 Serial.print(downSpeed);
 Serial.print(" pixels/s");
 }else if(_finger1.down < 5)
 {
 down = 0;
 downSpeed = 0;
 }
 // Are we swiping up
 if (_finger1.up > 7)
 {
 up = 1;
 upSpeed = _finger1.xSpeed;
 Serial.print(", Swiping Up at ");
 Serial.print(upSpeed);
 Serial.print(" pixels/s");
 }else if(_finger1.up < 5)
 {
 up = 0;
 upSpeed = 0;
 }
 // Are we swiping left
 if (_finger1.left > 7)
 {
 left = 1;
 leftSpeed = _finger1.ySpeed;
 Serial.print(", Swiping left at ");
 Serial.print(leftSpeed);
 Serial.print(" pixels/s");
 }else if(_finger1.left < 5)
 {
 left = 0;
 leftSpeed = 0;
 }
 // Are we swiping right
 if (_finger1.right > 7)
 {
 right = 1;
 rightSpeed = _finger1.ySpeed;
 Serial.print(", Swiping right at ");
 Serial.print(rightSpeed);
 Serial.print(" pixels/s");
 }else if(_finger1.right < 5)
 {
 right = 0;
 rightSpeed = 0;
 }
 // TODO:
 // Are we doing a single tap
 // TODO:
 // Are we doing a double tap
 // TODO:
 // Are we just holding
 break;
 case 2:
 Serial.print(" Up Counts: "), Serial.print(_finger1.up), Serial.print(" | "), Serial.print(_finger2.up);
 Serial.print(", Down Counts: "), Serial.print(_finger1.down), Serial.print(" | "), Serial.print(_finger2.down);
 Serial.print(", Right Counts: "), Serial.print(_finger1.right), Serial.print(" | "), Serial.print(_finger2.right);
 Serial.print(", Left Counts: "), Serial.print(_finger1.left), Serial.print(" | "), Serial.print(_finger2.left);
 // Are we swiping down
 if (_finger1.down > 7 && _finger2.down > 7)
 {
 down = 1;
 downSpeed = (_finger1.xSpeed + _finger2.xSpeed) / 2;
 Serial.print(", Swiping Down at ");
 Serial.print(downSpeed);
 Serial.print(" pixels/s");
 }else if(_finger1.down < 5 || _finger2.down < 5)
 {
 down = 0;
 downSpeed = 0;
 }
 // Are we swiping up
 if (_finger1.up > 7 && _finger2.up > 7)
 {
 up = 1;
 upSpeed = (_finger1.xSpeed + _finger2.xSpeed) / 2;
 Serial.print(", Swiping Up at ");
 Serial.print(upSpeed);
 Serial.print(" pixels/s");
 }else if(_finger1.up < 5 || _finger2.up < 5)
 {
 up = 0;
 upSpeed = 0;
 }
 // TODO:
 // Are we touching adjacent corners
 // TODO:
 // Are we just holding
 break;
 default:
 // Don't really care for now
 break;
 }
 Serial.println("");
};
9

testTouch.h 1of2

#ifndef _testTouch_h
 #define _testTouch_h

 #include "GestureRecognition.h"

 const ts_event quickTap[5] = {
 {1, 144940, 1, 405, 311, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 144960, 1, 405, 311, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 144979, 1, 405, 311, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 144998, 1, 405, 311, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 144998, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };

 const ts_event slowTap[13] = {
 {1, 145977, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 145996, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146016, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146036, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146055, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146075, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146094, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146114, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146133, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146153, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146172, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 146192, 1, 390, 304, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 146192, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };

 const ts_event doubleTap[10] = {
 {1, 618763, 1, 414, 244, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 618782, 1, 414, 244, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 618802, 1, 414, 244, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 618821, 1, 414, 244, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 618821, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 618938, 1, 418, 238, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 618958, 1, 418, 238, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 618978, 1, 418, 238, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 618997, 1, 418, 238, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 618997, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };

 const ts_event oneFingerSwipeDown[22] = {
 {1, 426051, 1, 1, 208, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426071, 1, 1, 208, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426090, 1, 1, 208, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426110, 1, 1, 208, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426129, 1, 7, 213, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426149, 1, 52, 213, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426168, 1, 78, 213, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426188, 1, 108, 212, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426208, 1, 139, 212, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426227, 1, 172, 212, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426247, 1, 214, 215, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426266, 1, 261, 217, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426285, 1, 310, 213, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426305, 1, 360, 210, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426325, 1, 411, 206, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426345, 1, 464, 204, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426364, 1, 518, 204, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426384, 1, 573, 206, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426403, 1, 627, 206, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426423, 1, 685, 204, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 426442, 1, 740, 204, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 426442, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };

 const ts_event oneFingerSwipeUp[31] = {
 {1, 776583, 1, 735, 226, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776603, 1, 735, 226, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776622, 1, 735, 226, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776642, 1, 735, 226, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776662, 1, 735, 226, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776681, 1, 729, 231, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776701, 1, 723, 232, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776721, 1, 713, 231, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776740, 1, 703, 231, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776760, 1, 691, 228, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776779, 1, 677, 228, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776799, 1, 660, 228, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776819, 1, 642, 228, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776838, 1, 620, 231, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776858, 1, 594, 232, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776877, 1, 569, 232, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776897, 1, 540, 232, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776916, 1, 512, 229, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776936, 1, 484, 226, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776955, 1, 459, 224, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776975, 1, 429, 222, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 776995, 1, 395, 221, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777014, 1, 366, 221, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777034, 1, 332, 219, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777053, 1, 299, 219, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777072, 1, 258, 216, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777093, 1, 215, 212, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777112, 1, 169, 210, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777132, 1, 125, 208, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 777151, 1, 90, 208, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 777151, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };

 const ts_event twoFingerSwipeDown[49] = {
 {2, 24857, 1, 1, 219, 2, 1, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {2, 24877, 1, 1, 219, 2, 1, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {2, 24897, 1, 3, 226, 2, 1, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {2, 24916, 1, 11, 228, 2, 1, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {2, 24936, 1, 30, 228, 2, 1, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {2, 24955, 1, 54, 228, 2, 25, 354, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 24975, 1, 65, 229, 2, 56, 354, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 24994, 1, 75, 229, 2, 69, 354, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25013, 1, 90, 230, 2, 83, 352, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25033, 1, 102, 228, 2, 98, 352, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25053, 1, 116, 228, 2, 113, 352, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25073, 1, 130, 228, 2, 126, 351, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25092, 1, 146, 228, 2, 143, 351, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25112, 1, 162, 227, 2, 160, 351, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25131, 1, 178, 228, 2, 175, 350, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25151, 1, 194, 228, 2, 192, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25170, 1, 209, 226, 2, 206, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25190, 1, 225, 224, 2, 222, 346, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25209, 1, 243, 224, 2, 235, 346, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25229, 1, 256, 222, 2, 251, 346, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25249, 1, 271, 222, 2, 268, 348, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25268, 1, 286, 225, 2, 282, 350, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25288, 1, 305, 227, 2, 299, 352, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25307, 1, 322, 228, 2, 315, 354, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25327, 1, 338, 231, 2, 331, 356, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25346, 1, 356, 234, 2, 347, 359, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25365, 1, 370, 234, 2, 363, 359, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25386, 1, 385, 234, 2, 379, 359, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25405, 1, 401, 237, 2, 393, 359, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25424, 1, 415, 237, 2, 410, 363, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25444, 1, 430, 234, 2, 425, 359, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25464, 1, 445, 236, 2, 439, 359, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25483, 1, 460, 237, 2, 456, 363, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25503, 1, 475, 237, 2, 468, 365, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25524, 1, 489, 237, 2, 484, 365, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25545, 1, 502, 239, 2, 500, 368, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25565, 1, 523, 240, 2, 516, 372, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25585, 1, 539, 242, 2, 530, 373, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25604, 1, 553, 244, 2, 547, 375, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25623, 1, 571, 246, 2, 563, 377, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25644, 1, 589, 248, 2, 583, 379, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25667, 1, 609, 252, 2, 606, 381, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25687, 1, 634, 258, 2, 631, 384, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25705, 1, 655, 264, 2, 657, 384, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25725, 1, 681, 264, 2, 680, 385, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 25744, 1, 705, 264, 2, 710, 383, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {1, 25762, 1, 735, 264, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 25780, 1, 785, 267, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 25780, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };
10

testTouch.h 2of2

 const ts_event twoFingerSwipeUp[33] = {
 {1, 907734, 1, 799, 192, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 907755, 1, 799, 192, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {2, 907775, 1, 799, 192, 2, 799, 318, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907794, 1, 799, 192, 2, 799, 318, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907814, 1, 799, 192, 2, 799, 318, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907833, 1, 799, 192, 2, 799, 327, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907852, 1, 799, 192, 2, 799, 329, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907872, 1, 799, 192, 2, 799, 330, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907892, 1, 797, 201, 2, 795, 330, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907912, 1, 788, 200, 2, 774, 333, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907931, 1, 747, 202, 2, 735, 333, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907952, 1, 727, 202, 2, 715, 333, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907971, 1, 705, 202, 2, 694, 331, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 907990, 1, 679, 200, 2, 670, 327, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908010, 1, 652, 198, 2, 642, 327, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908030, 1, 625, 196, 2, 615, 323, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908050, 1, 594, 194, 2, 585, 323, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908069, 1, 565, 193, 2, 553, 321, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908088, 1, 531, 192, 2, 520, 321, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908109, 1, 495, 192, 2, 485, 318, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908128, 1, 463, 189, 2, 449, 311, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908147, 1, 426, 186, 2, 415, 308, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908168, 1, 388, 181, 2, 376, 302, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908190, 1, 342, 177, 2, 330, 294, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908209, 1, 306, 168, 2, 289, 291, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908228, 1, 265, 156, 2, 251, 283, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908248, 1, 223, 148, 2, 210, 276, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908269, 1, 182, 142, 2, 168, 263, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908288, 1, 143, 140, 2, 126, 260, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908307, 1, 100, 138, 2, 83, 258, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908327, 1, 52, 137, 2, 16, 258, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 908347, 1, 1, 135, 2, 1, 252, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {0, 908347, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };

 const ts_event adjacentCornerTouch[40] = {
 {1, 1073564, 1, 696, 414, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {1, 1073586, 1, 696, 414, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0},
 {2, 1073606, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073625, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073645, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073664, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073684, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073703, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073723, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073742, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073762, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073782, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073801, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073821, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073841, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073860, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073880, 1, 696, 414, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073903, 1, 697, 422, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073923, 1, 697, 422, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073942, 1, 697, 421, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073962, 1, 697, 421, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1073981, 1, 697, 421, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074001, 1, 697, 421, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074020, 1, 697, 421, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074040, 1, 697, 421, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074059, 1, 699, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074078, 1, 699, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074098, 1, 699, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074118, 1, 699, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074137, 1, 699, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074156, 1, 699, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074176, 1, 699, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074196, 1, 697, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074216, 1, 697, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074235, 1, 697, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074255, 1, 697, 420, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074275, 1, 696, 419, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074295, 1, 696, 417, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {2, 1074318, 1, 696, 411, 2, 65, 75, 3, 0, 0, 4, 0, 0, 5, 0, 0}, 
 {0, 1074318, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0}
 };

#endif

If you have any different ideas how this could be done feel free to share.

11

What kind of Arduino are you using?

The Uno has next to no RAM. If I were trying to do something like you are doing, I would not use an Uno: why make an already difficult task even more difficult (perhaps even impossible)?

12

odometer,
I'm using a teensy for now and am looking at a photon.