If I understand you right, you may have to split your project into the real time glove handling, on an Arduino, and the translation on a PC (like) computer. On the PC you'll need programs to edit and manage the gesture data base, recognition of gestures, and translation into text and audio.