Mac Question

Forum 2005-2010 (read only) Software Interfacing
1

I'm working on a touch pad similar to this one:

http://afrotechmods.com/forums/index.php/topic,8501.0.html

Here's my question:
On my Mac, is there any way I can move the mouse? (I don't want to click. I already found an auto-clicker) I was thinking something in automator and applescript. I know that I can simulate keypresses in applescript, but I need to simulate the mouse.

Ex:
Arduino sends coords, say, {500,600} down serial line. Shell script and C++ code that I wrote saves this to a file, which applescript can then read and put the mouse at the coordinates.

Help Please!

Also, I would prefer to write code than to download something-- I want to know how it works.

2

but you want to use the pad as a mouse? how would the automator read the file if the shell script is writing to it?

I'm not 100% sure, but I think it is possible to access the "serial" port from automator. I may, and most likely am, wrong.

Wouldn't it be simpler to emulate a mouse from the Arduino? Like a plug and play mouse?

3

How can I emulate the mouse?

(What I was going to do:)

On macs, serial connections are easily read and writable. So SH program reads the serial data and puts it in a file. Applescript reads te contents of file.

4

Have you used Processing before? If you don't have to use applescript, you could use the robot class in processing to do it pretty easily.

http://wiki.processing.org/w/Robot_class

it's pretty much just
robot.mouseMove(x, y);

5

THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! THANK YOU!!!! :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile:

6

But...

Cannot find a class or type named "Robot"

with this code:

/**
robot taken from http://wiki.processing.org/index.php/Robot_class
@author Ira Greenberg
*/
 
/* WARNING: This sketch takes over your mouse
 Press escape to close running sketch */
 
RoboMouse robo; 
Bot bot;
float tempX = 0;
float tempY = 0;
float springSpeed = .002;
float damping = .985;
 
void setup(){
  size(400, 400);
  noStroke();
  robo = new RoboMouse(frame.getLocation().x+width/2,
            frame.getLocation().y+height/2, 5, 0, 0);
  bot = new Bot(random(50 , width - 50), random(50, width - 50),
                   random(5, 20), random(.5, 5), random(.5, 5));
}
 
void draw(){
  fill(200, 40);
  rect(0, 0, width, height);
  robo.move();
  // Uncheck to keep mouse in sketch window
  // robo.checkBoundaries();
 
  fill(255);
  bot.create();
  bot.move();
  bot.checkBoundaries();
 
  // spring stuff
  float dx = (bot.x+8-robo.x)*springSpeed;
  float dy = (bot.y+8-robo.y)*springSpeed;
  tempX += dx;
  tempY += dy;
  robo.x += tempX;
  robo.y += tempY;
  tempX *= damping;
  tempY *= damping;
}
 
class RoboMouse extends RoundSprite{
 
  Robot robot;
  float localX, localY;
 
  RoboMouse(float x, float y, float radius, 
               float speedX, float speedY){
    super(x, y, radius, speedX, speedY);
    localX = frame.getLocation().x;
    localY = frame.getLocation().y;
    try { 
      robot = new Robot();
    } 
    catch (AWTException e) {
      e.printStackTrace(); 
    }
  }
 
  void checkBoundaries(){
    if (x>width-radius+localX){
      x = width-radius+localX;
      speedX *= -1;
    }
    if (x<radius+localX){
      x = radius+localX;
      speedX *= -1;
    }
    if (y>height-radius+localY){
      y = height-radius+localY;
      speedY *= -1;
    }
    if (y<radius+localY){
      y = radius+localY;
      speedY *= -1;
    }
  }
 
  void move(){
    x += speedX;
    y += speedY;
    robot.mouseMove(frame.getLocation().x+(int)x, 
                    frame.getLocation().y+(int)y);
  }
}
 
class Bot extends RoundSprite{
 
  Bot(float x, float y, float radius, 
         float speedX, float speedY){
    super(x, y, radius, speedX, speedY);
  }
 
  void create(){
    ellipse(x, y, radius*2, radius*2);
  }
}
 
// extend me please
class RoundSprite {
 
  float x, y, radius;
  float speedX, speedY;
  RoundSprite(float x, float y, float radius,
                  float speedX, float speedY){
    this.x = x;
    this.y = y;
    this.radius = radius;
    this.speedX = speedX;
    this.speedY = speedY;
  }
  void move(){
    x += speedX;
    y += speedY;
  }
 
  void checkBoundaries(){
    if (x>width-radius){
      x = width-radius;
      speedX *= -1;
 
    }
    if (x<radius){
      x = radius;
      speedX *= -1;
    }
    if (y>height-radius){
      y = height-radius;
      speedY *= -1;
    }
    if (y<radius){
      y = radius;
      speedY *= -1;
    }
  }
}

(example from processing wiki)

7

you need to import the library, so try putting this at the beginning of the sketch:

import java.awt.Robot;
8

OK, but now:

/**
robot taken from http://wiki.processing.org/index.php/Robot_class
@author Ira Greenberg
*/

/* WARNING: This sketch takes over your mouse
 Press escape to close running sketch */
import java.awt.Robot; 
RoboMouse robo;
Bot bot;
float tempX = 0;
float tempY = 0;
float springSpeed = .002;
float damping = .985;

void setup(){
  size(400, 400);
  noStroke();
  robo = new RoboMouse(frame.getLocation().x+width/2,
            frame.getLocation().y+height/2, 5, 0, 0);
  bot = new Bot(random(50 , width - 50), random(50, width - 50),
                   random(5, 20), random(.5, 5), random(.5, 5));
}

void draw(){
  fill(200, 40);
  rect(0, 0, width, height);
  robo.move();
  // Uncheck to keep mouse in sketch window
  // robo.checkBoundaries();

  fill(255);
  bot.create();
  bot.move();
  bot.checkBoundaries();

  // spring stuff
  float dx = (bot.x+8-robo.x)*springSpeed;
  float dy = (bot.y+8-robo.y)*springSpeed;
  tempX += dx;
  tempY += dy;
  robo.x += tempX;
  robo.y += tempY;
  tempX *= damping;
  tempY *= damping;
}

class RoboMouse extends RoundSprite{

  Robot robot;
  float localX, localY;

  RoboMouse(float x, float y, float radius,
               float speedX, float speedY){
    super(x, y, radius, speedX, speedY);
    localX = frame.getLocation().x;
    localY = frame.getLocation().y;
    try {
      robot = new Robot();
    }
    [glow]catch (AWTException e) {[/glow]
      e.printStackTrace();
    }
  }

  void checkBoundaries(){
    if (x>width-radius+localX){
      x = width-radius+localX;
      speedX *= -1;
    }
    if (x<radius+localX){
      x = radius+localX;
      speedX *= -1;
    }
    if (y>height-radius+localY){
      y = height-radius+localY;
      speedY *= -1;
    }
    if (y<radius+localY){
      y = radius+localY;
      speedY *= -1;
    }
  }

  void move(){
    x += speedX;
    y += speedY;
    robot.mouseMove(frame.getLocation().x+(int)x,
                    frame.getLocation().y+(int)y);
  }
}

class Bot extends RoundSprite{

  Bot(float x, float y, float radius,
         float speedX, float speedY){
    super(x, y, radius, speedX, speedY);
  }

  void create(){
    ellipse(x, y, radius*2, radius*2);
  }
}

// extend me please
class RoundSprite {

  float x, y, radius;
  float speedX, speedY;
  RoundSprite(float x, float y, float radius,
                  float speedX, float speedY){
    this.x = x;
    this.y = y;
    this.radius = radius;
    this.speedX = speedX;
    this.speedY = speedY;
  }
  void move(){
    x += speedX;
    y += speedY;
  }

  void checkBoundaries(){
    if (x>width-radius){
      x = width-radius;
      speedX *= -1;

    }
    if (x<radius){
      x = radius;
      speedX *= -1;
    }
    if (y>height-radius){
      y = height-radius;
      speedY *= -1;
    }
    if (y<radius){
      y = radius;
      speedY *= -1;
    }
  }
}

Where I highlighted, it said:

Cannot find a class or type named "AWTException."

Thanks in advance, Kevin.

9

hmm, when I run this it doesn't require me to import anything. maybe if you do it once you don't need to do it again?

anyway, try importing 'java.awt.AWTException' as well. If that doesn't work, try just 'java.'. I use the wildcard() in java frequently, but I don't know if it'll work in processing.

10

YAY! It works!

Now how would I feed Serial data into processing variables?
(Arduino is sending points in format:

500 600
550 890

and so forth)

11

Can I use processing to click the Left Mouse?

12

as for clicking the mouse, yes you can. see JDK 20 Documentation - Home for all the commands

as for processing parsing the data, you need to format it so it knows when each packet ends. One way to do this is to have commas separating each variable and a newline at the end.

Processing Code:

void serialEvent(Serial myPort) {
 // get the ASCII string:
 String inString = myPort.readStringUntil('\n');
 
 if (inString != null) {
 // trim off any whitespace:
 inString = trim(inString);
 // split the string on the commas and convert the
 // resulting substrings into an integer array:
 float[] vals = float(split(inString, ","));
 // if the array has at least three elements, you know
 // you got the whole thing.  Put the numbers in the
 // variables:
 if (vals.length >=2) {
 // map them to the range 0-255:
 firstValue = map(vals[0], 0, 1023, width, 0);
 secondValue = map(vals[1], 0, 1023, 0, height);

Arduino code:

const int firstPin = 5;    
const int secondPin =4;   

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  Serial.print(analogRead(firstPin));
  Serial.print(",");
  Serial.println(analogRead(secondPin)); // note the carriage return
  
  delay(10);
}

of course, you would need to modify the sketches to fit your needs.

[EDIT: I changed it to two variables instead of four to fit the current situation]

13

Thanks!
On the oracle site you metntioned, I would need to press AND release the mouse button, right?

(Also, can I use any java function in Processing?)

14

exactly, yeah. I had trouble figuring that one out. I couldn't figure out why there were constant 'A's all over the place because I didn't see the release command at first.

All the java functions I've tried have worked in Processing, but that certainly does not mean they all do. The easiest way is probably to stick it in and see if it yells at you for it :wink: