Unity3d + Arduino (Read serial in Unity3d)

So, I've been bored, and trying to learn the limits of Unity3d free edition. I also work with arduino (obviously), and came up with the idea that many people have.

Why can't use use an accelerometer and use it to view the 3d world that way?

Well, I did some research on the matter, and it seemed that everyone decided that you had to have a plugin in order to send and receive information. I've found that is not true... to a point. (plugins mean you have to have the pro version which is currently $1200usd)

Importing the right stuff gets you to be able to read serial, and after a bit of annoyance on getting it to read the information right, and delaying the arduino by 10ms so that I could actually read the information, it works fine. At least in the standalone player in Windows. I'm not sure on mac, but I know that web player doesn't support it.

So, here I sit with some spiffy code, and decided that even though I haven't been in the Unity3d community long, I would simply post a bit of code that lets you get information from a serial device. (Note: You can use ANY serial device as long as you know how to read the information.)

C# Code:

using UnityEngine;
using System.Collections;
using System.IO.Ports;

public class serialRotation : MonoBehaviour {

      SerialPort stream = new SerialPort("COM4", 9600); //Set the port (com4) and the baud rate (9600, is standard on most devices)
      float[] lastRot = {0,0,0}; //Need the last rotation to tell how far to spin the camera
      
      
      void Start () {
            stream.Open(); //Open the Serial Stream.
      }
      
      // Update is called once per frame
      void Update () {
            string value = stream.ReadLine(); //Read the information
            string[] vec3 = value.Split(','); //My arduino script returns a 3 part value (IE: 12,30,18)
            if(vec3[0] != "" && vec3[1] != "" && vec3[2] != "") //Check if all values are recieved
            {
                  transform.Rotate(                  //Rotate the camera based on the new values
                                                float.Parse(vec3[0])-lastRot[0], 
                                                float.Parse(vec3[1])-lastRot[1], 
                                                float.Parse(vec3[2])-lastRot[2], 
                                                Space.Self
                                          );
                  lastRot[0] = float.Parse(vec3[0]);  //Set new values to last time values for the next loop
                  lastRot[1] = float.Parse(vec3[1]);
                  lastRot[2] = float.Parse(vec3[2]);
                  stream.BaseStream.Flush(); //Clear the serial information so we assure we get new information.
            }
      }
      
      void OnGUI()
      {
            string newString = "Connected: " + transform.rotation.x + ", " + transform.rotation.y + ", " + transform.rotation.z;
            GUI.Label(new Rect(10,10,300,100), newString); //Display new values
            // Though, it seems that it outputs the value in percentage O-o I don't know why.
      }
}

Arduino test code:

float value = 0;
boolean flip = false;

void setup()
{

  Serial.begin(9600);
}

void loop()
{
  if(value >= 1023)
  {
      flip = true;
  }else if(value <= 0)
  {
      flip = false;
  }

  if(flip == false)
  {
     value += 1; 
  }else{
     value -= 1; 
  }

  float values[] = {0,0,value};

  Serial.flush(); //clears any possible left over information
  Serial.print(map(values[0],0,1023,0,360)); //Resizes values from the range on arduino input to 360 of rotation.
  Serial.print(",");
  Serial.print(map(values[1],0,1023,0,360));
  Serial.print(",");
  Serial.print(map(values[2],0,1023,0,360));
  Serial.println();

  delay(10);
}

What's the practical use? Well, I want to stick a screen to my face and the arduino to the top of my head and get some VR kind of effects from it. Wouldn't be too bad to get true VR effects for less then $200. Add to that a decent controller and a great game and you have the VR of the future!

I'd also like to be able to use my arms and such in this kind of system, but that's going a little overboard at this point. (though, with some bend sensors and some potentiometers I could probably create a decent control system)

So, let me know what you guys think! And, yeah... I don't have an accelerometer yet. Bought one today though, so it should be here with in the week. :slight_smile:

very interesting.
Please report on any achievements-
d

Ok, so it turns out, while my arduino code was fine, my unity3d code was messing things up. :slight_smile: But it's working now... I just need a digital compass, or some way to tell I'm looking left and right (not a tilt left and right, but a rotate left and right)

:slight_smile: But as it is, it works great. With the Mimo monitor hooked up though, the arduino's reporting of the accelerometer was lagging by almost 3 seconds. Simple fix was increase the delay (from 10 to 30) and increase the baud from 9600 to 38400 Now there is a little lag, but that's due to the smoothing process in unity.

I'd give a video demo, but I don't have the best camera in the world, and as such, nothing would be seen probably.

Anyway, here is the arduino code:

void setup()
{
  Serial.begin(38400);
}

void loop()
{
  float values[] = {analogRead(0),analogRead(1),analogRead(2)};
  Serial.flush();
  Serial.print(map(values[0],0,1023,0,359));
  Serial.print(",");
  Serial.print(map(values[1],0,1023,0,359));
  Serial.print(",");
  Serial.print(map(values[2],0,1023,0,359));

  Serial.println();
  delay(30);
}

and here the unity3d code:

using UnityEngine;
using System.Collections;
using System.IO.Ports;

public class serialRotation : MonoBehaviour {

      SerialPort stream = new SerialPort("COM4", 38400); //Set the port (com4) and the baud rate (9600, is standard on most devices)
      float[] lastRot = {0,0,0}; //Need the last rotation to tell how far to spin the camera
      Vector3 rot;
      Vector3 offset;
      
      
      void Start () {
            stream.Open(); //Open the Serial Stream.
      }
      
      // Update is called once per frame
      void Update () {
            string value = stream.ReadLine(); //Read the information
            string[] vec3 = value.Split(','); //My arduino script returns a 3 part value (IE: 12,30,18)
            if(vec3[0] != "" && vec3[1] != "" && vec3[2] != "") //Check if all values are recieved
            {                  
                  rot = new Vector3(float.Parse(vec3[0]),float.Parse(vec3[1]),float.Parse(vec3[2]));
                              //Read the information and put it in a vector3
                  transform.rotation = Quaternion.Slerp(transform.rotation, 
                                                                              Quaternion.Euler(0,rot.x,rot.y),
                                                                              Time.deltaTime*3);
                              //Take the vector3 and apply it to the object this script is applied.
                  stream.BaseStream.Flush(); //Clear the serial information so we assure we get new information.
            }
      }
      
      void OnGUI()
      {
            string newString = "Connected: " + transform.eulerAngles;
            GUI.Label(new Rect(10,10,300,100), newString); //Display new values
            GUI.Label(new Rect(10,30,300,100), "\t" + rot);
      }
}

:smiley: I'm so happy it works ;D

Here you can find some Unity-Arduino examples. Doesn't work on osx yet, suggestions are welcome.

http://code.google.com/p/unity-arduino-serial-connection/

master(s)

++

I posted it on a blog.

((Sorry about bringing up a dead post, but it is relevent as it updates the code to be more efficient.))

So, I've gotten some free time, and have pushed into this a little more. I got upset recently with the code above due to the fact that it's not very great. Pretty much if it doesn't quite fit in at the right time on both systems, it drops information. So, I've been redoing it a bit.

I'm using a custom handshake of sorts this time. Effectively, the arduino waits to get any data from the serial port (doesn't matter what at this point) then sends data back. This way it's only talking when data is requested.

I know the arduino end works, but the unity end keeps breaking. (I think because I'm in a 64bit os.) You can try what I have if you want.

Arduino (using a wiichuck adapter. You can change that part to whatever you want.)

#include <Wire.h>
#include <string.h>
#include "nunchuck_funcs.h"

byte accx,accy,zbut,cbut;

int joyx, joyy;

void setup()
{
    Serial.begin(19200);
    nunchuck_setpowerpins();
    nunchuck_init(); // send the initilization handshake
}

void loop()
{
  if(Serial.available()>0)
  {
    Serial.flush();
    nunchuck_get_data();
    zbut = nunchuck_zbutton();
    cbut = nunchuck_cbutton(); 
    joyx = nunchuck_joyx();
    joyy = nunchuck_joyy();
    if(joyx <100) //ADDED TO KEEP BYTE SIZES THE SAME
    {
      Serial.print("0");
    }
    Serial.print((int)joyx);
    Serial.print(",");
    if(joyy <100) //ADDED TO KEEP BYTE SIZES THE SAME
    {
      Serial.print("0");
    }
    Serial.print((int)joyy);
    Serial.print(",");
    Serial.print((int)zbut);
    Serial.print(",");
    Serial.print((int)cbut);
    Serial.println();
  }
}

AND here is the c# stuff in unity that should work if you have a 32bit os. (I may try some compatibility mode stuff to get it working for me.)

using UnityEngine;
using System.Collections;
using System.IO.Ports;

public class arduinoSerialCommunication : MonoBehaviour {

      SerialPort stream = new SerialPort("COM3", 19200); //Set the port (com3) and the baud rate (9600, is standard on most devices)
      string receivedData = "EMPTY";
      public string sendToArduino = "1";

      void Start () {
            stream.Open(); //Open the Serial Stream.
            stream.Write("1");
      }

      void Update () {
            if(stream.BytesToRead >=11)//my arduino will always send back 11 bytes of information, so wait til it's that large to avoid dropping data.
            {
                  receivedData = stream.ReadLine();
                  stream.WriteLine(sendToArduino);
            }
      }

      void OnGUI()
      {
            GUI.Label(new Rect(10,10,300,100), receivedData); //Display new values
      }
}

Great! Any news?