Tracking a person's motion with a thermal sensor?

Hello
I'd like to have something or other (a head, a gun, whatever) that turns to face a person and then follows that person's motion as they move across in front of the head. To detect a person, PIRs seemed like an obvious choice, but I don't seem to be able to determine position accurately. This video

shows an array of PIRs, but doesn't seem to be terribly accurate either. So I think I'd use a couple of PIRs for initiating the movement of the head/gun - if someone is detected on the right, say, the head/gun initially turns to the right to look for whatever was detected by the right-hand PIR. I've experimented with ultrasonic sensors and can see that constantly shifting a sensor left and right to home in on the shortest distance can work, though obviously results in disconcertingly jittery movement, so I figured that using three sensors, checking whether the left or right one returns a shorter distance to an object and therefore moving left or right to bring the shortest distance back to the centre sensor would work. I've tried it out simply by shifting sensors around on my desk and observing the results and think it would work. In fact the idea seems to be confirmed by the concept underlying this sculpture: Chris Eckert In the Studio: Sculpture R&D. However, the problem with ultrasonic sensors and with the IR sensors used in the sculpture is that they don't differentiate between human beings and other objects, so the head, gun or whatever could end up targeting a chair or desk lamp. So I wondered if anyone out there has any experience of using a thermal sensor for tracking movement of bodies emitting thermal radiation. This seems like a possible candidate: http://uk.farnell.com/omron-d6t-mems-thermal-ir-sensor. The statement:

Signals generated by the sensor output allow temperature information to be assigned to a particular cell (1x8=8 or 4x4=16 pixels) meaning that the sensor can determine the position of any sensed object.

is intriguing as it suggests it would be possible to detect temperatures detected by different cells and then turn the sensor until the highest temperature is detected by a central cell. Could it work with an Arduino? Does my approach seem feasible or am I just barking up the wrong tree?
Thanks.

The thermal sensor looks intriguing. You will have to look carefully at the data sheet to see what the angular field of view of each sensor is (and how much overlap), but it might work. I have been working on a similar project and will look at the data sheet myself.

However, I thought I would toss out another idea, which worked for me. I mounted two photocells in black tubes about 5 cm long, and mounted the parallel assembly on a servo. The idea was that the servo would rotate to balance the output of the photocells, and that this would be part of a head on a sculpture.

It worked to detect humans moving across the room but the result was surprising depending on how the photocells were wired, or how the response was programmed. A person with clothing darker than the surroundings would be "attractive" or "repulsive" depending on whether the assembly was programmed to rotate towards lower or higher light levels. But it did track! Conversely a person with lighter clothing would elicit the opposite response. One solution to this is variability would be to have the viewers naked :wink:

I prefer the "head" over the "gun" and think most people would feel similarly.

Thanks for your input. Interesting to learn that photocells worked. I had considered that possibility (without any practical experimentation), but had rejected it because, as regards tracking, they seem to be used mainly for line-following or light-seeking robots, where the photocell is pretty close to the light source and seems to require a marked contrast between light and dark.

I did have a look at the data sheet for the temperature sensor, but, as I’m new to this electronics stuff, most of it was gobbledegook to me.

I’d also prefer a head to a gun, but, while researching motion tracking on the Web, I noticed that there seems to be much more interest in controlling Nerf guns and other ‘sentries’ than in rotating a head, so thought I’d insert that keyword into my post.

I’d be interested to know what you make of the D6T-44L-06 datasheet. In comparison with other Arduino add-ons, it’s a pretty expensive sensor, so I’d prefer to save my money if it’s likely to be a dead end.

This is a system I worked on some years ago:

Impressive! The graphic on the left shows very accurate tracking. So there's mileage in using a thermal sensor, it seems, perhaps just not the one I can afford.

Either Omron sensor would probably do you want and the price doesn't seem that bad -- around US$ 50 at Mouser. The angular resolution seems similar to the thermal imager used in the IR Blue project, but with fewer pixels and half the price. If you are not familiar with the IR Blue imager, this link will show you what to expect in terms of thermal image resolution: http://www.rhworkshop.com/pages/product-ir-blue-dm

The 1x8 sensor looks promising to me, assuming that viewers would all be standing.

Thank you for your feedback. I hadn't come across the IR-Blue Project. The video makes it look very promising for my purposes, so I've ordered the Omron sensor. It will be interesting to see if I can make any sense of it, as I know very little of electronics and, though I'm familiar with Java, PHP and VB, C++ seems a bit brain-bending in places. Still, it's never too late to learn, I guess, even for an old age pensioner. Again, thanks for your investigations and your advice.

Hi I'm trying to do the same thing. I have managed to use some code to read the number of pixels of each channel, however, I would like to use this to give temperature measurements. Has anyone got any ideas, how to do this using wire.h

Cheers

Hi
I eventually gave up on this. Having watched Omron’s advertising video, I had naively assumed that they had discovered some deep voodoo which would allow their sensor to detect 37 degrees anywhere within a room, but of course the detected temperature falls with distance and is affected by clothing, and I couldn’t find any way of consistently relating distance (using an ultrasound sensor) to temperature. As far as I went using the 1 x 8 sensor turned on its side, it would fairly reliably detect the presence of a person, and, if they moved out of the way, it would turn in the right direction, but would reliably follow them only as long as they moved obligingly slowly. So now I’m wondering if, using two sonars, I can triangulate their position and move the servo immediately to a specific angle, rather than doing an incremental sweep. I’d use the Omron sensor simply to check that the target is within a likely temperature range. Anyway, although I’ve pretty much given up, I thought I’d share what I’ve discovered in the hope that someone can make better use of it than I have.
Code samples and examples of use: sample code may be found at Thermal IR-Camera - SGMK-SSAM-WIKI and http://bmedesign.engr.wisc.edu/projects/file/?fid=3202
Harness can be purchased from http://international.switch-science.com/catalog/1305/. I placed an order online and they kindly then sent me an email explaining that the web page was only for Japanese users and how I could place an order from the UK. You can, of course, make your own with parts purchased from http://www.digikey.co.uk, but make sure you have an electron microscope handy in case you drop a bit.
Documentation is at http://www.mouser.com/pdfdocs/D6T01_ThermalIRSensorWhitepaper.pdf and http://www.farnell.com/datasheets/1675620.pdf with sample code (I don’t know what language) to talk to the sensor via I2C on page 10 of the former.
Tutorials If you are new to I2C as I was, see Jeremy Blum’s excellent tutorial at http://www.jeremyblum.com/2011/02/13/arduino-tutorial-7-i2c-and-processing/, as well as the introduction to I2C at http://tronixstuff.com/2010/10/20/tutorial-arduino-and-the-i2c-bus/.
Processing code Finally, here’s an Arduino sketch and a Processing sketch which will allow you to see on your computer what the Omron sensor detects. It’s for the 1 x 8 sensor, but, if you use the 4 x 4 sensor, you just need to increase the array size and the loop iterations (see page 10 of http://www.mouser.com/pdfdocs/D6T01_ThermalIRSensorWhitepaper.pdf)
Arduino

#include <Wire.h>
int  rbuf [ 18 ] ;//the buffer storing all of the data we read from the sensor (2 bytes per cell of the column of 8, plus two bytes for the reference temperature and one for the packet error check code)
int  TDATA [ 8 ] ;//contains temperature data for each cell of the sensor's matrix (each being two bytes)
int  T_PTAT;//stores the reference temperature recorded from the thermal sensor ( see http://www.mouser.com/pdfdocs/D6T01_ThermalIRSensorWhitepaper.pdf page 8)

void setup(){
	Serial.begin(9600);
	Wire.begin();
}

void loop(){
	readTemperatures();
}

void readTemperatures(){

	String buf = "";
	int  i = 0 ;
	Wire.beginTransmission(0x0A);
	Wire.write(0x4C);

	int writeStatus = Wire.endTransmission();
	if(writeStatus != 0){
		//Serial.println("Writing failed");
		return;
	}
	delay(100);
	Wire.requestFrom(0x0A, 18);
	for  (i  =  0 ;  i  <  19 ;  i++)  {
		rbuf [i]  =  Wire.read();
	}
	T_PTAT  =  (rbuf[0]+(rbuf[1] << 8));
	for  (i  =  0 ;  i  <  8 ;  i++)  {
		TDATA [i] = (rbuf [(i * 2 + 2)] + ( rbuf [(i * 2 + 3)] << 8 ));//offset by 2 because T_PTAT occupies the first 2 bytes
		buf = buf + (TDATA[i]);
		//I would have expected not to end the string with a comma,
		//but if I don't add a trailing comma, the split function in the Processing sketch
		//won't get the last value as an array element. C++ remains a mystery to me.
		buf = buf + (",");
	}
	Serial.println(buf);

}

Processing

import processing.serial.*;
Serial myPort;  // Create object from Serial class

int [ ]  TDATA  =  new  int[ 9 ] ;//all coming from Arduino as temp * 10, so type can be int and will display 10 times the actual detected temperature
float  Tptat ;
String  buf ;
color [ ]  TColor  =  { #400040, #000080, #006060, #008000, #C0C000, #E0A000 , #E00000, #F08080};

void setup(){
	size(1280,160);
	Serial.list();
	String portName = Serial.list()[1]; //change the 0 to a 1 or 2 etc. to match your port;  0 on Windows is normally COM1. If, say, COM4 is the second available port, you need Serial.list()[1] here.
	myPort = new Serial(this, portName, 9600);
	myPort.bufferUntil('\n');
}

void draw()
{
	int coord = 0;

	for(int i = 0; i < 8; i++){
		if(TDATA[i] < 10){
			fill(TColor[0]);
		}
		else if(TDATA[i]< 50){
			fill(TColor[1]);
		}
		else if(TDATA[i] < 100){
			fill(TColor[2]);
		}
		else if(TDATA[i] < 150){
			fill(TColor[3]);
		}
		else if(TDATA[i] < 200){
			fill(TColor[4]);
		}
		else if(TDATA[i] < 250){
			fill(TColor[5]);
		}
		else if(TDATA[i] < 300){
			fill(TColor[7]);
		}
		else{
			fill(TColor[6]);
		}
		rect(coord * 160, 0, 160, 160);


		textSize(20);
		fill(0, 0, 0);
		text(str(TDATA[i]), i*160+20, 20);
		
		coord++;
	}
}


void serialEvent(Serial myPort){
	buf = myPort.readStringUntil('\n');
	//println(buf);
	TDATA = int(split(buf, ','));
}

Thanks very much for posting this informative update! Sad that the outcome is negative, but then, as I've discovered too, the advertising for these thermal sensors is rather misleading. The wide field of view and consequent averaging over that field severely limits the range and speed with which humans can be detected. The narrow-beam sensors from Melexis might be more useful, but I doubt it.

Hi wittgenstein,

Thanks for your post on human sensor you had worked on.
Even I am trying similar project, but one human at a time. I would just need to see if there is human presence.
I am using the same sensor 1x8.

My question is

  1. You got two sketches, one arduino and processing. why you didnt combine them?
  2. Does it display which pixel detected the human?

your reply is much appreciated!

I was looking on the adafruit store and I found a similar sensor but instead, it is 8x8 and it works well with arduinos as far as I can tell.
Another thing that I found that might be useful are sensors of a company called "Heimann sensor" which have resolutions of up to 32x31 and they communicate over SPI, but I could not find anywhere to purchase these sensors.

jremington:
Thanks very much for posting this informative update! Sad that the outcome is negative, but then, as I've discovered too, the advertising for these thermal sensors is rather misleading. The wide field of view and consequent averaging over that field severely limits the range and speed with which humans can be detected. The narrow-beam sensors from Melexis might be more useful, but I doubt it.

This is why the good IR sensors are export controlled, they are used in guided missiles - for instance
thermal cameras for commercial use are restricted to under 10 frames/second.

If you want a narrow field of view you can use an aperture over the sensor, or a polythene lens (cheaper
than germanium!!).

This looks like very elaborate spam. The original thread is nearly 5 years old now.