Temperature array, best way to to recognize humans?

Hii guys,

I’m looking for the best way to recognize humans via an IR sensor. I have the Sparkfun MLX9064 (110 degrees) and it works really well. It’s talks trough I2C and it’s giving me an array of 768 pixels with the temperature for every pixel (32x24 pixels image).

Now I’m brainstorming about the best way to recognize how many people are inside one room. Maybe something like this:

→ Calculate min and max temps
→ Recognize warmer temperatures
→ Recognize how many pixels on the same spot are above average temps (one human)

But how to skip the pixels that belong to one person…?

I’m just wandering if anybody got a great idea (can be something completely different). Let’s brainstorm together :slight_smile:

Just for your understanding, this is picture you get from the sensor (example code, one guy upperright corner sitting, one guy holding the sensor left and just cups of coffee and laptops on an table):

Code so far:

#include <Wire.h>

#include "MLX90640_API.h"
#include "MLX90640_I2C_Driver.h"

const byte MLX90640_address = 0x33; //Default 7-bit unshifted address of the MLX90640

#define TA_SHIFT 8 //Default shift for MLX90640 in open air

float mlx90640To[768];
paramsMLX90640 mlx90640;

//float TotaalWaarde;
float PixelWaarde=0; 
float GemiddeldeWaarde=0;
float MinPixelWaarde= 100;
float MaxPixelWaarde=0;

void setup()
  Wire.setClock(400000); //Increase I2C clock speed to 400kHz

  Serial.begin(115200); //Fast serial as possible
  while (!Serial); //Wait for user to open terminal
  //Serial.println("MLX90640 IR Array Example");

  if (isConnected() == false)
    Serial.println("MLX90640 not detected at default I2C address. Please check wiring. Freezing.");
    while (1);

  //Get device parameters - We only have to do this once
  int status;
  uint16_t eeMLX90640[832];
  status = MLX90640_DumpEE(MLX90640_address, eeMLX90640);
  if (status != 0)
    Serial.println("Failed to load system parameters");

  status = MLX90640_ExtractParameters(eeMLX90640, &mlx90640);
  if (status != 0)
    Serial.println("Parameter extraction failed");

  //Once params are extracted, we can release eeMLX90640 array

  //MLX90640_SetRefreshRate(MLX90640_address, 0x02); //Set rate to 2Hz
  MLX90640_SetRefreshRate(MLX90640_address, 0x03); //Set rate to 4Hz
  //MLX90640_SetRefreshRate(MLX90640_address, 0x07); //Set rate to 64Hz

void loop()
  while (Serial){
  for (byte x = 0 ; x < 2 ; x++)
    uint16_t mlx90640Frame[834];
    int status = MLX90640_GetFrameData(MLX90640_address, mlx90640Frame);

    float vdd = MLX90640_GetVdd(mlx90640Frame, &mlx90640);
    float Ta = MLX90640_GetTa(mlx90640Frame, &mlx90640);

    float tr = Ta - TA_SHIFT; //Reflected temperature based on the sensor ambient temperature
    float emissivity = 0.95;

    MLX90640_CalculateTo(mlx90640Frame, &mlx90640, emissivity, tr, mlx90640To);
  float TotaalWaarde=0;
  for (int x = 0 ; x < 768 ; x++)
    //if(x % 8 == 0) Serial.println();
    PixelWaarde = mlx90640To[x];
    TotaalWaarde += PixelWaarde;
    if (PixelWaarde < MinPixelWaarde){
      MinPixelWaarde = PixelWaarde;
    if (PixelWaarde > MaxPixelWaarde){
      MaxPixelWaarde = PixelWaarde;

  GemiddeldeWaarde = TotaalWaarde / 768;
  Serial.print("Min Temperature measured: ");
  Serial.println (MinPixelWaarde);
  Serial.print("Max Temperature measured: ");
  Serial.println (MaxPixelWaarde);
  Serial.print("Average temperature: ");
  delay (5000);

//Returns true if the MLX90640 is detected on the I2C bus
boolean isConnected()
  if (Wire.endTransmission() != 0)
    return (false); //Sensor did not ACK
  return (true);

Totally impossible to determine and actual person as opposed to a chair where the person has been setting for 5 minutes. Both radiate the same IR for several minutes.


Totally impossible to determine and actual person as opposed to a chair where the person has been setting for 5 minutes. Both radiate the same IR for several minutes.


Did some tests, that's not really a problem :slight_smile: just looking for a way to count people with it

Count hot blobs, taking size and distance into account. Or use machine learning techniques to train a neural network on a large representative data set.

Quite a difficult problem.

just looking for a way to count people with it

To count how many people are in the room? Or to determine that there is at least one person?

Two people embracing will look just like one large person to an IR camera.

Look into grass-fire algorithm to determine the blobs from the heat map.

This is one of those image-recognition problems that appears easy but is actually very difficult for a computer to do.

Distance is one of the big variables that can't be easily eliminated. In your example picture, the guy down the back is a smaller blob than the hands of the guy in the front. So how do you not count the hot-spot of "hands" but count the person-with-their-back-to-the-camera?

Is the Pixy camera code open-source? I think the algorithms the Pixy people have would help here.