Using Uno as a data collection interface

Hi all,

Background: I am reviving an old flow visualization system (a laser Doppler velocimeter (LDV)). It's missing a card that attaches to a PC’s motherboard to collect incoming data from the LDV (via a ribbon cable). The card, in conjunction with manufacturer’s software, produces information that includes velocity data. The ribbon cable (between LDV and PC) carries 16 bits with 3 signal lines that communicate with this card/PC. The 16 bits ingest 0’s and 1’s (16 inputs together = “word”), and the 3 signal lines manage the capturing of these words. When the LDV makes “words” ready, the signal lines trigger and notify the card/PC to capture that word/16bits. The 0’s/1’s in these words can be interpreted to velocity via a formula. LDV can output data at a rate of up to 287,000 words per second.

Problem: The interface card is missing and, frankly, a bit outdated. The software is also outdated, though it could be utilized with some tinkering (but rather bypass having to use this software).

Current Solution: I’m trying to use an Arduino Uno to serve in place of this card, using the digital i/o’s to capture this data and send it to the PC so it can be output in some sort of spreadsheet, where I can manipulate the 0’s/1’s via a given formula to acquire velocity data. I’ve tried to acquire data by reading the HIGH/LOWs of the inputs to the Uno to import into a spreadsheet, and used port registers to make it read faster. I can read an input and write it back out, but nowhere near where I need to be. I have a previous post on the port registers portion, and got great answers, but I'm stuck in terms of the “bigger picture.”

Questions: What commands/code should I use to get the Uno to: 1) Ingest the 0’s/1’s quickly and 2) send it to the PC to a spreadsheet, while 3) being able to capture up to 287,000 words per minute? 4) How can I then ingest it into a spreadsheet, and 5) Is the Uno even handle a 287,000 word/min rate, and how do I determine that?

Here’s the current code. It can only capture a very low amount of inputs at a time, haven’t tested anything faster… Not even sure if it's feasible for what I'm aiming to do...

void setup() { pinMode(2,INPUT_PULLUP); pinMode(3,INPUT_PULLUP); pinMode(4,INPUT_PULLUP); pinMode(5,INPUT_PULLUP);

DDRB = B00001111; }

void loop() { byte x = PIND; //PIND reads PORT D pins // DDDDDDDD // 76543210 x = x & B00111100; x = x >> 2; PORTB = x; }

Thoughts?

287,000 16 bit words/minute is less than 10,000 bytes/second. That amount of data can be transmitted over a serial connection to a PC using a baud rate of 115200.

The question is how to read out the cable and assemble the 16 bit words at that rate, but it seems doable. At the moment, you are reading only 4 bits. How would you get the rest?

Hmm, so are you saying that serial reading (i.e. SerialRead, etc) is sufficient to read all the 16 bits per 287,000 words/min without having to deal with port registers? (Sorry, I'm not the best at figuring out data transfer rates)

The current code was a test to see if I could get a read on one of the ports (port D) and output it another port (to port B) to see if it reads out properly. It works, so I guess it proves reading it by port registers is possible (to increase the read speed). My intention was to read each port at a time (rather than one bit at a time) so that instead of reading 16 separate bites, it would only have to read 3 separate ports (reducing the amount of steps, because I initially wasn't sure if the Uno could keep up with 16 bits/word at 287,000 words/min). However, what you said about the transmitted data being doable serially at these rates.. does that mean I don't need to use port registers?

My train of thought, after the Uno captures these bits and sends it to the PC (if serially) via the USB cable, I would also need to develop a Processing script to capture these bits and store it in some sort of spreadsheet. In this spreadsheet, I can then manually enter formulas and get useful data for each 'word'.

Thoughts? What would be the easiest way to acquire these 16 bits quickly and get it into a spreadsheet? If the data transfer rates seem okay, should I be sticking with Serial.Read and Serial.Write commands?

Hi waynewayne

@jremington was talking, I think, about the data transfer from Arduino to PC being feasible using serial.

Reading the 16 input lines could be done either by reading Arduino ports, or with a shift register (one 16-bit or two 8-bit ones daisy-chained) that you latch and shift in to the Arduino when the LDV signals that a word is ready.

If using Arduino ports and a Uno, you would need to split the 16 bits across the three registers, due to pins used for other purposes / not brought out to the headers. On a Mega, you could read the 16 bits in two registers.

Regards

Ray

Hello Hackscribble,

By shift register, do you mean: https://www.sparkfun.com/products/733

Is there anyway to do it just from the Uno’s I/O pins? I only need to “read” the pins and capture the 0’s/1’s in a spreadsheet or some sort of text format

How feasible is using the Mega to read the 16 bits in two registers? How would I go about doing that?

Thanks!
Wayne

With Uno, use 74HC165, parallel load shift register, and SPI.transfer to bring the data in quick.

PORTD = PORTD & 0b11111011; // clear bit 2
PORTD = PORTD | 0b00000100; // set bit 2, loads data into the shift registers
lowByte = SPI.transfer(0); // read 1st byte, SCK drives clock, MISO accepts the data, MOSI outputs 0x00
highByte = SPI.transfer(0); // read 2nd byte

Works with Mega also. Select a convenient pin for the load line.
SS must be an output for the Mega to remain SPI master.

By shift register, do you mean: https://www.sparkfun.com/products/733

The Sparkfun shift register is a serial in, parallel out type. As @Crossroads says, you would need a parallel in, serial out type like 74HC165.

If you absolutely want to avoid additional hardware, using just a Uno may be possible and a Mega almost certainly. Can you explain more about how the 3 signal lines work? How does the LDV signal that a word is ready? Will the Uno need to signal anything back to the LDV?

So if jremington's math is correct, implying that I can transfer this data at 16 bits/word, and 287,000 words/second, can I avoid the hassle of using port registers, and just read each data bit one-by-one?

Here's my attempt to doing that (see code below). I'm not very confident with my coding (beginner here) and I'm sure there are inefficiencies/redundancies in my code. Can you guys take a look and let me know if it makes sense?

[u]FYI[/u] DBxx: DataBit number (i.e. from databits 00 to 15 for all 16 bits) CPUReady: sends a signal to LDV to let LDV know that the PC/card/Uno is ready for data CReq: Cycle Request. this is a signal from the LDV to the PC/card/Uno to notify the PC/card/Uno that a "word" is ready for collection. Tricky thing about this is that CReq is on for about 600ns, can the Uno capture such a short signal? DataHold: when the PC/card/Uno gets a positive CReq signal from the LDV, the PC/card/Uno will trigger this DataHold line that runs to the LDV to notify the LDV to "hold" the current word (to give the PC/card/Uno enough time to collect it). I assume when the PC/card/Uno turns this DataHold off, the LDV will proceed to load the next word

PC/card/Uno: represents the PC/Uno setup where the missing card (original previous interface card) that the Uno replaces (to replace the missing interface card) is connected to.

Also, I'm trying to ultimately print the 0's/1's into a .csv (comma separated value) file so I can import it into excel. The second part of this project will include writing the Processing code for the PC so it can import the .csv and create an excel format file. Is this a good idea?

int DB00 = 0
int DB01 = 1
int DB02 = 2
int DB03 = 3
int DB04 = 4
int DB05 = 5
int DB06 = 6
int DB07 = 7
int DB08 = 8
int DB09 = 9
int DB10 = 10
int DB11 = 11
int DB12 = 12
int DB13 = 13
int DB14 = A00
int DB15 = A01
int CPUReady = A02
int DataHold = A03
int CReq = A04


void setup() 
{
  Serial.begin(9600); 
  
  pinMode(DB00, INPUT) //using this (and below) to set all data pins to be inputs 
  pinMode(DB01, INPUT)
  pinMode(DB02, INPUT)
  pinMode(DB03, INPUT)
  pinMode(DB04, INPUT)
  pinMode(DB05, INPUT)
  pinMode(DB06, INPUT)
  pinMode(DB07, INPUT)
  pinMode(DB08, INPUT)
  pinMode(DB09, INPUT)
  pinMode(DB10, INPUT)
  pinMode(DB11, INPUT)
  pinMode(DB12, INPUT)
  pinMode(DB13, INPUT)  
  pinMode(DB14, INPUT)
  pinMode(DB15, INPUT)
  pinMode(CPUReady, OUTPUT) //from machine to LDV system, to notify readiness
  pinMode(DataHold, OUTPUT)  // it should be internal, right?
  pinMode(CReq, INPUT) //600ns signal from LDV system to notify the arduino to begin reading (Polled Mode)
  
  digitalWrite(CPUReady, HIGH); //to let LDV know PC/Uno is ready to begin collection

}



void loop() {

bit00 = digitalRead(DB00) //bit## pin names
bit01 = digitalRead(DB01) //bit## pin names
bit02 = digitalRead(DB02) //bit## pin names
bit03 = digitalRead(DB03) //bit## pin names
bit04 = digitalRead(DB04) //bit## pin names
bit05 = digitalRead(DB05) //bit## pin names
bit06 = digitalRead(DB06) //bit## pin names
bit07 = digitalRead(DB07) //bit## pin names
bit08 = digitalRead(DB08) //bit## pin names
bit09 = digitalRead(DB09) //bit## pin names
bit10 = digitalRead(DB10) //bit## pin names
bit11 = digitalRead(DB11) //bit## pin names
bit12 = analogRead(DB12) //bit## pin names
bit13 = analogRead(DB13) //bit## pin names
bit14 = analogRead(DB14) //bit## pin names
bit15 = analogRead(DB15) //bit## pin names
bitCPUReady = analogRead(CPUReady)      //bit name for Ready signal
bitDataHold = analogRead(DataHold)    //bit name for DataHold signal
bitCReq = analogRead(CReq)              //bit name for Cycle Request signal


if (bitCPUReady > 0)


  if (bitCReq > 0) //need to detect 600ns signal 
  {
  
    digitalWrite(bitDataHold, HIGH);
    
    Serial.print(bit00); //sets the 16 bits into a series of 0/1's separated by commas
    Serial.print(",");
    Serial.print(bit01);
    Serial.print(",");
    Serial.print(bit02);
    Serial.print(",");
    Serial.print(bit03);
    Serial.print(",");
    Serial.print(bit04);
    Serial.print(",");
    Serial.print(bit05);
    Serial.print(",");
    Serial.print(bit06);
    Serial.print(",");
    Serial.print(bit07);
    Serial.print(",");
    Serial.print(bit08);
    Serial.print(",");
    Serial.print(bit09);
    Serial.print(",");
    Serial.print(bit10);
    Serial.print(",");
    Serial.print(bit11);
    Serial.print(",");
    Serial.print(bit12);
    Serial.print(",");
    Serial.print(bit13);
    Serial.print(",");
    Serial.print(bit14);
    Serial.print(",");
    Serial.println(bit15); //last line needs to be a println (print line, not sure why)

    //this prints the 16 bits into "words" in a .csv format (comma separated)
    
   digitalWrite(bitDataHold, LOW);

  }

  else
  {
  //nothing  
  }

else
{
 //nothing 
}


}

Thanks CrossRoads, thanks Hackscribble! Good stuff and I think I'm trying to see if I can get the Uno to work before exploring the shift registers (I'm still trying to get my head wrapped around it). You guys beat me to my post, just previously, where I did my best to clarify the logic of what I'm trying to do. What do you guys think of the code and approach? Let me know!! :)

Do you have a manual or data sheet for the LDV that shows timing diagrams for the control signals?

Sending the bits as '0' and '1' characters will slow down the Arduino - PC transfer rate significantly. The calculations in the earlier posts assumed 2 bytes per word to carry the 16 bits, with formatting done on the PC.

In practice, you would probably need to include some extra marker bytes to keep the PC program in sync with the Arduino.

If you have a Mega or as '1284P with access to full 8 bit ports, I'd go with that.

// detect when data is valid based on control signals
:
:
// then read two ports and send it out
lowerByte = PINF;
upperByte = PINK;

Serial.write (lowerByte};
Serial.write (upperByte);

// on to next read

I have a LOT of manuals on the LDV, but not sure what you mean by timing diagrams for control signals? Can you elaborate it in a way that helps me narrow down where to be looking?

So I should be using the port registers, as referred to, in the earlier posts?

And what do you mean by extra marker bytes? Like some separator indicators between "words"?

(Sorry, I'm not exactly a programmer...)

CrossRoads, thanks for the code! I actually have an Uno, but would you suggest that a Mega may be more useful/versatile than the Uno? I have been stuck in the Uno approach, since it's still pretty new to me, but if you suggest the Mega is better, more powerful, etc., then I wouldn't mind ordering one to drive closer to a solution. Would you recommend me to pick up a Mega?

And what do you mean by extra marker bytes? Like some separator indicators between “words”?

That’s right. The challenge is how to distinguish the marker bytes from real data, since (presumably) the 16-bit word can take on any value?

Can you give a bit more background on how you plan to use the LDV? How long will it be running (at 287,000 words/minute) before you stop reading data and move over to analysing it on the PC?

I have a LOT of manuals on the LDV, but not sure what you mean by timing diagrams for control signals? Can you elaborate it in a way that helps me narrow down where to be looking?

Have a look at page 5 of the attached datasheet for an example. Tables of timing limits and waveform diagrams to show the relationship between signal lines and data lines.

MCP9804.pdf (593 KB)

How about this revision. FYI I’m only adding the Serial.print(“begin_word”); and the Serial.print(“end_word”); to the printing section just to help distinguish the start/end of a word. Would this help?

int DB00 = 0
int DB01 = 1
int DB02 = 2
int DB03 = 3
int DB04 = 4
int DB05 = 5
int DB06 = 6
int DB07 = 7
int DB08 = 8
int DB09 = 9
int DB10 = 10
int DB11 = 11
int DB12 = 12
int DB13 = 13
int DB14 = A00
int DB15 = A01
int CPUReady = A02
int DataHold = A03
int CReq = A04


void setup() 
{
  Serial.begin(9600); 
  
  pinMode(DB00, INPUT) //using this (and below) to set all data pins to be inputs 
  pinMode(DB01, INPUT)
  pinMode(DB02, INPUT)
  pinMode(DB03, INPUT)
  pinMode(DB04, INPUT)
  pinMode(DB05, INPUT)
  pinMode(DB06, INPUT)
  pinMode(DB07, INPUT)
  pinMode(DB08, INPUT)
  pinMode(DB09, INPUT)
  pinMode(DB10, INPUT)
  pinMode(DB11, INPUT)
  pinMode(DB12, INPUT)
  pinMode(DB13, INPUT)  
  pinMode(DB14, INPUT)
  pinMode(DB15, INPUT)
  pinMode(CPUReady, OUTPUT) //from machine to LDV system, to notify readiness
  pinMode(DataHold, OUTPUT)  // it should be internal, right?
  pinMode(CReq, INPUT) //600ns signal from LDV system to notify the arduino to begin reading (Polled Mode)
  
  digitalWrite(CPUReady, HIGH); //to let LDV know PC/Uno is ready to begin collection

}



void loop() {

bit00 = digitalRead(DB00) //bit## pin names
bit01 = digitalRead(DB01) //bit## pin names
bit02 = digitalRead(DB02) //bit## pin names
bit03 = digitalRead(DB03) //bit## pin names
bit04 = digitalRead(DB04) //bit## pin names
bit05 = digitalRead(DB05) //bit## pin names
bit06 = digitalRead(DB06) //bit## pin names
bit07 = digitalRead(DB07) //bit## pin names
bit08 = digitalRead(DB08) //bit## pin names
bit09 = digitalRead(DB09) //bit## pin names
bit10 = digitalRead(DB10) //bit## pin names
bit11 = digitalRead(DB11) //bit## pin names
bit12 = analogRead(DB12) //bit## pin names
bit13 = analogRead(DB13) //bit## pin names
bit14 = analogRead(DB14) //bit## pin names
bit15 = analogRead(DB15) //bit## pin names
bitCPUReady = analogRead(CPUReady)      //bit name for Ready signal
bitDataHold = analogRead(DataHold)    //bit name for DataHold signal
bitCReq = analogRead(CReq)              //bit name for Cycle Request signal


if (bitCPUReady > 0)


  if (bitCReq > 0) //need to detect 600ns signal 
  {
  
    digitalWrite(bitDataHold, HIGH);
    
    Serial.print("begin_word");
    Serial.print(",");   
    Serial.print(bit00); //sets the 16 bits into a series of 0/1's separated by commas
    Serial.print(",");
    Serial.print(bit01);
    Serial.print(",");
    Serial.print(bit02);
    Serial.print(",");
    Serial.print(bit03);
    Serial.print(",");
    Serial.print(bit04);
    Serial.print(",");
    Serial.print(bit05);
    Serial.print(",");
    Serial.print(bit06);
    Serial.print(",");
    Serial.print(bit07);
    Serial.print(",");
    Serial.print(bit08);
    Serial.print(",");
    Serial.print(bit09);
    Serial.print(",");
    Serial.print(bit10);
    Serial.print(",");
    Serial.print(bit11);
    Serial.print(",");
    Serial.print(bit12);
    Serial.print(",");
    Serial.print(bit13);
    Serial.print(",");
    Serial.print(bit14);
    Serial.print(",");
    Serial.print(bit15);
    Serial.print(",");
    Serial.println("end_word"); //last line needs to be a println (print line, not sure why)

    //this prints the 16 bits into "words" in a .csv format (comma separated)
    
   digitalWrite(bitDataHold, LOW);

  }

  else
  {
  //nothing  
  }

else
{
 //nothing 
}


}

I plan on using the LDV in an experiment on high-speed airflow. Essentially testing the airflow in a scramjet combustion chamber per different geometries. The LDV will be collecting velocity data in a sub-sonic region of this chamber as well as the supersonic region. The source of the air-flow will be from a shock-tube. Some people use supersonic wind tunnels in this area of study, but I don’t have access to one, so I’m using a shock tube instead. A quick nut-shell of how a shock tube works: It’s a long tube (varying in sizes) where a diagraphm divides the interior into two sections. There is a high pressure side and a a low pressure side. To operate, air/gas is pumped into the high pressure side to a determined pressure level. The diagraphm is bursted, either by exceeding its pressure limit, or by a controlled charge, thus sending a sudden burst of air downstream sending a shock wave into a test chamber. The shock wave brings a supersonic air flow that the test chamber needs. The downside to this method is the data collection period is very short (on the scale of milliseconds) but it’s been done before.

So to answer your other question. The LDV can be collecting/sending-to-PC at UP TO 287,000 words/min, and the time period would likely be say, 5 seconds or so (arbritrary small number). The actual collection time frame of useful data will probably be less than a second, but I assume from getting reading to collect to complete is around 5 seconds to include starting the collection, trigger the shock tube, collect data, then turn off the collection. Are there any constraints on doing something like this?

Thanks for the example datasheet. It’s strange because I haven’t seen anything like that in my LDV documentations, but I can check again. I’ll get back to you on this!

Also, would the Mega be better than Uno on this project?

Thanks for the background.

Each run is relatively short and it sounds like you can control the start of the LDV, Arduino and PC program to keep them all in step. Also, one missed byte might well be very obvious in the data on the PC, so you could discard the data and do another run.

So, for Arduino to PC, just write two bytes per word. Don't format them as one character per bit - this will slow down the serial data flow. In other words, go with @Crossroads' approach.

You will need some code on the PC to read in the pairs of bytes, store them temporarily and format them for analysis by spreadsheet. A 5 second run will generate less than 50K of data.

On the Arduino, you may need to use an interrupt to trigger the port reads in response to the 600ns pulse from the LDV. In the absence of timing diagrams, is there a written description of the sequence of the three signal lines?

The advantage of the Mega or 1284P is that you can use two ports to read the data without any bit shifting etc, making the code easier to write and faster.

ADDED... Do you have documentation on what the 16 bits represent? Are they are an integer, a float?

Gotcha, So essentially the first 8 bits will be one byte and the second 8 bits will be the second byte? So I should get the Mega? What’s the difference between, say, these three:

http://www.amazon.com/Arduino-Compatible-Atmega2560-Mega2560-Board/dp/B00JTBMD7E/ref=sr_1_3?ie=UTF8&qid=1436987997&sr=8-3&keywords=arduino+mega

Why the drastic price differences? Which one should I get?


As for the interrupt function, I’ve gathered that it allows a background code to run while another set of code is running? Is this right and how does it factor into the logic here? Also, how do I figure out if they’re integers or floats?


Additionally, I’ve attached a few “scanned” sections (sorry no scanner, used a camera instead) on the manual sections covering what the 16 bits represent (see attached PDF):
Page 1: covers the pinouts from the LDV. Note that Data Bits from 00 to 15, and the signal lines of CPU Ready, Data Hold, and Cycle Request, as well as the grounds.
Page 2,3: Covers the word transfer sequence (Word A, Word B, Transit Time (also a word) and Word C) in two modes: Random and Coincidence (not yet fully understood). Also still trying to understand the time aspect. Note that “FIND software” is the original manufacturer software that I am trying to bypass by analyzing the actual raw data.
Page 4,5: Breaks down the data format structures of the words into what each bit or set of bits mean. Used to derive different values that can then be plugged into conversion formulas.
Page 6,7: Contains the Velocity formula, how it utilizes the values broken down from the Words (i.e. “nc”, “T mant”, “(exp-3)”).

Does this information help? Let me know what you think!
Thankssss!!

Cropped sections of LDV manual.pdf (790 KB)

I'll look at the links and the PDF.

About the interrupt, it's not about running in the background in this case. I think it may be required because your code needs to respond to a short pulse, 600ns I think you said? Just checking for the level of an input pin in loop() might not be quick enough to detect that, depending on what else is happening in loop(). The interrupt service routine would set a flag that your loop() code would act on to start the transfer.

Not sure what you meant by floats or integers? I guess the documentation you have scanned will say whether the 16-bit words contain floats or integers. Floats would need two words minimum. But your current assumption is Arduino captures raw binary and sends to PC, and PC converts to meaningful data and processes?

The PDF is useful for the format of the data frames (i.e. the collection of words in Fig C2 and C3) and how to interpret and convert the raw binary. It also indicates (on p1) that CPU Ready, Data Hold and Cycle Request are active low.

It also says (pp2-3) that the FIND software is used to set the operating mode and, hence, which type of data frame is sent and whether the frame includes the optional words. Does this mean that the original PC controller had another interface to the LDV to send commands to it?

The PDF doesn't help about the sequence and timing of the three control signals. In an earlier post, you said ...

CPUReady: sends a signal to LDV to let LDV know that the PC/card/Uno is ready for data CReq: Cycle Request. this is a signal from the LDV to the PC/card/Uno to notify the PC/card/Uno that a "word" is ready for collection. Tricky thing about this is that CReq is on for about 600ns, can the Uno capture such a short signal? DataHold: when the PC/card/Uno gets a positive CReq signal from the LDV, the PC/card/Uno will trigger this DataHold line that runs to the LDV to notify the LDV to "hold" the current word (to give the PC/card/Uno enough time to collect it). I assume when the PC/card/Uno turns this DataHold off, the LDV will proceed to load the next word

Was this based on tests you have run or is there another source of information?

Googling IFA-750, I found a French company that sells software to interface with the LDV using a more modern hardware interface card. http://www.fist.fr/en/catalogue-cnrs/method-for-preparing-2s-3r-4s-4-hydroxyisoleucine-and-analogues-thereof/