New User - Project Data Collection

Projects General Guidance
1

Hi All

I am a New User to the Arduino, my background is Program Development using VB.net, only used C in small amounts. I have no electronics background but eager to learn.

My project is a Standalone Data Collection system that will send its information VIA wired ethernet to a backend database.

My Design project needs the following.

Processing Unit: Purchased Arduino UNO
LCD Screen for Viewing Information at Unit: Purchased 16x4 Character LCD
Keypad For Menu and Data Entry: Purchased 16 Button Kepad 4 x 4 Matrix
RJ45 Compatible for Network Connection: Purchased Arduino Ethernet Shield
Real Time Clock: Purchased Real Time Clock Module
2 LEDs, 1 Green and 1 Red: Purchased Small 5v Lamps and Housings
1 Photoelectric Sensor: Purchased SICK Photoelectric Sensor 10 - 36v
12v DC Power Supply: Already Have

What the unit will need to do. This unit is a 2 fold operation.
A) To count components that pass the Photosensor
B) To record downtime on the line. So when the Photosensor doesnt see a component after 30 seconds the unit will activate the RED LED and the LCD will Display Production Down and to choose an error. The operator then types in the code via the keypad for the appropriate error which will then be displayed on the LCD. Once the Photosensor detects a component then the system goes back into production, Green LED activates and LCD displays current component total, Date / Time and Production Good.

The unit will have to pass any information back to the database every second, So Component Totals, Production Start with Date and Time, Production Stop with Date and Time, Downtime Reason with Start Date and Time and End Date and Time. The unit must also be able to store all this information in the event that the network goes down and it cant communicate with the Database server, then sends the information when the Network is online.

The following must be configurable at the Unit Itself via a password protected System Menu (Setting my kepad up as a multi key device)
Device Name
IP Address
Database Server Address
Gateway
Subnet Mask
Sensor Timer for setting Production Down
Downtime Errors (This may also be configurable via a web browser on a client PC)
Possibly a few more options not decided on these yet.

I know what it is that I want as a final outcome and have ordered the relevant hardware as above but I may have missed something, if you feel I have missed something please say. I will spend a while doing tutorials and learning how the Arduino works before attempting my final project.

If you have any other thoughts regarding my project please dont hesitate to say.

Thanks for any help you may give me in advance.

Regards

1druid1

2

You might need something to buffer the collected data, in case the network is down or a power failure. Maybe an SD Shield or EEPROM module.

You'll get more help if you provide the links to the datasheets of the components you bought.

3

1 Photoelectric Sensor: Purchased SICK Photoelectric Sensor 10 - 36v

Generally, you want to use 5v sensors when possible. We would have to see a datasheet to know if you can use the above.

4

Hi All

Just received all my components and just by looking I think I have already made a mistake and wish I had posted before ordering, we live and learn. My Arduino UNO has 14 Digital Input / Ouput (s) but I think I need a lot more than that. My Keypad requires 8 inputs, my LCD requires 11 Outputs, I also need to 2 Outputs for the LEDs and 1 Input for the sensor, from that count way above what the UNO can supply. As I am new to all this can you tell me if there is a way around this with the UNO or am I having to go another route.

Datasheets

Keypad: http://www.rapidonline.com/Electronic-Components/Data-entry-keypad-73486/?sid=07bb9930-21e6-460f-a8c5-f1f29136e839
LCD: http://www.hobbytronics.co.uk/datasheets/GDM2004D.pdf
UNO: http://www.hobbytronics.co.uk/arduino/arduino-uno

Cheers

Druid

5

You can explore this solution to solve the keypad problem:
http://www.arduino.cc/playground/Main/I2CPortExpanderAndKeypads

And there are other cheap ICs you can use to get more digital outputs, like the 74HC595. This way you can reduce the LCD pin requirement, for example.
Look arround for shift register solutions in the forums and the playground.

Hope this helps.

6

@Pgmartin - Thanks for the tip, I will look into that.

Ok I have had a little play with the Arduino, and got it to do the following things. Light up multiple LEDs, Print Text on the LCD, Connect to the Network and be able to browse data on the onboard SD Card. Now at the moment I want to play with the Network side of things. What I have done is setup a SQL 2008 Server and created a table with a couple of fields to store information in. If the Arduino was based on VB I think I could get it to write to the database quite easily, but as its based on C and my knowlegde is very limited I am struggling to find a start on how to establish a connection and parse an INSERT query.

Anyone got any lnks, tips or tutorials on this topic, I have used the search but couldnt find anything related to MS SQL and google didnt produce much information either. Any help is appreciated.

Cheers

Druid

7

I suspect that the reason you can't find much on the topic is that implementing a sql library would be too much for the Arduino to handle with its limited memory. What I have seen suggested is to use an intermediary program on the PC to accept your sql statements, call the database and deliver results. A custom VB program to talk to the Arduino over serial would do this. A web service is another possibility, since you already have network connectivity. Easier still is to set up a web page and use GET or POST to send your SQL. Alternatively in this case, you may not necessarily need to send SQL, just sufficient information to let a cgi script know what you're looking for and it can encapsulate the SQL that does it.

8

@wildbill Thanks for the reply, I am beginning to think the Arduino may not be the solution I am looking for.

My solution has to be based on a connection via the Ethernet as these units will not be connected to any PCs they will be standalone. Setting up a web page, wouldnt that then require you to load the webpage or could the Arduino do thats itself, as this has to be an automatic process. Either on a timer every few seconds the arduino send the data to the SQL Database, or via the inputs on the unit itself, Everytime the Sensor is broken or everytime a button is pressed the Arduino sends the information. This is only for an internal network aswell so cant go via the web.

Regards

Druid

9

Druid,

Impressive first post, very clear and logical. I see many first posts that are much less organized. I think you can certainly set up web server on an arduino. Read this, may be useful:

http://www.jo3ri.be/arduino/arduino-projects/network-settings-web-page-form-using-eeprom-to-save-submit

although arduino doesn't have much RAM, if you need to print long strings via ethernet, you can store long string literials in FLASH, which is decent in size 32KB, or use EEPROM or SD card.

For concerns of not having enough connections, you can either use an arduino mega with a lot more pins or use a serial panel that only costs you 2 IO pins and gives you LCD and keypads. Here is what I have in store. It's a serial pad you control via serial.print and sense buttons (if there is button press) via serial.read.

http://www.inmojo.com/store/liudr-arduino-and-physics-gadgets/item/serial-20x4-lcd-keypad-panel---phi-panel/

You can use any 2 pins and NewSofteSerial, a software serial library to communicate serially.

http://arduiniana.org/libraries/newsoftserial/

10

1druid1:
My solution has to be based on a connection via the Ethernet as these units will not be connected to any PCs they will be standalone. Setting up a web page, wouldnt that then require you to load the webpage or could the Arduino do thats itself, as this has to be an automatic process. Either on a timer every few seconds the arduino send the data to the SQL Database, or via the inputs on the unit itself, Everytime the Sensor is broken or everytime a button is pressed the Arduino sends the information. This is only for an internal network aswell so cant go via the web.

I meant send the data to a web page running on a web server on your internal network - that machine can communicate with the SQL database. Is the machine running the database able to run iis? By the sound of it, this may be a work project, so having the authority to mess with the server may be the crucial issue.

11

@wildbill Thanks for clearing that up for me. Yes this is a works project, unfortuantly I cannot setup a web server within our network, company policy. So I need to be able to do this without a webserver. I will keep on scouring the net to see if i can come up with something.

12

Does wireless work for you?

13

@Liudr I would love to use wireless, unfortuantly because of the sheilding on some of the machines wireless is very unstable so it we will have to use a Wired RJ45 connection

14

On the LCD pin requirements, FYI, while for some reason the SparkFun pages say that 11 pins are required, really only 6 are needed. See:

Joe

P.S. Also, you could use a digital encoder chip (74LS148) to encode your 8 keypad bits as 4 bits. So maybe you'd have enough pins on the Uno.

P.P.S. Not sure of the model of real-time clock you got, but this one is pretty inaccurate:

Evil Mad Science has one that advertises +/- 2 minutes per year for the same price:

15

@jmknapp.. Hmmm yes I beleive its the sparkfun module I have, only difference is that the one I have is mounted on a Red PCB and not a Green PCB, do you know how inaccurate it is?

As everything uses digital inputs and outputs and as I have run out, looking through some of the spare parts the engineering has at work I have noticed that we have a couple of 74HC595 and MCP23017-E/SP chips. I dont know much about these apart from they can be used to expand Digital Inputs and Outputs, can I use these in conjuction with the RTC, Sensors, LEDs, Keypad, LCD and Ethernet Sheild to get everything running of the UNO, if so anyone got a wiring example of using them together?

Cheers

Druid

16

Druid,

Yes you can use shift registers to add more digital pins but then you will need to modify the libraries a lot ot make it work. Specific hardware interface like I2c or spi only run on specific pins.

My suggestion still is to get a serial lcd + keypad panel that will save you like 16 pins and only uses 2 pins to control.

What about having arduino stations in the factory and buy a high end micro controller with ethernet to process data? this way each arduino node is just an intelligent datw collection and relay node and a more expensive node processes and does sql stuff? You can duplicate many nodes and they all have identical software and hardware except for different mac address.

17

1druid1:
@wildbill Thanks for clearing that up for me. Yes this is a works project, unfortuantly I cannot setup a web server within our network, company policy. So I need to be able to do this without a webserver. I will keep on scouring the net to see if i can come up with something.

The answer is probably the same, but can you write and install a program on an internal machine that the Arduino can communicate with using tcp or udp? Such a program could handle talking to the database.

18

@Liudr.. I do like your LCD and Keypad combo but at the moment the hardware is bought so want to see what I can do with that first.

As for the Microcontroller, I would rather do as wildbill suggests and write a piece of software that does the relaying, unfortuantly I have a couple of systems that use a polling PC and I want to get away from these systems, hence the reason for the Arduino sending the data instead of the data getting pulled from them. Creating a relaying and polling system just adds another step in the loop that I want to get away from, I know it may not be possible without one and I may end up having to create a VB collection system.

@Wildbill, I wouldnt use UDP, setting up error correction and ensuring the data is received fully is to much of a headache and my programming skills are not that advanced.

I will keep on plugging away and see what I can do.. Please keep your help and suggestions coming as they are great advice and its food for thought, I think I have jumped in feet first with this project whereas I should have posted for advice first :slight_smile: Thanks for all your patience.

Druid

19

Hi All

I am slowly working my way through parts of this project, have been trying to work on the ethernet side of things for the data sync with the main database. Wanting a little break from this so I have setup my keypad, its a 4 x 4 Matrix keypad, unfortuantly the datasheet for it was wrong so it took a little trial and error to get it working correctly. All my buttons now print as they should, but what I would like if someone has it, is the code for the Multi Key version, for example Key 1 = (A,B,C,1) ect.

Cheers

Druid

20

Hahaha, ball back to my court again!

As you may have noticed, my phi-panel has multi-tap integrated! To express the free spirit of open-source software and hardware, I hereby release my multi-tap function into the "wild":

My blog page:

Original arduino forum page (nobody replied saying, yeah! we want the code so I was a bit let down and didn't release my code):
http://arduino.cc/forum/index.php/topic,68986.0.html

I know you are doing your project for your company so it's not exactly "not-for-profit" per my release terms but if you're willing to purchase my panels (in case you like this code), I'd be OK with it :wink:

This was two night's work although I program quite fast :slight_smile: