Data-Logger, need advice!

Projects General Guidance
1

Hi. I need to record the voltage and current of four solar panels, and be able to plot the data on a graph and present it on a website.

Current, my setup is as follow:

Arduino #1 has some current, voltage and temperature sensors with an xBee module. Data are sent to Arduino #2 via serial. Arduino #2 has an LCD screen with buttons. I can view whatever data I want through the LCD screen. Finally, the data will pass from Arduino #2 to Arduino #3 via I2C. When new data arrive, it will write them to the onbaord SD card as a .csv file. Arduino #3 is setup as a webserver, and I am currently trying to make a .htm to present the data.

My question is, do you think there exists a better approach? Is the Ethernet shied good enough to host a website that can read a massive amount of data from the SD card? Should I use a computer like the raspberry pi to do it? I know nothing about networking so I am not sure about the part about setting up a server, making a website, plotting the data, etc.

Any input will be appreciated, thanks!

2

a massive amount of data from the SD card?

Pls specify..

3

Thanks for your speedy response! Here are some sample data:

data.csv

2013,4,6,17,45,0,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.59,0.1,26.67,79.99,24.38,79.99,24.57,76.22,26.19,79.14,21,7,167
2013,4,6,17,45,3,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.59,0.1,26.67,79.99,24.32,79.99,24.57,76.22,26.7,78.92,22,7,183
2013,4,6,17,45,7,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.59,0.1,26.67,79.99,24.32,79.99,24.51,76.11,26.7,78.92,21,7,167
2013,4,6,17,45,11,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.59,0.1,26.67,79.99,24.32,79.99,24.57,76.22,26.1,78.81,21,7,167
2013,4,6,17,45,15,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.59,0.1,26.67,79.99,24.32,79.99,24.57,76.22,26.1,78.81,21,7,167
2013,4,6,17,45,19,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.59,0.1,26.67,79.99,24.38,79.99,24.57,76.22,26.1,78.81,21,7,167
2013,4,6,17,51,18,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,26.13,79.3,23,8,174
2013,4,6,17,51,22,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,26.7,78.92,23,8,174
2013,4,6,17,51,26,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,26.1,78.81,23,8,174
2013,4,6,17,51,30,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,26.1,78.81,23,8,174
2013,4,6,17,51,34,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,26.1,78.81,23,8,174
2013,4,6,17,51,38,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,25.94,78.69,23,8,174
2013,4,6,17,51,41,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.63,76.33,25.94,78.69,23,8,174
2013,4,6,17,51,45,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.94,78.69,23,8,174
2013,4,6,17,51,49,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,25.94,78.69,23,8,174
2013,4,6,17,51,53,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,25.88,78.58,23,8,174
2013,4,6,17,51,57,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.32,79.99,24.57,76.22,25.88,78.58,23,8,174
2013,4,6,17,52,1,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.88,78.58,23,8,174
2013,4,6,17,52,5,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.88,78.58,23,8,174
2013,4,6,17,52,8,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.88,78.58,23,8,174
2013,4,6,17,52,12,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.88,78.58,23,8,174
2013,4,6,17,52,16,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.88,78.58,23,8,174
2013,4,6,17,52,20,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.88,78.58,24,8,191
2013,4,6,17,52,24,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.82,78.47,23,8,174
2013,4,6,17,52,28,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.82,78.47,24,8,191
2013,4,6,17,52,32,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.82,78.47,24,8,191
2013,4,6,17,52,35,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.82,78.47,24,8,191
2013,4,6,17,52,39,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.82,78.47,23,8,174
2013,4,6,17,52,43,0.52,0.51,0.0,0.0,0.52,0.37,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.82,78.47,24,8,191
2013,4,6,17,52,47,0.52,0.51,0.0,0.0,0.52,0.36,0.14,0.51,2.60,0.1,26.67,79.99,24.38,79.99,24.57,76.22,25.82,78.47,24,8,191

I guess I kind of simplified my descriptions a bit. The first 6 values are date and time, the next 8 are current, then voltage, power in kW, temp1_c, temp1_f, temp2_c, temp2_f, temp3_c, temp3_f, temp4_c, temp4_f, the last three represent UV, visible light and IR index. New data come in every 3-4 seconds and it will go on 24/7, and I want to keep the data for at least a year. I want to be able to select x and y axis as whatever variable I want, so I think the site needs to be able to go to the SD card, get its data from an ever growing data file, and plot the data. Can the Arduino with ethernet shield do that? Thanks!

I am also kind of confused about the ethernet shield. The site says that we can't use both the SD slot and w5100 at the same time. So I am not sure if it is capable of reading data from the SD card and present the data on the website at the same time...

4

Writing those data records to the sdcard every 3-4seconds - that is doable. You have to consider an sdcard has a write latency, 250ms worst case (so you have to wait 250ms until the sdcard is ready with its internal stuff). If you going to share SPI with the sdcard and W5100 then maybe a buffering is needed.. Anyhow, the amount of the data is not huge to be processed with single arduino, afaik.

5

Im trying to do a very similar project reading battery voltage. Would you be able to share any code or schematics? Thanks Ray