Hi! Long time lurker, first time post. I think I have this question in the right section here.
I have some HTML that uses a background image that the .css uses. When I load the sketch on my atMega2560 everything displays correctly including the background image, no problems. When I load the EXACT same code on my UNO (code size is under 32k), the page doesn't display at all. After commenting out a small piece of sketch at a time, I narrowed it down to the area of HTML and code that deals with displaying the background image.
I'm wondering if there is a difference in how to display HTML images between the two cards?? Sounds dumb I know. I don't know why it displays fine on the Mega, but not the UNO and hope someone can explain it.
I move the Ethernet Shield (with SD card) as a whole from the Mega to the UNO. I've verified the jpeg (17.1kb) is in the /images directory too. When I load the sketch on the UNO, it takes up 89% of program memory and 69% of dynamic. I first thought that was the issue, I included it here just in case.
The Arduino Uno and the Arduino Mega 2560 are very similar for software. They use the same source code of the same libraries. The Mega 2560 has extra timers, and libraries using timers can select unused timers more easily on the Mega 2560. The main difference is of course the size of the sram.
They do have a different SS pin for the SPI bus. You might have to set it as output before using the libraries. Perhaps you can also disable the SlaveSelect pins for the SD and the Ethernet before using the libraries. I suggest to check you code for that. I mean for the Mega 2560. You can forget about the Uno :o
The 69% sram usage is a lot. The Ethernet library and (and probably also the SD library) allocate buffers during runtime.
Did you know that using Ethernet and a SD card is never a good idea with a Arduino Uno ? Everyone that uses an Arduino Uno with Ethernet and a SD card has your problem. The first minimal sketch works, but after adding a few things, it fails.
Everyone includes me as well. I also started with a Ethernet shield on a Arduino Uno
Have you tried the simple step of putting some printLn statements into the file reading part of you code to see whether the file is successfully opened and how much is read? This would give you a major clue as to what might be happening.
You say tht 69% of dynamic memory is used, how can you tell - there are not calls to any of the functions that show memory usage in the code that you have posted. The SD file will allocate a 512byte buffer when opening a file, if the 69% of memory used does not include the buffers then the 512 bytes will add another 25% of memory usage immediately and the ethernet buffer is likely to add more as well. That means you will have at least 94% in use and probably trying to use over 100% when opening the file. The Mega has 8K rather than 2K of RAM and would explain why it works but the Uno fails.
Koepel:
The Arduino Uno and the Arduino Mega 2560 are very similar for software. They use the same source code of the same libraries. The Mega 2560 has extra timers, and libraries using timers can select unused timers more easily on the Mega 2560. The main difference is of course the size of the sram.
They do have a different SS pin for the SPI bus. You might have to set it as output before using the libraries. Perhaps you can also disable the SlaveSelect pins for the SD and the Ethernet before using the libraries. I suggest to check you code for that. I mean for the Mega 2560. You can forget about the Uno :o
The 69% sram usage is a lot. The Ethernet library and (and probably also the SD library) allocate buffers during runtime.
Did you know that using Ethernet and a SD card is never a good idea with a Arduino Uno ? Everyone that uses an Arduino Uno with Ethernet and a SD card has your problem. The first minimal sketch works, but after adding a few things, it fails.
Everyone includes me as well. I also started with a Ethernet shield on a Arduino Uno
I appreciate the feedback and ideas. I'll look at setting the SS pin definition before the libraries and see if that helps me. I honestly didn't even think about timers so I'll dig deeper and find out if there's a conflict or something.
I believe the chunk of memory usage (69%) is indeed by the libraries. I'll see if I can reprogram the same function(s) without the use of a library (and not using the library as an easy crutch).
countrypaul:
Have you tried the simple step of putting some printLn statements into the file reading part of you code to see whether the file is successfully opened and how much is read? This would give you a major clue as to what might be happening.
You say tht 69% of dynamic memory is used, how can you tell - there are not calls to any of the functions that show memory usage in the code that you have posted. The SD file will allocate a 512byte buffer when opening a file, if the 69% of memory used does not include the buffers then the 512 bytes will add another 25% of memory usage immediately and the ethernet buffer is likely to add more as well. That means you will have at least 94% in use and probably trying to use over 100% when opening the file. The Mega has 8K rather than 2K of RAM and would explain why it works but the Uno fails.
Indeed I did use Serial.println's through the steps to debug it and how I discovered what point it fails. At one point I commented out the 'favicon.ico' section to see if it was possibly an issue with images in general, but that wasn't the issue either.
The memory percentages were taken from the console after compiling/verifying, not during actual runtime.
Let me take a look and see if I'm off on the memory allocation and that being the issue. Thanks for pointing that out.
When there is not enough sram, then debugging has little use. The smallest change in the source code can make the compiler decide to make other choices and create a different binairy result and thus it will fail in an other way. If you add, for example, a single Serial.println(), then it might suddenly appear to work. Such problems are typical for overwriting sram.
When using a Arduino Uno with the Ethernet shield will work with a minimal sketch. As soon as a sensor is added and data is stored on the SD card, then there is not enough memory. This happens to everyone. There is no bug or poblem to solve, it is not possible.
Well, if you are not able to use or buy an other Arduino board, then you might be able to make it work with a very small sketch. But after some time you want to add something anyway
Do you want to spend enormous amount of time and energy to make it work on a Arduino Uno ? Even when there is 95% chance that it will fail anyway ?
Then you could start with a freeMemory() function that show runtime the used memory.
I have written a high water mark: HighWaterMark.ino.
But seriously, this all is useless. Everyone starts with a Uno and then moves on to a Arduino Mega 2560 with a Ethernet Shield.
well, you can host several pages, CSS, Javascript,.. in the program memory of the UNO and read several sensors. Strip down your jpegs, consider to use SVG's if you really think it is necessary. Just get rid of the SD card and put everything in the flash of the UNO.
Koepel:
When there is not enough sram, then debugging has little use. The smallest change in the source code can make the compiler decide to make other choices and create a different binairy result and thus it will fail in an other way. If you add, for example, a single Serial.println(), then it might suddenly appear to work. Such problems are typical for overwriting sram.
When using a Arduino Uno with the Ethernet shield will work with a minimal sketch. As soon as a sensor is added and data is stored on the SD card, then there is not enough memory. This happens to everyone. There is no bug or poblem to solve, it is not possible.
Well, if you are not able to use or buy an other Arduino board, then you might be able to make it work with a very small sketch. But after some time you want to add something anyway
Do you want to spend enormous amount of time and energy to make it work on a Arduino Uno ? Even when there is 95% chance that it will fail anyway ?
Then you could start with a freeMemory() function that show runtime the used memory.
I have written a high water mark: HighWaterMark.ino.
But seriously, this all is useless. Everyone starts with a Uno and then moves on to a Arduino Mega 2560 with a Ethernet Shield.
Do you know ESP32 ?
I'll try getting freeMemory() in there somewhere. It has me perplexed, that's for sure. I had taken just a quick look at your HighWaterMark here at work. Definitely going to run that one tonight to see how it works. Thanks for sharing!
By the way, no I do not know ESP32. I recently purchased a Mega with the ESP8266 Wi-Fi incorporated I'm waiting to arrive. Hoping it arrives around Christmas while I'm off work so I can play with that.
noiasca:
well, you can host several pages, CSS, Javascript,.. in the program memory of the UNO and read several sensors. Strip down your jpegs, consider to use SVG's if you really think it is necessary. Just get rid of the SD card and put everything in the flash of the UNO.
I'll have to try that next. I had seen your post about webservers and getting the favicon.ico written into code. Very informative, thank you for that.
The Mega + ESP8266 need to communicate with each other. That combination is double the trouble and half the fun.
The ESP8266 on its own has more memory and is faster than the Mega. The ESP8266 can even set a part of its Flash as storage to store files, for example the files of your website.
The ESP32 is the newer version of the ESP8266. So if you go straight for the ESP32, then you avoid a few quirky things of the ESP8266.
I used this to learn how to use the ESP32: 160+ ESP32 Projects, Tutorials and Guides with Arduino IDE | Random Nerd Tutorials.
How about the Uno ? Are you ready to give up on that ? or shall I try harder to convince you that it is not suited for Ethernet + SD
I'm assuming the board mods in that post need to be done for a Wiznet 5100/SD Card and Uno combination? Am I understanding that correctly? If so, that's a major design flaw that should never have passed a quality check. I noticed the schematics are for the 5500 chip, this applies to both the 5100 and 5500?
Koepel:
The Mega + ESP8266 need to communicate with each other. That combination is double the trouble and half the fun.
The ESP8266 on its own has more memory and is faster than the Mega. The ESP8266 can even set a part of its Flash as storage to store files, for example the files of your website.
The ESP32 is the newer version of the ESP8266. So if you go straight for the ESP32, then you avoid a few quirky things of the ESP8266.
I used this to learn how to use the ESP32: 160+ ESP32 Projects, Tutorials and Guides with Arduino IDE | Random Nerd Tutorials.
How about the Uno ? Are you ready to give up on that ? or shall I try harder to convince you that it is not suited for Ethernet + SD
Well, heck. The Mega/8266 is already shipped, so I guess I'm committed on at least trying to make that work for a bit. I'll have to start looking into the ESP32 as well, or sell the incoming card as soon as it arrives!
I didn't play with the card last night (got side tracked taking pictures of Jupiter/Saturn). Maybe I'll take a swing at it tonight before I give up all hope on the Uno.
As a side bar question, is there a limit to how much HTML code (page size) I can throw at the Uno/Mega before it regurgitates bits and tells me 'no'? HAHA I want to make sure I rule out any other possibilities that might pose a problem before I totally scrap the Uno.
of course there are limits. You only have a total of 32KB.
Just as an example:
Der Sketch verwendet 31918 Bytes (98%) des Programmspeicherplatzes. Das Maximum sind 32256 Bytes.
Globale Variablen verwenden 1072 Bytes (52%) des dynamischen Speichers, 976 Bytes für lokale Variablen verbleiben. Das Maximum sind 2048 Bytes.
that's a Webserver with a DHT and a BMP160 Sensor, some relays to be switched via HTML, the CSS, a JavaScript FetchAPI to refresh values, the JSON resource for the update, a webclient to send values to a server, two MAX7219 displays to display data locally, a LDR to dim the MAX7219.
When I need more than the remaining 300 Bytes I would need to delete some Serial Debug prints and cleanup the code (or get rid of the I2C Sensor and replace it with something with SPI...)
@noiasca, is the webpage in PROGMEM (Flash) or on a SD memory card ? Did you run the freeMemory() function to know the ram usage during runtime ?
@mtdewjunkie, the size of files or how many files does not really matter. If the same buffer is used to pass data on to the Ethernet shield, then the sram memory usage will be the same.
There is another big advantage for the ESP boards. The Arduino Ethernet shield might have problems with sockets that are not closed due to a bad connection. If all sockets are occupied, then it will not longer work. The ESP boards have that at a higher level.
Koepel: @mtdewjunkie, the size of files or how many files does not really matter. If the same buffer is used to pass data on to the Ethernet shield, then the sram memory usage will be the same.
There is another big advantage for the ESP boards. The Arduino Ethernet shield might have problems with sockets that are not closed due to a bad connection. If all sockets are occupied, then it will not longer work. The ESP boards have that at a higher level.
Ooooo... that's good to know. It didn't even cross my mind about the possibility of a socket problem/conflicting on the Ethernet Shield. If I determine that might be the issue, I'll find out what to do and report back here.
