I use several Unos, Nanos, and bare 328Ps in a system for my bicycle. They are connected either through USB or Bluetooth to a host, which is currently a minilaptop with windows Xp and my Python programs there. The host displays data during riding and may also control e.g the electric motor using Arduinos (pedelec)
Python is an interpreted language and therefore assumably slow. The laptop processor utilization jumps easily to 100 % even when only one of the Arduinos is sending at speed 115200 bps and my Python program simply receives from the serial port and stores the received data to its memory areas.
If a noninterpreted language is used instead of Python, I suppose the program execution speed (mainly in receiving data ) could be at least 10 x. Is that correct?
I have a free version of Microsoft Visual Studio 2010 with wide and confusing set of libraries for C++ and alternatives to program (console applications, windows, win32, .net , windows forms etc). Which alternative would you recommend to try to have efficient code? .
(there is currently no need to process massive amounts of data, but it is nice to have that possibility too)
If a noninterpreted language is used instead of Python, I suppose the program execution speed (mainly in receiving data ) could be at least 10 x. Is that correct?
Not necessarily. It depends on where the bottleneck is.
Which alternative would you recommend to try to have efficient code? .
The choices you list define the type of application, not the programming language. Do you need a console application (no user interface) or an application with a user interface?
(there is currently no need to process massive amounts of data
PaulS:
[
The choices you list define the type of application, not the programming language. Do you need a console application (no user interface) or an application with a user interface?
The preferred programming language is C++ as in Arduino, but almost any language is considered.
Console application is an alternative, anyhow, if the code for it is considerably more efficient than programs with fancy interfaces. It could gather data and other programs in memory could take care of displaying data with lower priority.
I was wondering why the 2000 megahertz CPU of the laptop is 100 % occupied, when a 16 megaherz Arduino is sending serial data to it at 115200 baoud speed, only. Very strange.
Perhaps I have to make small testprograms for several alternatives and find out where the bottlenecks really are. Arduino monitor could receive data about 1 megabit per second, albait with too many errors (as far as I recall right my test about 1 year ago).
Then, what does Python do?
Python code currently gathers small amounts of data, or as much as it has time for :), and it is for testing ideas with few lines of code. But if there is cpu-time available, the laptop could e.g. calculate motion data 100 times for each wheel rotation instead of one, getting more accurate results.
Your CPU is maybe used at 100% because you may have an infinite loop in your program that uses all the processor's power. If you use the same program with another non interpreted language it will do the same. But it's only in the case of an infinite loop. can you post your code so we could have an idea of what your problem is?
I have a free version of Microsoft Visual Studio 2010 with wide and confusing set of libraries for C++ and alternatives to program (console applications, windows, win32, .net , windows forms etc). Which alternative would you recommend to try to have efficient code? .
I don't know about efficiency when trying to run under Windoze OS, but Visual Studio also has free Express
compilers for Visual C# and also Visual Basic. You can choose the one you care for the most. From what I
can tell, they all use the same underlying core routines and libraries, and only the higher level syntax is
different.
You do have to run the Serial comms tasks as background tasks, using Delegates, with the Windows Forms
displays as foreground tasks, to get proper operation. Non-trivial mess. This way the comms tasks allow
the foreground tasks, as well as the entire rest of the OS, to update correctly.
The download file on this page shows how it's done for both VC# and VB,
Patouf:
Your CPU is maybe used at 100% because you may have an infinite loop in your program that uses all the processor's power. If you use the same program with another non interpreted language it will do the same. But it's only in the case of an infinite loop. can you post your code so we could have an idea of what your problem is?
The example above prints the number of lines sent from Arduino. When Arduino has stopped sending (e.g. in 30 seconds) , the program waits still 10 seconds and then prints the number of lines. (later, after the snippet above is the code to do more with the received lines. they are correct)
I see in the windows XP task manager/ cpu performance/ going up to 100 % and coming down.
There is firewall Comodo and virus scan avg active in the computer all the time. If they examine every received byte thoroughly... I have to try disabling them now.
Edit:
When disabling net connection, firewall and avg-virusprotection, the cpu activity was about 80 % on an average with the above loop, varying from 70% to 100%
Edit2:
When receiving with Arduino 1.0.1 monitor, the cpu load was 100 % all the time. But the monitor did more work, it displayed the received data during the transfer, of course. The volume of data in 30 seconds was about 60 kilobytes. (= 2 kilobytes / s, only!). Strange?
Edit3:
Avr studio 5 has a terminal window like Arduino monitor. It received the same data from Arduino during 30 seconds. All the time the cpu load was 100 %.
I cannot believe that a 2000 Megaherz computer cannot receive serially much much more than this 2 kilobytes / second.
Edit4:
Hyperterminal of Windows XP received and displayed the lines sent by Arduino. Average cpu-load was about 80 %.
Can we now draw a conclusion, that the bottleneck in receiving data cannot be in the user program/programming language/libraries but it is in the serial communication (drivers etc) or the cpu performance measuring in the task manager is wrong?
optimistx:
stores the received data to its memory areas.
What does this mean, exactly? Depending on the storage mechanisms being used, this could be a very resource-intensive activity.
It might be so, thanks for the idea.
It is the lines -variable above. In python it is a list of strings.
Howver, the tests above with AVR Studio 5 and Arduino IDE monitor and Windows XP hyperterminal reveal that the cpu load goes up to nearly 100 % in all of them, when receiving and displaying the received data. To display in the window might be as resource-intensive as appending strings to lists in python.
But this 2 kb/second transfer rate in practice is miserable. Might be that something is wrong with the whole computer (although it is not obvious, there are no other symptoms). I'll try with another computer.
Edit.
The other computer has also Windows XP. Arduino monitor used about 80-100 % of the cpu during transfer. It was about the same when scrolling active or not active.
But hyperterminal of Windows XP could receive and display all the lines without cpu-load increasing visibly; perhaps at most 10-20 %.
So there must be a way for Windows XP to receive faster than 2 kilobytes/sec. There is something wrong in my first computer obviously.
I have not Python installed in this latter computer yet, and not AVR Studio 5. I'll try later.
You do have to run the Serial comms tasks as background tasks, using Delegates, with the Windows Forms
displays as foreground tasks, to get proper operation. Non-trivial mess. This way the comms tasks allow
the foreground tasks, as well as the entire rest of the OS, to update correctly.
...
Yes, non-trivial mess ;). Look at the python code above, few lines of code, no mess. I looked at the examples in the net with dcb's and pages and pages full of code and not even sure, if those examples would work.
Ok, to be fair, it need not be 5 lines of code, but I could want write 2-3 pages to do the serial transfer fast.
I never found what caused it, but my old XP box used to thrash the CPU doing things that I didn't expect it to find hard such as displaying low resolution Youtube videos. It didn't have that problem back when it was new, and I guessed that either XP itself or one of the various security products I had installed were not in a healthy state - but apart from the thrashing, there were no obvious signs of problems. When it got too bad I would reboot it, and it seemed to improve things slightly - although that may have been wishful thinking.
You could try dropping the serial port down to lower speeds and see if you can get a feel for how fast you can run the link before the PC is overloaded.
Given that you got decent performance with hyperterminal, there is some chance you can write / find an application which works similarly fast. A 'C' or C++ program to read from a serial port and dump to a file would only be a few lines of code and would be easy to build using any version of Visual Studio. There are umpteen types of application you could use and none of them are right or wrong, but for a minimal application I'd start with a C++ console application.
I just tried running your Python code on Linux, with the Arduino sending about 2k/sec (modified the ASCIITable example). CPU usage was never more than 8% of one core. With the Arduino sending data constantly at 115200 it never used more than 25% of one core.
Well you already suspect your bicycle PC for doing something else wrong, but I just wanted to point out that python compiles the code before running it (albeit bytecode, not machine code).
Maybe you can update the motherboard drivers (chipset) and see if that resolves the problem?
Because the serial port is built-in, right? No add-on card or USB conversion dongle?
This is a long-shot but in some earlier days when hard drive access caused a windows computer to really slow down one checked if hard drive where run in DMA mode or PIO. Where the former "skips" the processor in some steps and the latter forces the CPU to do the dirty work itself. Can there be something similar going on with serial port? So it for some reason must spend all its time listening to the COM-port to not miss anything.
Yes, non-trivial mess . Look at the python code above, few lines of code, no mess.
Maybe that's the problem, it doesn't deal with background tasks correctly, so as to let the foreground
tasks update properly - as I mentioned before. IE, too simple. ????
Under Windoze, all those screen and mouse updates, including the ones on the Desktop outside of your
program are dealt with on a round-robin basis. Every little window does 'not' have its own pre-empter.
As mentioned, VC# & VB use background/foreground tasks and Delegates. I use a modified version of
Smiley's Simple Terminal, and run it all the time at 115.2 without problem on my XP machines.
I installed Python 2.7.3 to another computer with Windows XP service pack3, from python.org. Also installed pyserial module from the same pages.
Everything went smoothly for a beginner like me.
I typed to python exactly the same lines as earlier in this thread, except the port name is 'COM11' (looked at the devices in "my computer').
Arduino sent 60 kilobytes of data in 30 seconds as earlier.
Windows task manager showed cpu load about 60 %.
Not good at all!
When, anyhow, the hyperterminal can receive the same Arduino data with neglible cpu-usage, there is hope to find a way for more efficient transfer somehow.
I am grateful for alla ideas and viewpoints in this thread. The linux test with python is also encouraging: I might use linux instead of windows xp, if this problem becomes too complicated for me.
Why I am considering to use laptop instead of combining arduinos together with a small display?
I like to have big texts and images to look at. (my sight is not so good anymore)
There are occasions when computing power is useful to have.
It is pleasure to be able to write lines of code to arduino so that every microsecond is in one's control. Doing 16 million istructions a second is fantastic opportunity to achieve almost anything ...
But at the same time, a computer with winXP and 2000 million instructions per a second is very difficult for me to process a tiny number of bytes without gathering gigabytes of programs together and spending days and weeks to write the program lines, and asking experts, consuming their valuable time .
Cython gives you the combined power of Python and C to let you - write Python code that calls back and forth from and to C or C++ code natively at any point. - easily tune readable Python code into plain C performance by adding static type declarations. - use combined source code level debugging to find bugs in your Python, Cython and C code. - interact efficiently with large data sets, e.g. using multi-dimensional NumPy arrays. - quickly build your applications within the large, mature and widely used CPython ecosystem. - integrate natively with existing code and data from legacy, low-level or high-performance libraries and applications.
robtillaart:
have you considered using cython?
from - www.cython.org
Thank you, I'll definitely try it.
In the meantime I modified a Microsoft example about serialports used for chatting, C++/CLR code (Console application) :
It transfered from Arduino Uno to WinXPsp3 with the nominal speed 115200 bd setting, and displayed the data on the console window. 321 kilobytes in 30 seconds without errors (10.7 kilobytes/s).As fast as one could hope. The cpu load due to transfer and display was about 60 %, not bad that either.
I modified the Arduino program to have
Serial.begin(230400);
although Arduino monitor maximun setting is 115200. Arduino monitor hung and had to be killed.
But the C++/CLR code could receive everything that Arduino sent, without errors(I believe so, the output looked regular and nice), 17 kilobytes / second. CPU load was about 70 % due to this. Not bad. I might later try higher nominal speeds.
Conclusion: It is probably possible to transfer at least 17 kilobytes /second from Arduino Uno to winXp host with the suitable receiving program in the host. (this might be error prone, however).
Very interesting link. My project resembles that. I wonder how Tom has succeeded.
500 000 bd rate was probably successful today; I must add some checksums to be sure. Temperature drifts of clocks and circuits may give problems.
Would we need 2 easy-to-use programs to test the connections, one for the host and one for the Arduino. Very simple test, which calculates error rates. It starts from low speeds and goes up until too many errors occur. It could send e.g. an ascii table many times during 5 seconds. Perhaps there exists such already here in Arduino forum?