Go Down

Topic: Cosa: An Object-Oriented Platform for Arduino programming (Read 98 times) previous topic - next topic

kowalski

#420
Jul 24, 2014, 07:38 pm Last Edit: Jul 24, 2014, 10:26 pm by kowalski Reason: 1

I can see new shell function "execute_P", I believe it is taking the input from the prog mem instead of the buffer? I didn't find it in the online documentation.

@g_ad

That one is easy; my dropbox has not yet synced the latest update. But it should be ready now.


Could you please tell me one more small thing that I didn't find the answer for yet - how to understand the uart blocking and non blocking "uart.set_blocking(SLEEP_MODE_IDLE)" what does it actually do?

The issue is what should uart.gets() do when there is no more characters in the input buffer. The non-blocking behavior is to return directly with what was available. That would require building up the command line until the new-line character is seen.

The set_blocking() tells the device to wait for more characters and return when it receives the new-line. The function gets() will call yield() while waiting for more character. And the actual sleep mode is defined by the implementation of yield(). There are two major implementations of yield(). One that uses power down and the second that does context switches.

Links down the rabbit hole.
https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/IOStream.cpp#L355
https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/main.cpp#L118
https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/Nucleo/Thread.cpp#L31

Cheers!

g_ad

@kowalski

The issue is what should uart.gets() do when there is no more characters in the input buffer. The non-blocking behavior is to return directly with what was available. That would require building up the command line until the new-line character is seen.

The set_blocking() tells the device to wait for more characters and return when it receives the new-line. The function gets() will call yield() while waiting for more character. And the actual sleep mode is defined by the implementation of yield(). There are two major implementations of yield(). One that uses power down and the second that does context switches.

Links down the rabbit hole.
https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/IOStream.cpp#L355
https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/main.cpp#L118
https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/Nucleo/Thread.cpp#L31


Thank you, I was already exploring some  parts of the rabbit hole :) but there is plenty of interesting stuff for many different adventures ;).

Thanks to your answer I actually spotted something that I've ignored before - the shell handling actually blocks the main loop waiting for the command line and it might interfere if there is some sort of "multitasking" ...
First I'm going to get familiar with the library and next I will mess a bit to capture the string incoming from uart and pass it to the shell when ready so it will not block the main loop :).

Appreciate your help as always,
Kind regards,
Adam

g_ad

@kowalski

I think I've found some weakens in CosaShell - if you provide too long string of arguments it doesn't return some sort of error -illegal argument but processes part of the arguments and continues with the rest of the string after, please see :

entered from terminal:
Code: [Select]

args 1111111111 2222222222 3333333333 4444444444 5555555555 6666666666


system response:
Code: [Select]

arduino:$ args 1111111111 2222222222 3333333333 4444444444 5555555555 666
argument: 1111111111
argument: 2222222222
argument: 3333333333
argument: 4444444444
argument: 5555555555
argument: 666
arduino:$ 6666666
illegal command
arduino:$


Kind regards,
Adam

kowalski

@g_ad

That is actually exactly the behavior it "should" have right now when the line is too long. The input buffer is limited to 64 characters (in the current version). It is a configuration variable. The error handling could be improved. Please see the configuration constants: https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/Shell.hh#L117

I will see what I can do about that in Shell::run() as the too long command lines will result in multiple gets() calls. These will have to be ignored and reported as an error.

Latest update; The script handling will allow parameter binding (standard shell script syntax, $1 for parameter/option one, etc). Please see the updated CosaShell.ino/Commands.cpp. The script blink will require a delay time parameter (in ms).

Cheers!

g_ad

@kowalski

That is actually exactly the behavior it "should" have right now when the line is too long. The input buffer is limited to 64 characters (in the current version). It is a configuration variable. The error handling could be improved. Please see the configuration constants: https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/Shell.hh#L117

I will see what I can do about that in Shell::run() as the too long command lines will result in multiple gets() calls. These will have to be ignored and reported as an error.


Thank you Mikael, please treat my posts informational, by any mean not as a change request, I only shared it as result of the crash testing ;) and if it can help you with the development that's good, otherwise just ignore it :). I'm happy to do some beta/stress testing ;)

I think I'm going to capture the uart in the main loop and pass the string to shell when the command line is ready to be processed so I will not block the main loop while waiting for the command line, this way I can quite easily test the string before passing it to shell to avoid multiple gets() calls :).

Kind regards,
Adam

g_ad

@kowalski

After the latest update the CosaShell example doesn't compile:
Code: [Select]

In file included from /Commands.h:27,
                 from Commands.cpp:21:
... \arduino-1.0.5-r2\hardware\Cosa\cores\cosa/Cosa/Shell.hh:121: error: ISO C++ forbids initialization of member 'ARGV_MAX'
... \arduino-1.0.5-r2\hardware\Cosa\cores\cosa/Cosa/Shell.hh:121: error: making 'ARGV_MAX' static


Kind regards,
Adam

kowalski

#426
Jul 26, 2014, 01:07 am Last Edit: Jul 26, 2014, 01:11 am by kowalski Reason: 1
@g_ad

Update available that fixes that for Arduino 1.0.X.

BW: It is better to use the github issues list for this type of problems.

Cheers!

kowalski

#427
Jul 26, 2014, 01:32 pm Last Edit: Jul 26, 2014, 01:43 pm by kowalski Reason: 1
A new screenshot of the Cosa Shell example sketch. There are a number of new commands; dump memory, play tone, repeat command lines, etc.

The options parser makes it really easy to write action functions with multiple options and variable number of arguments. Below is a snippet from CosaShell Commands. It implemented the memory dump command which has radix options, memory address and block size:
Code: [Select]

static const char DUMP_NAME[] __PROGMEM =
 "dump";
static const char DUMP_HELP[] __PROGMEM =
 "[-b|-d] POS [SIZE] -- dump memory";
static int dump_action(int argc, char* argv[])
{
 IOStream::Base base = IOStream::hex;
 uint32_t addr = 0L;
 size_t size = 256;
 char* option;
 char* value;
 char* sp;
 int ix;
 while ((ix = shell.get(option, value)) == 0) {
   if (strcmp_P(option, PSTR("b")) == 0)
     base = IOStream::bin;
   else if (strcmp_P(option, PSTR("d")) == 0)
     base = IOStream::dec;
   else
     return (-1);
 }
 if (ix < argc) {
   addr = strtoul(argv[ix++], &sp, 16);
   if (*sp != 0) return (-1);
 }
 if (ix < argc) {
   size = strtoul(argv[ix++], &sp, 10);
   if (*sp != 0) return (-1);
 }
 if (ix != argc) return (-1);
 cout.print(addr, (void*) addr, size, base);
 return (0);
}

The first section uses Shell::get(option, value) to parse the given options. The next section handles possible parameter; memory address to dump from (default 0), and size of memory block (default 256). For more details please see https://github.com/mikaelpatel/Cosa/blob/master/examples/Sandbox/CosaShell/CosaShell.ino.

The Cosa Flash File System (CFFS) shell has also been updated with some new commands:

For more details please see https://github.com/mikaelpatel/Cosa/blob/master/examples/Sandbox/CosaCFFSshell/Commands.cpp

Cheers!

g_ad

@kowalski

Update available that fixes that for Arduino 1.0.X.

BW: It is better to use the github issues list for this type of problems.


Thank you very much and I'm sorry for not reporting it on githube, you've mentioned it before  :smiley-red:

Now I want to mention something about CosaShell functionality and please correct me if I should do this on github as well, this is not so much about error/bug-issue. Btw - what should be reported on github and what on forum?

I want to mention something about CosaShell functionality.
In my professional life I deal with a lot of different devices through the console-rs232 and I'm using Putty on my Windows7; because of that Putty is my preferred way for serial monitor on Arduino instead of the built-in one.
Yesterday I was going to test the new Shell library and to my surprise I got nothing on my Putty apart from the prompt, everything was working with the built-in serial monitor so the conclusion was very simple - the Putty doesn't use LF as line termination, it uses CR. I tried to simulate other terminals and all of them interpret 'enter' as CR.
The above means that using the library in real development might be tricky and require code changing for at least Windows+Putty users :/
I'm far from requesting any change but if you would like to do some correction then instead of checking for LF line termination it could be checked for LF or CR.
Another problem is with sending line termination from the board to external terminal - all the other devices I used to work with definitely use CR, maybe they send CR+LF but to proof that I would have to capture the whole stream, not just the visible characters, however I can see that when I run the examples from the library that is using LF, then that is exactly what I can see on the terminal - the new line doesn't start in column 0 but stays in the same column as the line above.

As usual, if the above doesn't make sense please ignore it :)
Kind regards,
Adam

kowalski

#429
Jul 27, 2014, 09:55 am Last Edit: Jul 27, 2014, 09:58 am by kowalski Reason: 1
@g_ad

Hi Adam. I use this topic of updates on the development of Cosa. There are questions from users and as other forums users mostly help each other. These questions also get mixed up in this topic.

Bugs, errors and feature requests should be posted on the Issues list on github. In general I do not add new boards, modules, devices, etc, that I cannot test or have my own project for.  Pull requests that cannot be tested are not merged (of the same reason).

Now the more interesting question; handling of NEWLINE. Linux LF, Mac CR and PC both CR+LF, and others. http://en.wikipedia.org/wiki/Newline

I did some tests with configuration of Putty and the Terminal window setting "Implicit CR in every LF" (and local echo auto) fixes one side of the issue. The other will require adding support for translation (as in many Linux tools). The simple fix was to translate CR to LF in gets().

The more elegant solution is to add a translation setting to IOStream::Device and default to a common standard. The NEWLINE handling needs to be abstracted to all type of devices (e.g. terminal, network, files, etc).

I do not use Putty and I work mostly on Linux/Ubuntu. To get the fastest possible development turn-around I have actually added a build script (modified Arduino-Makefile) and miniterm for serial monitor (also modified). This allows build of only what has changed, upload and starting the serial monitor as a single command. The Arduino IDE build is 3-5 times slower in build. A lot of unnecessary operations are performed as it does not use make.

Cheers!

g_ad

@kowalski

@g_ad

Hi Adam. I use this topic of updates on the development of Cosa. There are questions from users and as other forums users mostly help each other. These questions also get mixed up in this topic.

Bugs, errors and feature requests should be posted on the Issues list on github. In general I do not add new boards, modules, devices, etc, that I cannot test or have my own project for.  Pull requests that cannot be tested are not merged (of the same reason).

Ok, I think I got it, I'll learn how to use github.



Now the more interesting question; handling of NEWLINE. Linux LF, Mac CR and PC both CR+LF, and others. http://en.wikipedia.org/wiki/Newline

I did some tests with configuration of Putty and the Terminal window setting "Implicit CR in every LF" (and local echo auto) fixes one side of the issue. The other will require adding support for translation (as in many Linux tools). The simple fix was to translate CR to LF in gets().

The more elegant solution is to add a translation setting to IOStream::Device and default to a common standard. The NEWLINE handling needs to be abstracted to all type of devices (e.g. terminal, network, files, etc).


Yes, I know the "Implicit CR in every LF" and I have it enabled for the arduino serial monitor, the problem appeared only when I've tried to use the Shell library with Putty, so yes the real problem is one direction, however the "Implicit CR in every LF" would be probably not elegant solution for the final device. I can change the part of the code for myself I suppose but wanted to share with you the finding in case you would like to implement it in the library.

The more I think of what I want to use, the more convinced I am that I will do some capturing of the Uart input, maybe implementing backspace and echo + eventually the CR to LF conversion on the main loop and then passing the cmd string to the shell. At the moment I'm still testing and learning from your code.

Thank you,
Adam

kowalski

#431
Jul 29, 2014, 12:30 pm Last Edit: Jul 29, 2014, 12:37 pm by kowalski Reason: 1
The news on the latest updates in Cosa.

1. IOStream
The IOStream class has a new member function to allow simple command line reading; IOStream::readline(buf, size). It will read available characters and append to the given string and return the buffer pointer when the line has been completed. The function will handle backspace, carriage-return and line-feed.

2. Shell
The command line support is now completed. The Shell::run() member function will handle the read, parse and execution of command lines. The function is non-blocking and should be called in the loop(). Below is a screenshot of the example sketch CosaShell. This sketch shows how to setup the shell for serial input, run the top-loop, and how to capture idle time.

As the Shell class uses the new IOStream::readline() function all echo and backspace is handled by this function. Below is a screenshot of using PuTTY with the shell. This is the default PuTTY setting (on Ubuntu).


3. Refactoring
a. SLEEP
The macro SLEEP (Cosa/Types.h) has been removed and the function sleep() should be used instead.
b. IOBuffer::gets()
This member function has been removed and the default implementation (IOStream::Device) is used instead.
c. IOStream::print(const void *ptr, size_t size, Base base, uint8_t max)
This member function is used to dump memory block. An additional base variant is now available that will prefix each line with a source address (i.e. not the given buffers address).

4. Versioning
The Cosa project does not have explicit versioning as it is non-commercial and sharing but to make it simpler for users there are specific snapshots of source and documentation. For more details see the install description and the section "Download Versions". https://github.com/mikaelpatel/Cosa/blob/master/doc/02-install.md#download-versions. Please note that using the build script will require some additional configuration. Please see https://github.com/mikaelpatel/Cosa/blob/master/build/Cosa.mk and previous post on this issue.

Cheers!

g_ad

@kowalski

The news on the latest updates in Cosa.
...


This is brilliant :), thank you for all your hard work.

Kind regards,
Adam

kowalski


This is brilliant :), thank you for all your hard work.

@g_ad

You are welcome!

I think that the Shell support is now more in line with what you where expecting from the start :). The development and refactoring has a lot to do with your input and tests - THANKS. I wanted to start-off with a simple blocking version and move slowly towards non-blocking as in the "final" version. With your input the update and evolution was somewhat faster.

I specifically added the "idle" command to the example sketch (CosaShell) to demonstrate the non-blocking IO and power down sleep that is going on in the background. There are more to be done on scripts; use EEMEM for storage, allow definition as commands, etc, but this will do fine for the release.

Thanks again for collaborating!

g_ad

@kowalski

You are welcome!

I think that the Shell support is now more in line with what you where expecting from the start :). The development and refactoring has a lot to do with your input and tests - THANKS. I wanted to start-off with a simple blocking version and move slowly towards non-blocking as in the "final" version. With your input the update and evolution was somewhat faster.

I specifically added the "idle" command to the example sketch (CosaShell) to demonstrate the non-blocking IO and power down sleep that is going on in the background. There are more to be done on scripts; use EEMEM for storage, allow definition as commands, etc, but this will do fine for the release.

Thanks again for collaborating!


Thanks a lot, this is indeed something I was aiming for and I didn't expect anyone doing the hard work for me ;). I'm very happy that I was able to help somehow, hopefully more to come, maybe one day will be able to contribute some code more than ideas :).
I already looked at the code and figured out the "idle" :) but still need to understand all around the "power down sleep" - I didn't yet think about the power management.

You are a star !!! :)

Go Up