Embedded Tool Kit

Hi all,

I'd like to show off a project of mine, which I hope you will find useful.

It's a collection of lightweight and useful classes. It's got classes for string manipulation with atoi and atof implementations, an array wrapper that has a similar look and feel to std::vector, memory pools, basic math and a few algorithms such as swap and bubble_sort.

It also provides classes to handle vectors, matrices and quaternions. Great for orientation sensors, drones and other sensor fusion applications.

There are a bunch of navigation classes and heaps of other stuff too . .

ETK contains no calls to malloc or new, it does not use exceptions and it uses no external libraries except for stdint.h and type_traits (for static assertions). There are unit tests, but they aren't 100% complete and there are a few known bugs. However, the intention is that ETK is more of less in accordance with high integrity coding standards such as MISRA.

Please let me know what you think.

Where is the user manual?

IMHO that is almost more important than the code - especially if there is more than one function.

...R

The user manual is very much a work in progress. I agree very much that it's super important and I'll be adding doxygen comments over the next few weeks.

In the mean time, there are a few blobs of documentation on my blog.

http://www.camelsoftware.com/blog/2015/12/11/ditching-c-libraries-concatenation-of-c-strings-without-stdio/

float Coordinate::cross_track_distance(Coordinate from, Coordinate to)
{
    float d13, brng13, brng12;
    d13 = from.distance_to(*this);
    brng13 = DEG_TO_RAD*(from.bearing_to(*this));
    brng12 = DEG_TO_RAD*(from.bearing_to(to));
    int R = 6371000.0f; //radius of the world in meters
    return asin(sin(d13/R)*sin(brng13-brng12)) * R;
}

should *this not be from?

Thanks for sharing BTW

Camel:
The user manual is very much a work in progress. I agree very much that it's super important and I'll be adding doxygen comments over the next few weeks.

In the mean time, there are a few blobs of documentation on my blog.

http://www.camelsoftware.com/blog/2015/12/11/ditching-c-libraries-concatenation-of-c-strings-without-stdio/

better keep the doc in the GHrepo ?

robtillaart:

float Coordinate::cross_track_distance(Coordinate from, Coordinate to)

{
   float d13, brng13, brng12;
   d13 = from.distance_to(this);
   brng13 = DEG_TO_RAD
(from.bearing_to(this));
   brng12 = DEG_TO_RAD
(from.bearing_to(to));
   int R = 6371000.0f; //radius of the world in meters
   return asin(sin(d13/R)*sin(brng13-brng12)) * R;
}



should *this not be from?

Hmmm, I'm pretty sure that code is ok. The unit test for that function is actually commented out so it could well be broken. Thanks for the heads up.

A the picture is again worth a 1000 words ..

Woot! I've been adding doxygen comments all day. I've also worked out how to use gh-pages so the documentation is now live on github.io

http://supercamel.github.io/EmbeddedToolKit/docs/html/index.html

Thanks, that documentation is a big help.

I will bookmark this Thread for study when I get a bit of time.

Am I correct to assume your StaticStrings are just a version of standard C strings and do not suffer from the memory corruption problems that the String class can?

...R

Robin2:
Thanks, that documentation is a big help.

I will bookmark this Thread for study when I get a bit of time.

Am I correct to assume your StaticStrings are just a version of standard C strings and do not suffer from the memory corruption problems that the String class can?

...R

When you say standard C strings, I think of character arrays and functions such as strcpy and sprintf. On the inside StaticStrings use an etk::List which is essentially a char array, but they don't depend on any C libraries. They're so encapsulated that I don't think you could call StaticStrings a version of standard C strings. It's probably a bit closer to std::string, except StaticStrings obviously can run out of space whereas std::string continues to allocate memory as required. Normal use of StaticStrings won't corrupt memory like the standard C string functions could as buffer overruns are protected against.

Camel:
When you say standard C strings, I think of character arrays and functions such as strcpy and sprintf.

Yes, that is what I meant.

It's probably a bit closer to std::string, except StaticStrings obviously can run out of space whereas std::string continues to allocate memory as required.

But this confuses me. A char array does not "continue to allocate memory" AFAIK whereas the String class does. And the String class causes problems in the limited amount of memory on an Arduino.

Does your string or rope class grab extra SRAM that the user would not be aware of? If it does it seems to me it could easily consume all the available memory and cause a crash.

...R

Robin2:
Yes, that is what I meant.
But this confuses me. A char array does not "continue to allocate memory" AFAIK whereas the String class does. And the String class causes problems in the limited amount of memory on an Arduino.

Does your string or rope class grab extra SRAM that the user would not be aware of? If it does it seems to me it could easily consume all the available memory and cause a crash.

...R

Ah, ok. No, StaticString and Rope do not grab extra memory. They both use fixed sized arrays. StaticString contains it's own memory. Rope requires a pointer to an external buffer. The maximum length of a StaticString is specified by the template parameter. StaticString<5> ss; for example will creating a string with a maximum length of 5 character. If you write ss = "Hello world"; ss will contain "Hello". Attempting to write past the end of the string like this ss[8] = 'c' will also silently fail.

Thanks, that is very clear and sounds like a good concept. There are quite a few problems among the Forum questions in which the cause is writing beyond the bounds of an array.

...R

I've added some more interesting stuff, notably the EvoPID class.
https://github.com/supercamel/EmbeddedToolKit/blob/master/inc/etk/evopid.h

EvoPID is a self-tuning PID controller that uses an EA (evolutionary algorithm) to search for optimal PID gains. I've used the EvoPID concept (not this exact code, but similar) in a UAV flight control system before, and it could finely auto-tune a drone within 10mins of operation.

EAs are great because they can solve complex problems fairly quickly. They do this by having a 'population of genomes'. This is EA lingo for 'a bunch of random PID settings'. Each genome / PID setting is given a go at controlling the system and is given a fitness rating. Once all 8 PID gains have been rated, survival of the fittest takes place. The best ranked PID gain is randomly mixed with one of the next best PID gains to generate another whole new population. Sometimes there is a genetic mutation (ie. a PID gain is varied randomly). This is necessary to keep the search from settling on local optimas. The cycle repeats . . . favouring the best PID gains each time. Eventually the entire population of PID gains will be pretty close to the optimal settings.

The most critical part of EAs is how they are rated. If the rating algorithm is giving good scores to bad genomes, then the EA will not work very well. This is especially possible when a system with bad gains has reached a stable point and is not being perturbed. I've provided a basic PID rater with ETK which should be suitable for most control systems, but it's relatively straightforward to implement your own PID rater with ETK.

Due to the random nature of genetic mutations in EAs, it's probable that it will at some point select PID gains that are really bad. This could shake a system to pieces and be disastrous. The basic PID rater can detect and abort fatal mutations. This means that bad performers will be stopped quickly and removed from the gene pool, thereby preventing a potential accident.

All of this bla bla means great control systems without wasting time fiddling with settings.

Example code and documentation coming soon . . .

Camel:
EAs are great because they can solve complex problems fairly quickly. They do this by having a 'population of genomes'.

Interesting idea.

I wonder if the likelihood of of your code being used would be increased by putting like-minded code into separate Toolkits rather than having a large mixed bag.

...R

Robin2:
Interesting idea.

I wonder if the likelihood of of your code being used would be increased by putting like-minded code into separate Toolkits rather than having a large mixed bag.

...R

I've been wondering that myself, actually. The thing is, there's a fair amount of interdependence through ETK. Breaking it into separate toolkits would introduce dependency headaches. Having it all bundled together makes that a non-issue. Perhaps putting more work into documentation would be the way to go. There'd be sections for container objects, string manipulation, mathematics, control theory, signal processing and so on. What do you think about that?

Added an Arduino compatible port of StaticString to the repo.

https://github.com/supercamel/EmbeddedToolKit/raw/master/arduino_libs/StaticString.zip

Camel:
Perhaps putting more work into documentation would be the way to go. There'd be sections for container objects, string manipulation, mathematics, control theory, signal processing and so on. What do you think about that?

I think I lost a reply I had drafted last Friday and, between one thing and another I had forgotten.

Compartmentalising the documentation could go a long way to making it more accessible to less-experienced users. For example a beginner might find the Bits class useful but scalarLinearKalman scares me :slight_smile:

...R

EvoPid auto-tuning PID controller explanation and example code as promised.

LINK

ETK actually won't compile as-is for AVR chips. It uses C++14 language features and I don't think the AVR compiler supports that yet. The latest g++-4.9 for ARM does so in theory it could compile for the Arduino Due. The Arduino IDE only supports C++11 for the Due so far, though.

It'd be a lot easier to port to C++11 than C++98, so the Due is viable but I'm not going to do that. Arduino should upgrade compilers to and pass -std=c++14 :stuck_out_tongue:

So I've been fiddling with path finding algorithms for a robotics project lately and it seems the vast majority of these algorithms require tree-like data structures. You can't easily do that without using dynamic memory and pointers and whatnot. Admittedly, tree structures generally use nodes that are all the same size, so they're not a huge heap fragmentation risk. But for the sake of keeping ETK on the straight and narrow I decided to see how trees could be implemented WITHOUT new/delete.

What I came up with is etk::ObjPool. ObjPool is a memory pool class that can only allocate one type of object. The allocation function, ObjPool::alloc(), calls the constructor for the object and returns a 'pool pointer' which is a reference counting smart pointer. When all pool pointers to an allocated object are destroyed, the objects destructor is called and it's memory is returned to the pool.

It's impossible for an ObjPool to cause memory fragmentation, and it's impossible (at least, very difficult) for an ObjPool to leak memory. So it's a pretty good option if you need to implement trees or linked lists on a micro-controller or embedded device.

The downside of course is that memory pools nearly always use more memory than they need, and if they run out of memory they cannot just grab some more. Choosing the size of the pool could be tricky and any coding using it should be able to recover from allocation failures.

In some circumstances smart pointers might a overkill for a memory pool, especially if it's declared on the stack since the entire pool would get cleaned up anyway. If memory management isn't required and you wish to avoid the overhead of 'pool pointers', ObjPool has a raw_alloc function that allocates and returns a regular C pointer. If you need to, you can free the pointer using ObjPool::free. Note that ObjPool::raw_alloc and ObjPool::free do NOT call constructors or destructors and you would need to use placement new/delete instead.

This is all new code, so if anyone could run their eyes over it I'd be very grateful.

objpool.h
linked list example