Please whitelist a "test/" directory in the Arduino library specification

I've developed a system called arduino_ci that enables you to run automated tests on Arduino libraries, both locally and remotely (as part of a service like Travis CI). Among other things, this includes abilities such as

• Fully simulating the physical world (e.g. clock, pin inputs, serial inputs, etc)
• Testing against multiple boards with no hardware present
• TAP-formatted test output

(More discussion in this thread.)

The trick is where to put the files that contain the tests. I would prefer not to put it in "extras/", both for closer proximity to the code begin tested and because (in my opinion) the ability to run tests is an important and necessary part of any software project. What chance is there of amending the library specification to whitelist a directory called "test/" where all the automated tests would live?

Sounds more like a github request than a forum request.

ballscrewbob:
Sounds more like a github request than a forum request.

Sounds good. Which of the 110 repositories do you think is most relevant to this request (and ensuing discussion)?

That would be up to you to work out based on what you want to do exactly.

The devs get to github more than they get in here thats all.

It's rare for the Arduino developers to read the forum, even though you'd think someone would monitor this section. However, this is a good place to get some community input on a proposal. Sometimes we can resolve things here on the forum (such as when there is a solution already in place or one in the works). Other times the discussion here can refine the proposal before it is finally submitted to the developers. So the system is not completely broken.

The tool used for handling libraries is arduino-builder:

However, if this is simply a matter of changing the specification, rather than actually changing code, then that would not be the best place to submit the issue report. Instead I would recommend the arduino/Arduino repository:

In addition to containing the code for the Arduino IDE, that repository acts as the general catch-all for issue reports that aren't specific to any of the other repositories.

I recommend including the proposed text you would like to have added/removed from the specification. The specification is actually a publicly editable wiki but I don't think it would be appropriate to simply make the change on your own initiative. However, if you can get approval from a member of the Arduino team you can then go and make the edit yourself so there will be no need to wait for them to do it. I would actually state that you're willing to do this as it may make the developers more receptive to just saying "sounds good, go ahead" instead of waiting until they actually have the time.

I should clarify what I meant by "white list". The relevant part of the specification is here:

The source code found in src folder and all its subfolders is compiled and linked in the user’s sketch. Only the src folder is added to the include search path (both when compiling the sketch and the library).

...

For backward compatibility with Arduino 1.0.x, the library author may opt to place source code into the root folder, instead of the folder called src. In this case the 1.0 library format is applied and the source code is searched from the library root folder and the utility folder

So to me this is a "white list" of the library root folder, and the utility folder or the src folder for library source files. In addition there is a "black list" of sorts:

The content of the extras folder is totally ignored by the IDE

So you could maybe consider any other folders to be in a gray area. Unfortunately the Arduino IDE does not actually follow the specification regarding the extras folder since sketches in that location are still shown in the File > Examples menu. However, at least with the specification in place we can definitively consider that to be a bug and hope for it to be fixed eventually. I don't think that issue is relevant to arduino_ci since you use .cpp files rather than .ino.

I submitted a new topic on the google group but it hasn't appeared yet (I assume there's some moderation step that happens for first posts, otherwise I've made some big blunder in the UI and never sent my original message).

I'll check out those other repositories in a bit.

The GITHUB links Pert gave you are the best...Google is not that reliable for the Arduino aspect.

The developers mailing list is another good place to get input from the Arduino community. It tends to be the more advanced developers on there. Some members of the Arduino team do read that mailing list, though they don't always participate in the discussions. I think they manually approve every message that goes on there and sometimes it does take quite a while for the posts to be approved. I think it's a reasonable place to post the proposal. If you get approval from a member of the Arduino team there then you can proceed directly to editing the wiki. If some days pass after your post with only replies from community members or none at all then I would go ahead with a GitHub issue report.