Code practice and testing

From https://forum.arduino.cc/index.php?topic=732159.msg4926493#msg4926493

@bperrybap

While I know it is very common practice these days, not just in Arduino code, but all over the place to write code that essentially does some sort of polling so it very much depends on a particular speed of some sort of looping code, IMO, it is very bad practice and it concerns me greatly.

My original reply to the above wandered off the original subject somewhat, as I thought it was worthy of debate I am starting a new topic, rather than hijacking the original one. My reply is below.

My original reply

Bill,
I have you down as someone experienced and knowledgeable so please can you say more about that? All my code polls everything as much as possible, I try to write it as tight as I know how with minimal delay and nothing happening unless it is needed. When it comes to displays I update them when there is something to update, the obvious example being a clock, which I would update once per second for the obvious reason that seconds change once per, er...., second(!).

I don't want code that waits for hardware, however fast I2C runs it still runs slow enough that there is time for the CPU to do something else useful while a byte is sent or received, wasting that time is plain wrong to my way of thinking.

The worst example I have come across was some TCP client code which waited for a response from the server before returning, taking 5 seconds or so if the server didn't respond. This is intolerable! I've not done anything useful with TCP as I have found UDP sufficient to my needs and much simpler to use. Losing the odd packet doesn't matter if you know you are going to lose them and take measures to mitigate that.

In case it helps I am hobby programmer with no training, I started with Z80 assembly language in the mid 1990s, then PIC assembly, then PIC C, now Arduino C/C++ (although I don't really use C++).

Bill then sent me a PM with his response. I asked him if this could be a public discussion and he agreed. Please wait for his reply to this before responding to either of us.

Thanks.

Sure I'm here to hopefully add something useful to the discussion.
--- bill

Someone seems to have complained that code is looping. In all my years of programming, since 1965, I have NEVER seen programs, including main-frame operating systems that did not LOOP.
Paul

Paul_KD7HB:
Someone seems to have complained that code is looping. In all my years of programming, since 1965, I have NEVER seen programs, including main-frame operating systems that did not LOOP.
Paul

My comment was not just about looping, it had more to do with constant polling vs using interrupt or event driven scheduling but there was even more to it than that.

It is possible to have a system that never uses a main continuous constantly running loop polling things to get everything done but instead processes what needs to be done, as it happens, and eventually does some kind of halt/sleep when there is nothing to be serviced.

But my comment was really driven by my concern that I'm starting to see lots of code and coding practices where there is not much system architecture planning up front and EVERYTHING constantly runs through a main loop, including in some cases where there are very real-time requirements vs using interrupts or using events in the case of using an OS or task scheduler.
A constant polling type of design also tends to fall apart when it comes to power management since, if you need to constantly poll, you can't put the processor to sleep to save power.

I get why it is done, it is often easier, sometimes it is because the code was not fully planned out but rather was done on the fly or slowly evolved to include more and more things in a main loop, until everything breaks because the overall system was never truly designed.

Don't get me wrong, rigid timing requirements for a system using a main loop that polls everything has it place and I've done things that way many times for certain situations but that kind of system can sometimes be difficult to extend and can have issues when trying to interface or use other s/w components that cannot guarantee latencies or that can have variable timing.

What concerns me most in the s/w industry at large is that I see it moving away from what I would call "Computer Science" and a traditional product life cycle, where up front time is allocated to plan a project and define its architecture to meet the project/product requirements to trying to emphasize moving as fast as possible to get to some kind of a deliverable.
Once the delivery schedule is the priority and is in control, important steps start to get short circuited or even skipped.
I think some of this has been driven by how quickly code can be compiled/built and how easy it is to push out updates for little to no cost.
The mindset seems to be shifting to that it is ok to push out something even it isn't quite ready or hasn't been tested very much since it can be updated in the field quickly and fairly easily at a low cost to fix issues.
i.e. is more important to show progress by pushing a release, than taking more time to push out something that is better designed and more stable.
I believe that this creates a reactionary development process rather than proactive development model.
and IMO, causes products in the field to have more issues and be of a lower overall quality.

With respect to Arduino, Arduino actually promotes some things that bother me.
The whole concept of "tinkering" and "trial and error" "programming" goes all the way back to the Arduino founders.
While I get it for certain situations and for less technical or even non technical people that have limited programming skills, that are trying to just get something/anything working; however, it does make me quiver to watch it and see it actually promoted as a development style.

I am just not a "trial and error" person.
For my projects I go by a more traditional process:
define requirements, plan, design, code, then test.

--- bill

You wrote "I am just not a "trial and error" person.
For my projects I go by a more traditional process:
define requirements, plan, design, code, then test.".

That alone describes a looping on project management. Your thought process goes through the same loop every project. Why is that? Because it works and it avoids errors.
My data processing carrier was writing programs that processed various records, one at a time. From log scaling to payroll to automobile insurance to various banking applications to the final one which was high speed check processing. In every case it was doing everything over and over, for each individual record or for each check in the MICR reader. How would your bank function if it only ever processed your record if there was a check or withdrawal or deposit causing every account to be processed. For banks, every record must be processed every day/night to look for timed situations that may require some process. That is the ONLY WAY to prove the integrity of the files. There are no interrupts that can predict when a loan payment is past due or a service charge must be made or interest must be paid to the account.
Do you remember ever having to print a data file that was stored on some device? Did you tell the print program to only print on interrupts that you gave it? No, you printed from some starting location and looped through all the positions you wanted and then stopped. Then you looped through the print to find the data you needed to see. All being done by looping.
Paul

Paul,
You have completely and utterly totally missed my point related to looping.
Using or doing loops was not the point at all.
I was was pointing out that designing a system that uses that uses simplistic code running in a tight/fast loop that is constantly polling things vs using smarter & better coding techniques can be problematic once the system starts to get complex or when additional features need to be added that requires using or interacting with other s/w that cannot guarantee the timing requirements of such a simplistic polling design.

The other was that I see a lot projects and products these days using, IMO, poor product development methodologies and processes and taking short cuts, often driven by trying to push things out faster, that leads to more issues and failures in the field.

--- bill

Then, I guess you will have to provide some real-life examples of what you are writing about.
Paul

Hi Bill,

I think there is confusion here between hobby and professional programming. Some of your concerns might well be reasonable for a professional environment, but this site is not that. Sure we can discuss those things but in the end this site and the Arduino project are about teaching and a platform for hobbiests to use to animate their projects. A lot of the people we see on here are not even that interested in electronics, for example we might see and artist wanting to add interesting lights to an art installation they are making. Their thing is art, not electronics, but they realise electronics is another tool they can use to develop their art. Those people are not going to be interested in good software development practice, they are going to be interested in what make their project work. Take me for example, I keep away from Github because it confuses me. I see it as a useful, professional tool but one I have little use for, so I avoid it. Maybe one day, when I am doing something different, I will engage with it, but right now I don't want or need to.

PerryBebbington:
I keep away from Github because it confuses me. I see it as a useful, professional tool but one I have little use for, so I avoid it. Maybe one day, when I am doing something different, I will engage with it, but right now I don't want or need to.

You would be surprised how much your productivity can increase by using source control tools, even on personal projects.
I have not done any project professionally or personally without source control since around 1983 and would encourage everyone to always use source control tools for all their projects.
The tools back in the 80's were not nearly as capable as git is today.

Tools like git and meld are definitely productivity boosters.
meld is great for comparing files or entire directory trees.

git is a amazing tool but it does have a high learning curve; however,
I would still highly encourage you to reconsider your stance on using source control tools as there are some gui based tools that wrap around git that are very easy to use.
Not only can it make you more productive but it can provide protection against loss since you can sync the code to a cloud repository.

There are several gui based tools available that make using git (and hence github), very easy and natural to use without ever having to use the command line or having to spend a bunch of time to really learn git.

And while these tools only provide for a basic set of capabilities, for most uses, is it more than adequate.
The best easiest ones to use, IMO, integrate into the native OS file manager so you can see colored icons that indicate if a file has changed.
And if a file has changed you can click on it and see the changes made outlined in color.
You can do all the basic operations, like clone, commit, add, remove, tag, push, pull all with typically a single click of the mouse.
These tools make using source control super easy.
Without these types of tools, it is nearly impossible to keep track of what has changed.

One of the really nice capability of git is that it can work standalone or with a remote repository like github.
So if you don't ever want to use a remote repository like github, bitbucket, sourceforge, atlassian, etc... you don't have to and can keep everything local. A network repository is nice as it can provide a backup against your local machine having an issue.
Using a networked repository like github is nice not only for backup purposes, but also for sharing code your code with others and for tracking issues.

For linux there is RabbitVCS, and for Windows there is TortoiseGit
I have used both and both are really nice and easy to use as they hide all the underlying git commands and there complexities.
I would really encourage you to take a look at using source control tools and take a look at these tools as they make using source control super easy.

--- bill

Paul_KD7HB:
Then, I guess you will have to provide some real-life examples of what you are writing about.
Paul

For the case of looping vs using better but more advanced/complex coding techniques
here are couple of simple ones that show up every day here on the forum.
Things like use of delay() vs telling the user to use "blink without delay" technique.
Or people suddenly wanting non blocking i/o since all they are used to doing is putting everything into the loop() code and the i/o libraries they are using are synchronous and can block for the i/o to complete for a longer period of time than their loop() code can tolerate.

These are examples where using a system design of doing simple loop spinning and/or looping to poll on things no longer works.

IMO, while I see this quite often, telling Arduino users to use "blink without delay" technique, it is kind of cruel and for the most part I feel it is BS.
I say that because Arduino is supposed to be about making things easy, particularly since the focus of Arduino is for less technical users, users that are not really programmers that just want to integrate in some computerized functionality.
And pushing them to "blink without delay" is pushing them to an advanced type of coding that could be beyond their current capabilities.

IMO, Arduino should include some type of minimal scheduling and timer capabilities as part of the core library.
i.e. the ability for the user to schedule some code that can run on a periodic basis or be trigger to run from other parts of their code.
And no this does not require an actual OS with true task scheduling, it can be done with the existing limited Arduino framework capabilities.

There are some Arduino libraries out there that provide timer capabilities, or metro like scheduling but, IMO, none of them really are complete enough to provide many of the capabilities that users tend to often need.

I have often considered writing a library to do this. I probably should just go ahead and do it.

--- bill

Hi Bill,

Thanks for the info but nothing you have said fits with what I do or need. My PC is backed up to separate and separately stored hard drives every week. My biggest project is in MPLABX, which has tools for seeing changes and previous versions. I am the only person who works on my code, so if something changes it's because I changed it, so until I loose all my marbles I will know what has changed!

My Arduino stuff is in the Arduino IDE, which lacks the tools MPLABX has, but then my projects are much smaller. I have lots of little Arduino projects, I don't feel the need for anything fancy.

For anyone else reading this don't let me put you off Github, it is quite clearly a useful tool for some people, I just have not found the need for it.

Bill, if I understand your post, you solution is to hide all the details you detest inside "libraries" so the user can't see behind the curtain/
Paul

Bill,

Here is my code, it is not very responsive and often pressing the buttons does not cause anything to appear in the serial monitor.

const uint8_t BUTTONPIN = 2;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode (BUTTONPIN, INPUT_PULLUP);
  pinMode (LED_BUILTIN, OUTPUT);
}


void loop() {
  // put your main code here, to run repeatedly:
  if (digitalRead(BUTTONPIN) == LOW) {
    Serial.print("Button pressed");
  }
  digitalWrite(LED_BUILTIN, HIGH);
  delay(5000);
  digitalWrite(LED_BUILTIN, LOW);
  delay(5000);
}

I am new to coding and have no idea why this is.
Please help.
Thanks, A Noob.

So, serious point, how would you help someone with this without pushing them to blink without delay and similar techniques?

Paul_KD7HB:
Bill, if I understand your post, you solution is to hide all the details you detest inside "libraries" so the user can't see behind the curtain/
Paul

Paul,

I think that uncovers that there are (at least) 2 kinds of Arduino user: those that want to learn to code and those that want to get their project working somehow but are not fundamentally that interested in coding. For the first hiding what happens is bad, for the second hiding what happen is probably good.

PerryBebbington:
Paul,

I think that uncovers that there are (at least) 2 kinds of Arduino user: those that want to learn to code and those that want to get their project working somehow but are not fundamentally that interested in coding. For the first hiding what happens is bad, for the second hiding what happen is probably good.

That is likely correct. For the second, there are also the graphical way of programming for the Arduino that an occasional forum poster has mentioned. I have never felt inclined to look at them.
Paul

I hope this is not seen as interrupting, but ...

I totally agree with the fact that using a source code management system can improve productivity even for individuals. I first used CMS on VAX/VMS in the eighties and whilst there was no GUI available then, the improvements it made for those of us using it surprised all of us. I went on to use Sourcesafe (windows) in the nineties and more recently SVN and ToroiseSVN (Linux & windows).

I found that those that were most against using such as system always claimed they could remember all the changes they made, however, if they were forced to rollback a couple of days it nearly always proved impossible for then to do so easily. You only have to look at say MS Word and the ability to track edits to see the same sort approach used in a different field (great for checking supplier quotes to see what they have done if they are none too familiar with Word).

Whilst very simple projects can sometimes be completed in on go (with few a revisions to fix typos,bugs etc.) if any project spans multiple days, source code management even if just multiple copies be stored for yourself makes for much better productivity when things go astray.

countrypaul:
I hope this is not seen as interrupting, but ...

Not interrupting at all! I started this because I wanted a discussion.

I'm still not persuaded.... :o

Paul_KD7HB:
Bill, if I understand your post, you solution is to hide all the details you detest inside "libraries" so the user can't see behind the curtain/
Paul

For timers and scheduling of event based driven stuff yes.

PerryBebbington:
For anyone else reading this don't let me put you off Github, it is quite clearly a useful tool for some people, I just have not found the need for it.

Lets not confuse github with using source control.
github is just one of many platform sites that supports git repositories. It also has a web portal that can be used for simple git functions on repositories stored on github, as well as some other capaiblities.

Using source control tools does not require using github.
You can use git on other network git sites like bitbitbucket, etc...
git also allows you do everything local with no remote repository.

But git is not the only source control tool. While I happen to think it is the best one at the present time, other tools like SVN or mercurial as well as few others are also still being used.

What is more important is the concept of using source control management.
Being able to track changes and tag revisions. This is what can improve productivity as you will no longer spend any time trying to get back to that code that was working so well a few days ago that is now no longer working.

--- bill