Designing a new programming language for Arduino

potentiometer1 = create Potentiometer on pin 1

Take this line for instance. How is this any easier than creating a potentiometer class in C++ and putting it in a library for the noobs so they can write.

Potentiometer myPot(1);  // creates a pot on pin 1

and later

int val = myPot.read();

You've just added 3 new keywords to learn and get right, create, on and pin.

Delta_G: potentiometer1 = create Potentiometer on pin 1

Take this line for instance. How is this any easier than creating a potentiometer class in C++ and putting it in a library for the noobs so they can write.

Potentiometer myPot(1);  // creates a pot on pin 1

and later

int val = myPot.read();

The point of this is not just a different syntax, but different approach to the whole program structure. Everything follows a strict convention and a lot of the stuff comes 'out of the box' Timing is handled for you, so you don't need to worry about delay() halting your program flow.

You would loose some flexibility, but I think many things can be taken out if the language is aimed at beginners (pointer arithmetic for example)

Here is how I see it working:

device is the main entity in the language (think Class)

Every device follows this structure:

[i]device[/i] [b][color=green]DeviceName[/color][/b] ( [i]constructor parameters[/i] ) {

    [color=#5f3e31]write:[/color] pin1, pin2, pin3 [color=gray]// pins that the device can write to[/color]
    [color=#5f3e31]read:[/color] pin1, pin2, pin3 [color=gray]// pins that the device can read from[/color]

    [color=gray]// Here go predefined blocks that are built into the class[/color]
    [color=#5f3e31]init[/color] { ... } [color=gray]// basically constructor[/color]
    [color=#5f3e31]actions[/color] { ... } [color=gray]// all object 'methods'[/color]
    [color=#5f3e31]always[/color] { ... } [color=gray]// this code is executed all the time[/color]
    ....

    [color=gray]// then goes a list of all 'events' - code that executes when a condition is satisfied[/color]
    [color=gray]// they all start with keyword 'when'[/color]
    [b]when[/b] pin1 [b]is[/b] HIGH { ... }
    [b]when[/b] pin2 [b]is <[/b] 512 [b]and[/b] pin3 > 128 { ... }
    [b]when[/b] pin4 [b]is[/b] RISING { ... }
    ...

}

So there isn't much code apart from within the devices. (There can be other entities though, like timers for example, with their own set of "actions" and "events")

There would be a small runtime that would take care of timing and executing events. PS I do appreciate any criticism though, as it is entirely possible that I am missing some huge flaws in this.

YemSalat: PS I do appreciate any criticism though, as it is entirely possible that I am missing some huge flaws in this.

Absolutely. It's nothing personal. Every idea needs skeptics. That's how we weed out the shite. We don't always see the flaws in ideas we're too close to.

I'm with you that it might be fun to create a new set of syntax and for some projects it might be useful. I don't see how procedural code would work here once the plan got much bigger than read the pot, scale it, and write PWM to the led. I'm afraid this would run out of usefulness once the code got any bit of complicated and then the noob would have to go back to square 1 learning C++.

Delta_G: I don't see how procedural code would work here once the plan got much bigger than read the pot, scale it, and write PWM to the led. I'm afraid this would run out of usefulness once the code got any bit of complicated and then the noob would have to go back to square 1 learning C++.

That's a fair point, that's why I am trying to come up with various use cases where it can be applied. Overall I am planning to get away with 3 main entities for now: device, timer and state machine. I just haven't settled on all the concepts yet. If someone is planning on writing a web server or something like that - this would most likely not be the right tool for the job. I think it would be more well-suited for home automation projects and things like that.

Also, its worth noting that a lot of those projects never get "much bigger than read the pot, scale it, and write PWM to the led"

@YemSalat, what you are proposing will not really benefit from discussion.

If you can start a Thread in the Other Software Development section with a working version of your "transpiler" so that people can try it out you may get converts.

I would like a Python-like syntax simply so that I don't get confused switching between Python and Arduino code.

I was almost tempted to say I would like a Ruby-like syntax, except that I abandoned Ruby in favour of Python simply because Python has a better infrastructure and is more widely used (at least in the Arduino context).

...R

Have you looked at https://micropython.org/ ? I keep meaning to and see if its a good fit for this kind of use (presumably its meant to be!)

I am a newbie to the Arduino IDE and am completely baffled by it's complexity. I think that C++ is a great language for experienced programmers, but lousy for beginners. Its syntax is cryptic, and mistakes are easily made, however, even with mistakes, the compiler still may be able to make sense out of it, resulting in incomprehensible results. I think you should completely abandon C related languages as a choice, and make it more BASIC related. BASIC may make it slower, but can be improved by compiling it, rather than interpreted. It also has the advantage of being restricted in how many Statements can be used, not like C where there are unlimited functions available through its Libraries, but which are not that easily found by newbies. BASIC has also the inherent advantage in that it is very user friendly and easily learned by non-programmers.

What do you think? Do you reckon any of these ideas are worth considering?

What in your opinion should a beginner-oriented Arduino language look like? Is such a language even needed?

OK, you asked... Here are preamble statements : Arduino is a brainchild of schoolteachers, albeit nobody will admit to that. They put together nice piece of hardware, but stopped short to provide anything but very basic software. ( Most "libraries" are by folks outside Arduino )

That is on positive side of things.

On negative side - they bastardized perfectly workable language by hiding the foundation in the name of " not confusing the beginners". ( see the latest on 1.6.6. release) The language foundation - C - is nicely documented in a skinny book of around 200 plus pages. It is all in there and anybody who WANTS to learn it can read it over the weekend. The C++ is just an addition , but EVERYBODY needs the foundation first.

Any time you scan this forum you can see that some folks just do not want to learn the foundation. They just want to run robots!

You will not move Arduino from kindergartner level to first grade by having another language.

And I am not trying to discourage you, just telling the way I feel about things.

You do whatever floats your boat, but without Arduino.cc first removing the "do not confuse beginners " attitude you will be on your own.

Personally , if you could became member of Arduino inner circle of developers you could contribute more to everybody's benefit.

Cheers Vaclav

What I think might be more helpful would be a more noob friendly editor for the IDE. Something that spits out error messages that sound more like advice.

error: void value not ignored as it aught to be

could either be, or have in addition

this function does not return a value, so you can't assign it to this variable

Something that spits out an extra level of warnings like:

WARNING:  non-volatile variable never changes in body of while loop.  Loop will be infinite.

or even the ability to let people step through code so they can see their stuff getting hung up in a while loop or why an if statement seems to be "skipped".

I know in the java world there are code checkers that can even tell you for instance that a method is too long or does too many things and should be broken into individual functions. Or that you've done something else stupid. A code checker could be very useful for the noobs.

Basically, what I think would be useful would be to go through @PaulS's last 5000 posts and take the top 20 or so things he says and make compiler warnings out of them.

@MarkT will have a look, cheers

@hbl2015, I am not sure if basic is a good base, but I agree that having to look for various libraries can be confusing and it can be nice to have some 'essentials' built into the language

@Vaclav, thanks for sharing your opinion on this, but to be honest I have a pretty low chance of getting into the Arduino hq (not that I would want to anyway)

@Delta_G, I think it may actually be harder to rewrite all C++' errors and warnings then to make a separate language oO

@Robin2, I will probably have a go at writing a very basic transpiler in the next few days, which would allow you to just copy the code to IDE. I won't implement the main framework yet cause I have not completely worked it through, but I will try to add a bit of syntactic sugar at first (implicit types, nicer for/if statements, etc.)

I think that however the Arduino is programmed there is a problem with the mixed background of its users. I came to it with no experience of C let alone C++ but I had used other languages in the past, mostly various flavours of BASIC including VBA, Z80 assembler and monstrosities such as Mumps. As a result I needed to learn the C/C++ syntax for constructs that I expected to be there but logic is logic whatever language is used.

It is difficult to be certain, but were I in the position of a complete novice then from what I have seen of the proposed new language I don't think that they will be any better off. One problem will always be that, by definition, computer languages are created by experienced computer programmers. What sounds simple and logical to them is gobbledegook to outsiders.

UKHeliBob: I think that however the Arduino is programmed there is a problem with the mixed background of its users. I came to it with no experience of C let alone C++ but I had used other languages in the past, mostly various flavours of BASIC including VBA, Z80 assembler and monstrosities such as Mumps. As a result I needed to learn the C/C++ syntax for constructs that I expected to be there but logic is logic whatever language is used.

I think your situation is slightly different because you already were familiar with programming logic and concepts before you started with C/C++. For many people here - C++ is the first language that they learn, and I think it is a bad choice for the first language (even a bit of prior experience with Basic would help a lot in my opinion)

It is difficult to be certain, but were I in the position of a complete novice then from what I have seen of the proposed new language I don't think that they will be any better off. One problem will always be that, by definition, computer languages are created by experienced computer programmers. What sounds simple and logical to them is gobbledegook to outsiders.

This is also a fair point, hopefully I can make it more useful and easy to understand, that is why I really appreciate any feedback.

YemSalat: I will probably have a go at writing a very basic transpiler in the next few days,

I forgot to say that you also need to write the book that explains how to use your language.

Actually I think you should write the book first and then make a language that complies with the book. As you mentioned Ruby earlier you may be familiar with the "pickaxe" book - Programming Ruby by Dave Thomas. That is the sort of thing I have in mind - but maybe yours would not need 800 pages for Version 0.1

IMHO without proper documentation there is no point writing a language aimed at making things easier. And by proper documentation I mean explanations of why you should do things and what is the purpose of each thing, not just a few examples that each show a single "how" to use a feature with no attempt to show alternatives or exceptions.

...R

I really appreciate any feedback.

Let me know when you have something to test and I will assume my alternative identity of "British Standard Idiot" and put it to the test.

@Robin2 We'll see how it goes, I think version 0.1 will fit on a couple pages :)

@UKHeliBob hehe, that kind of attitude would certainly be helpful! I will post a separate topic in the "Projects" section once I have something to show.

A small update. So far I have settled on the tools, I will be using Jison to generate the parser - http://jison.org Jison is a JavaScript port of Bison compiler-compiler, I have been working mostly with nodejs in the past couple years, so I feel most comfortable writing the parser in JS. It also completely solves the cross-platform compatibility issue, as it will work across all operating systems out of the box.

The parser will build a syntax tree, then I will convert it to C/C++ code which can be just copy-pasted into the Arduino IDE.

For the first "milestone" I am planning to accomplish the following:

  • optional (inferred) types for variable declarations
  • some default types - int, float, long (Strings might have to wait until the next one)
  • if / else statements
  • for loops
  • functions
  • some basic built-in functions for pin manipulation

So, this would be just a 'proof of concept' version, nothing too serious. I will upload it to github once I have something to upload :)

No timeframe for now, but hopefully I will present something by the end of the week.

YemSalat: @Robin2 We'll see how it goes, I think version 0.1 will fit on a couple pages :)

Within reason, the more pages in the manual the better. Most open source software lacks adequate documentation.

...R

@Robin2, if the project goes well - I will definitely make sure to write proper documentation and prepare some starter resources.

A quick update.

I switched from Jison to pegjs for the compiler (http://pegjs.org) as it supports PEG notation which is much simpler to write. It also has some really useful examples: https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples Even a complete grammar for C!

So far I added the following features to the language parser:

  • declaring variables (6 types so far: byte, float, int, long str, bool)
  • 'optional' types
  • optional semicolons
  • assignment / comparison / math operations
  • if and switch statements
  • for loops
  • literals for time (like: [b]1m 15s[/b] - converts to an integer containing the number of milliseconds)
  • wait and every constructs for basic timing (like: every 5s { /* ... */ })

I am yet to build the compiler itself (I am also planning a separate evaluator), but the parser seems to work pretty well so far (couple small issues, but I know how to fix them).

Here is a simple example, this expression:

a = 5
b = (a + 2) * 3

Will generate the following syntax tree (in JSON format):

{
   "$": "COMPILE",
   "body": [
      {
         "$": "EXPRESSION",
         "expression": {
            "$": "ASSIGNMENT",
            "operator": "=",
            "left": {
               "$": "IDENTIFIER",
               "name": "a"
            },
            "right": {
               "$": "LITERAL",
               "value": 5,
               "type": "int"
            }
         }
      },
      {
         "$": "EXPRESSION",
         "expression": {
            "$": "ASSIGNMENT",
            "operator": "=",
            "left": {
               "$": "IDENTIFIER",
               "name": "b"
            },
            "right": {
               "$": "MATH_EXPRESSION",
               "operator": "*",
               "left": {
                  "$": "MATH_EXPRESSION",
                  "operator": "+",
                  "left": {
                     "$": "IDENTIFIER",
                     "name": "a"
                  },
                  "right": {
                     "$": "LITERAL",
                     "value": 2,
                     "type": "int"
                  }
               },
               "right": {
                  "$": "LITERAL",
                  "value": 3,
                  "type": "int"
               }
            }
         }
      }
   ]
}

The next thing that I am quite excited about - is that I started creating and IDE for the language, and I just keep realizing how useful it can be for beginners. I am using Ace (https://ace.c9.io/) for the text editor and it already has a ton of useful features built in (like autocomplete!), but what is even more important - I can evaluate all the code in 'live' mode and detect errors right away, without having to 'compile'. Ace has some nice APIs for code highlighting, etc.

Granted that the parser is very simple so far, but I did a few tests and I am able to run it in a web worker (it allows running multi-process code in JavaScript as JS is single-threaded by nature) - and it parses 1000 lines of code instantly as I type (I will share a demo soon)

I am not going to spend too much time on the IDE right now, just gonna build some basic stuff around the text editor so its easy to test the language when the first version is out.

Again, no time frame so far, but aiming to release something by the end of the weekend or early next week.

I will also definitely need help with optimizing the resulting C/C++ code, hopefully there is enough people around here who can help with that :)

If those two lines require a tree like that the world's forests will soon be devastated. :)

...R

Robin2: If those two lines require a tree like that the world's forests will soon be devastated. :)

...R

Hehe :) thats just because the language is low level and not dynamic, so all the instructions are quite explicit and almost always can convert straight to C/C++ without much modification. Anything more 'sophisticated' would require a more complex runtime which we just can't afford on the Arduino. (Also JSON format makes it look bigger then it is)

Just for the heck of it, I generated parse trees for the same expression in Python and C.

Original expression:

a = 5
b = (a + 2) * 3

Python AST (python has a built-in AST generator)

Module(
    None,
    Stmt(
      Assign(
        AssName('a', 'OP_ASSIGN'),
        Const(5)
      ),
      Assign(
        AssName('b', 'OP_ASSIGN'),
        Mul(
          Add(
            Name('a'),
            Const(2)
          ),
          Const(3)
        )
      )
    )
  )

C (generated with clang)

TranslationUnitDecl 0x10302d6c0 <> 
|-TypedefDecl 0x10302dc00 <>  implicit __int128_t '__int128'
|-TypedefDecl 0x10302dc60 <>  implicit __uint128_t 'unsigned __int128'
|-TypedefDecl 0x10302e020 <>  implicit __builtin_va_list '__va_list_tag [1]'
`-FunctionDecl 0x10302e0c0  line:1:5 main 'int (void)'
  `-CompoundStmt 0x10302e3d0 
    |-DeclStmt 0x10302e248 
    | `-VarDecl 0x10302e1d0  col:7 used a 'int' cinit
    |   `-IntegerLiteral 0x10302e228  'int' 5
    `-DeclStmt 0x10302e3b8 
      `-VarDecl 0x10302e270  col:7 b 'int' cinit
        `-BinaryOperator 0x10302e390  'int' '*'
          |-ParenExpr 0x10302e350  'int'
          | `-BinaryOperator 0x10302e328  'int' '+'
          |   |-ImplicitCastExpr 0x10302e310  'int' 
          |   | `-DeclRefExpr 0x10302e2c8  'int' lvalue Var 0x10302e1d0 'a' 'int'
          |   `-IntegerLiteral 0x10302e2f0  'int' 2
          `-IntegerLiteral 0x10302e370  'int' 3

As you can see, C generates a more 'verbose' AST, but it can be converted almost directly into assembly. While Python's AST is more concise - it requires a massive runtime to support all of the entities.