Arduino Forum

Development => Other Software Development => Topic started by: liuzengqiang on Apr 01, 2011, 06:20 pm

Title: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Apr 01, 2011, 06:20 pm
After some digging in the playground, I realized what I just released, a text-based user interface library (now only limited to select from list, menu, enter number, enter strings) does not belong to a proper category. So I created a sub category under "interface with hardware", called "User interface". I've populated it with GUI, TUI, and TUI->my phi_prompt library. I'll be considering porting my library to 20*4 character display and ks0108 dot matrix display.

Are there anyone else interested in turning part of your program (user interacting part) into a UI library? GUI (kind of hard for arduino) or TUI? We could discuss what elements we could include and what hardware the interface can support. Kind of make it into a Java thing that runs across several display and keypad/buttons hardware with the same look and feel to developers and users.

Some good-old TUIs:

http://en.wikipedia.org/wiki/Text_user_interface
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Apr 01, 2011, 09:31 pm
hum de hum.

I'd be interested to see how well a UI library might work.

I did kinda start a scrolling menu program for LCDs (16x2, 16x4, 20x4, 40x4) but didn't require it for anything so it never materialised.

I'd say a library might work - provide the LCD rows&cols (same ones used for the liquidcrystal library) and the number of options on the main menu - then functions for secondary menus perhaps.

You could even have a custom character 'scroll bar' down the side.

Obviously a graphic display menu library/UI could have much more too it and be a bit more 'UI' like.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: bubulindo on Apr 01, 2011, 11:06 pm
Remember to debounce the buttons... that's the major pain normally. :(
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Apr 01, 2011, 11:40 pm
Quote
Remember to debounce the buttons... that's the major pain normally.

It is?

I normally just use a while(button == HIGH){} to wait until it goes low again before it does anything.

Depends on the application really though.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: maniacbug on Apr 02, 2011, 12:25 am

I normally just use a while(button == HIGH){} to wait until it goes low again before it does anything.


As long as you're willing to wait for the 'up'.  That feels like input lag to me, so I usually use 20ms debounce.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Apr 02, 2011, 12:42 am
Quote
As long as you're willing to wait for the 'up'.  That feels like input lag to me, so I usually use 20ms debounce.[/quote[
I've never felt this seemed laggy.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Apr 02, 2011, 04:37 am

hum de hum.

I'd be interested to see how well a UI library might work.

I did kinda start a scrolling menu program for LCDs (16x2, 16x4, 20x4, 40x4) but didn't require it for anything so it never materialised.

I'd say a library might work - provide the LCD rows&cols (same ones used for the liquidcrystal library) and the number of options on the main menu - then functions for secondary menus perhaps.

You could even have a custom character 'scroll bar' down the side.

Obviously a graphic display menu library/UI could have much more too it and be a bit more 'UI' like.


mowcius,

I think having a UI will be a blessing for lots of developers. I recall spending lots of time develop the UI on my first project (2,700 lines). But I don't think I can separate the UI out like taking off a sweater :)

I then developed programs that do some footwork of UIs and eventually got the to point of phi_prompt, a library on 16*2 LCD with select list, menu (just select list + extra code), text input, number input.

I did the call convention you suggested but switched to calling like: select_list(a structure)
This way you can add features without changing call conventions. If the new features are not changing previous releases but only affect new functions, then there will be no need to rewrite :)

Here is the phi_prompt post:
I am thinking about adapting this to 20*4 displays but I don't have one :(

http://arduino.cc/forum/index.php/topic,56754.msg407552.html
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Apr 02, 2011, 11:59 am
If you want to modify it to include 20x4s/16x4s/40x4s then I can test it on them.

I'll see if I can make a start on a scrolling LCD menu code.

Quote
But I don't think I can separate the UI out like taking off a sweater

I know what you mean.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: bubulindo on Apr 02, 2011, 12:51 pm

Quote
Remember to debounce the buttons... that's the major pain normally.

It is?

I normally just use a while(button == HIGH){} to wait until it goes low again before it does anything.

Depends on the application really though.


Actually depends on the buttons... with the cheapest kind of buttons, this would fail.



I normally just use a while(button == HIGH){} to wait until it goes low again before it does anything.


As long as you're willing to wait for the 'up'.  That feels like input lag to me, so I usually use 20ms debounce.


with cheap buttons, this could also fail, and adds a bit more to the code. Either because you halt the processor during 20 ms, or because you need to run a timer to check this.

So the way to go, is really a mix of quality buttons, some debounce tecnique in hardware, either with the capacitor or some gates (there's an application note somewhere about this) and some software to check it. Depending on the buttons you use, you may need one or none of the others... but since this will be a generic library, you have to think of most, if not all, possibilities regarding the buttons.

I implemented this (with a 16x2 or 20x2, not sure now) with menus, submenus, and a 8 key keypad for a coffee machine about 10 years ago. And back then, the major problem was the debounce because we used some top of the line buttons (1,3 euro each) on the prototype (the customer decided on those because of the incorporated LED, feel and size with interchangeable keys) and debouncing didn't even cross my wildest dreams... until, because of cost concerns and because the key would pop out when pressed, they decided to go with cheaper buttons... and the nightmare began. LOL

But if you use those normal Omron type keys you might get away with little worries about the debounce. The best to do, get an oscilloscope, a few buttons and test them to see what happens and create a debounce strategy then.

Just a thought, get the program to run as a state machine with methods to move level (sub-menu) up, down and enter. That way, the library could (but wouldn't need to) have the button code included, but it would make it easier to adapt if you'd like to use a blackberry tracker ball or a small joystick, for example. ;)

Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Apr 02, 2011, 01:15 pm
Quote
Just a thought, get the program to run as a state machine with methods to move level (sub-menu) up, down and enter. That way, the library could (but wouldn't need to) have the button code included, but it would make it easier to adapt if you'd like to use a blackberry tracker ball or a small joystick, for example.

Sounds like you're not short of knowledge in this subject - fancy helping?

I also implemented something (a year or so back) which was similar but there's no way of 'extracting the code' as it was very specific (mainly due to being badly written).
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Apr 02, 2011, 05:44 pm
State machine is a good idea. I'm thinking about HMI standards in industrial control panels. I have one sitting on my couch.

To make the library small, I only implemented lists instead of lists and menus. To make a menu, just call select_list() to ask the user to select item from a list and switch upon the user selected list item :)

To implement multilevel menu/list, you just need to add another select_list() where the sub menu entry is. You also want to add a return in one of the sub menu's switch case.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: bubulindo on Apr 03, 2011, 12:16 pm
I think that the way to go would be to implement the methods to

- refresh screen
- next item (meaning the menu where we are now)
- previous item (or just implement a bounded buffer)
- enter
- back (this can be left out and in every submenu, there would be an option to move up on the menu tree... depends on the developer and number of options, really)

The user of the library would only have to fill out a structure that would have all the menu data in it, and the methods would merely cycle through that structure.

I can have a go at writing something for this... but not over weekend. ;)
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Apr 03, 2011, 07:08 pm
Good idea!

Right now my phi_prompt select_list() will take a list from PROGMEM and display it according to the parameters in the structure and interacts with the user (up button is next item, down is prev) until the user hits left, right, enter, or escape. So one function call then the return is final decision of the user.

The left and right buttons can be used in a larger display or just 16X2 to navigate between selections much like a dialog box with tab keys in Windows. Say you change one number and press left, you can change the previous field and right a few times to change another field. I have not implement that into an actual function of dialog box (including several interactive lists and number entries) but I'm already using the basic blocks in my clock adjustments:

I will just add a function that sets up a user interface like a dialog!

http://www.youtube.com/watch?v=L-3pKtf1VBE
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Apr 25, 2011, 08:45 pm
Just FYI, I'm updating this UI library to include the following things, more useful if you have a 16X4 or 20X4 display:
   1. Support 20X4 character screen with more flexible "2X2? list mode so you can list M columns by N rows with X  characters per row, all M, N, and X are your choice (Done)
   2. List your "2X2? list anywhere on a larger screen instead of the basic 16X2 screen (Done)
   3. Easy to call "YES/NO" dialog for user to choose.(Planning)
   4. Easy to display "YES/NO", "HIGH/LOW" options for numerical inputs so you don't have to construct a list and do it the list way.(Planning)
   5. Add scrolling texts that scrolls to fit in smaller space (planning)
   6. Add floating point number input (planning)
   7. Add event function so each time the user changes a number, a function is called to update operations (planning)
   8. Add dialog functions to make a dialog. This function creates a complete dialog with various controls and the user uses left and right keys to navigate among the controls and change their values. It is going to look like a dialog on your computer. You make some choices and push yes. (planning)

Any suggestions? Thanks.

Here is the link to the library:
http://liudr.wordpress.com/libraries/phi_prompt/
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Apr 25, 2011, 10:55 pm
Scrolling up/down menu? With custom character 'scroll bar' on the far right column?
That's what I'd like to see/code myself :)
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Apr 26, 2011, 12:43 am

Scrolling up/down menu? With custom character 'scroll bar' on the far right column?
That's what I'd like to see/code myself :)


Would it be scroll menu or scroll menu and scroll text display, like old-time DOS stuff?
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Apr 26, 2011, 10:53 am
I wasn't thinking of scrolling text (horizontally) too.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Apr 26, 2011, 04:39 pm
I just did a mockup of an OK dialog and Yes/No dialog. It's good on the 20X4 (should be good on 16X4) but will not be very good-looking on 16X2.
I'm also writing a lib init routine that takes lcd and buttons objects as we as lcd sizes parameters so my struct doesn't have to store them. Less space wasted but less flexible (you will have to reinitialize lib to work on a different LCD and different set of keys if you have multiple sets).

Will upload pics tonight.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on May 24, 2011, 11:12 pm
To follow up on this, I just released the latest version:

http://arduino.cc/forum/index.php/topic,62021.0.html

The videos are not very clear for the 20X4 display  :smiley-sad-blue: Will try videotaping again later.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: bubulindo on May 25, 2011, 03:24 pm

Scrolling up/down menu? With custom character 'scroll bar' on the far right column?
That's what I'd like to see/code myself :)


The only thing stopping that would be the size in memory to have all the user defined characters to be passed on to the LCD. :\

Ohh, sorry for not having enough time to dedicate to this. :(
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on May 25, 2011, 04:12 pm
Quote
The only thing stopping that would be the size in memory to have all the user defined characters to be passed on to the LCD. :\

There are ways round the limited number of characters...
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: bubulindo on May 25, 2011, 04:55 pm
I may have not explained myself correctly... I was talking about the size in Arduino's memory to hold the user defined characters... any formulas to create those characters that avoid the use of char arrays in memory?
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on May 25, 2011, 06:23 pm
Yes, create them in PROGMEM:
Code: [Select]
PROGMEM prog_char lcd_ch0[]={ 4,14,31,64,31,31,31,31,0};// Up triangle with block
PROGMEM prog_char lcd_ch1[]={ 4,14,31,64,64,64,64,64,0};//Up triangle
PROGMEM prog_char lcd_ch2[]={31,31,31,31,64,64,64,64,0};// Top block
PROGMEM prog_char lcd_ch3[]={64,64,64,64,31,31,31,31,0};// Bottom block
PROGMEM prog_char lcd_ch4[]={64,64,64,64,64,31,14, 4,0};// Down triangle
PROGMEM prog_char lcd_ch5[]={31,31,31,31,64,31,14, 4,0};// Down triangle with block
PROGMEM const char *ch_item[] = {lcd_ch0, lcd_ch1, lcd_ch2, lcd_ch3, lcd_ch4, lcd_ch5};



I've already made a scroll bar. Please watch some videos on the link I provided. There's only enough customized characters to create a rough scroll bar though. Dynamically shoving customized chars back and forth is too much work.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: bubulindo on May 26, 2011, 09:31 am
It's still in memory, isn't it?
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: sixeyes on May 26, 2011, 10:53 am
It's in program memory (aka Flash) not in data memory (aka RAM), so it's still using resources but one that there is more of.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on May 26, 2011, 06:06 pm

It's still in memory, isn't it?


So what is your point? Where would you store these data?
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on May 26, 2011, 06:08 pm

It's in program memory (aka Flash) not in data memory (aka RAM), so it's still using resources but one that there is more of.


Besides, the SRAM character array that is used to temporarily store the customized chars is in the init function so I suppose once that function exits, the array is gone and SRAM recovered.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on May 26, 2011, 09:55 pm
Quote
Besides, the SRAM character array that is used to temporarily store the customized chars is in the init function so I suppose once that function exits, the array is gone and SRAM recovered.

Correct - all that's needed is initialisation of those characters and then they are stored on the LCD, a simple request for that character to be written then recalls it from the LCDs memory to write to the display.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on May 30, 2011, 08:25 pm
Just added new line to long message display so now you can start a new line in a long message display with '\n'. Didn't know that would take me so long I was dreaming about it last night and figured it out this morning.

(http://liudr.files.wordpress.com/2011/05/text_area_03.jpg) (http://liudr.wordpress.com/libraries/phi_prompt/)

As you can see the first two lines have "new line" symbols '\n' after the last characters. I didn't put exact space to make it look like the picture!

It was not easy for something that probably nobody will ever notice.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: olikraus on Jun 06, 2011, 01:16 pm
Hi All

What a nice thread.
@liudr: Thanks for pointing out this thread.

So, to answer the inital question:
I currently work on a graphical user interface. It will be GPL, it is hosted here: http://code.google.com/p/m2tklib/ (http://code.google.com/p/m2tklib/)

History:
Sept 2010: Idea came up, as an addon for my dogm128 library
Okt/Nov 2010: Research on existing user interfaces: Not much open source, not specific to embedded systems, poor documentation. Except liudr's phi-prompt lib, but at that time it only has focus on text displays.
Nov 2010: Start working...

Goals:
- Licensed under GPL
- Small memory usage, original target was ATTINY with 16KB Flash, leaving enough space for the application itself
- Widget based approach (like any other major GUI), widgets are called "elements" inside m2tklib
- Message based communication
- lib itself in C, but also C++ API (full Arduino IDE integration)
- Portable according to any inputs (including a debounce algorithm), also portable according to the number of input buttons.
- Portable according to the display output: Support for Dogm128 lib finished. LiquidCrystal works.


Status:
Not yet released something, documentation is online (mostly). If there is some interest from community of this board, i can provide a alpha release.



Oliver


Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: olikraus on Jun 06, 2011, 01:49 pm
m2tklib example

here are some elements (widgets) arranged within the gridlist container (c2 = two columns)
Code: [Select]
M2_LABEL(el_label1, NULL, "red");
M2_RADIO(el_radio1, "v0", &select_color); 
M2_LABEL(el_label2, NULL, "green");
M2_RADIO(el_radio2, "v1", &select_color); 
M2_LABEL(el_label3, NULL, "blue");
M2_RADIO(el_radio3, "v2", &select_color); 
M2_BUTTON(el_cancel, NULL, "cancel", fn_cancel);
M2_BUTTON(el_ok, NULL, " ok ", fn_ok); 
M2_LIST(list) = { &el_label1, &el_radio1,     
&el_label2, &el_radio2,       
&el_label3, &el_radio3,     
&el_cancel, &el_ok  };
M2_GRIDLIST(list_element, "c2",list);


results are portable accross different output devices:
dogs102 (dogm128 lib):
(http://wiki.m2tklib.googlecode.com/hg/pic/dogm128/radio2.jpg)

16x4 LCD (LiquidCrystal Lib):
(http://wiki.m2tklib.googlecode.com/hg/pic/lcd/Radio.jpg)

Oliver


Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: mowcius on Jun 06, 2011, 05:56 pm
Neat :D
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Jun 06, 2011, 06:21 pm
Awesome! The code reads similar to java. My ultimate goal is to create something like a dialog or panel, with multiple inputs of different types, much like a dialog you see in java dialogs.

See this one I found randomly online:
(http://download.oracle.com/javase/tutorial/figures/2d/page-setup-dialog.gif)

There are lists, number entries, radio buttons, text entries and several buttons. Apparently I'm not going for their level of visual details but a rough text-based version.

You can use left/right or a tab to move among the fields on the panel and change values here and there.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: olikraus on Jun 06, 2011, 06:52 pm

There are lists, number entries, radio buttons, text entries and several buttons. Apparently I'm not going for their level of visual details but a rough text-based version.


This is what I wanted to do. m2tklib is expandable and in principle able to support any kind of input fields. Currently I have text input, radio/checkbox buttons, number inputs, normal buttons with several features (see here http://code.google.com/p/m2tklib/wiki/elref (http://code.google.com/p/m2tklib/wiki/elref))

There are also different container "widgets" which allow different approaches for the layout of dialog boxes. So except for the combo box, you could already implement the "page setup" dialog box.

The appearance of these elements has been separated from the dialog specification itself. It is the responsibility of the graphics driver to add some nice shadow borders or similar things to the elements.

Oliver
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Jun 06, 2011, 10:30 pm
Impressive. I'll take a lot of time to catch up with you. BTW, my documentation is final finished, after 33 pages. I sincerely hope someone will read it  :smiley-sweat: :D
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: olikraus on Jun 06, 2011, 11:27 pm
What i have seen so far is, that your phi-lib is a well known standard here. Well documented and proven in use. For any text based LCDs the choice #1. Additionally, some features like scrolling will probably never go to m2tklib. And finally, there is no release for m2tklib. While i hope that i can finish it soon, it is still vaporware...

Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: liuzengqiang on Jun 07, 2011, 07:39 pm
olikraus,

That was flattering. My library is not really a standard of any sort. I do have a decent manual though  :D
I've used LCD for quite some time and didn't see any libraries of user interface. I struggled to make my own and think lots have gone and will go through the same struggle so I made this library. It's anything but perfect although currently it's the only one posted here and on playground. Please forge ahead with your development. I will probably make a new release after the summer is over if I finish everything I plan to finish over the summer.
Title: Re: Anyone else developing a user interface (UI) on LCDs and keypads/buttons?
Post by: olikraus on Jul 04, 2011, 07:15 pm
Hi

First version of M2TKLIB is finished

see here: http://arduino.cc/forum/index.php/topic,65653.0.html

Project URL: http://code.google.com/p/m2tklib/

Oliver