Pages: [1]   Go Down
Author Topic: Proposal for a I/O API  (Read 1202 times)
0 Members and 1 Guest are viewing this topic.
Montreal
Offline Offline
Newbie
*
Karma: 0
Posts: 22
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

On the Arduino right now, there are many libraries to access different kinds of external or internal EEPROM or Flash memories but there's no single API to do that. It would be very nice to have a standard API with which to open, read and write to streams or files, similar to the fstream C++ library or the stdio (fopen, fprintf, etc).

The idea is to be able to use similar code wether you are writing to the uart, the EEPROM, a dataflash or an SD card.

Does such a library already exist? If not, I am ready to code one and I would like to use this post to discuss with the community about what is the best option.

I am considering two options.

1. Code a sub-class of Print.h that would implement additional functions for streams and files (such as open(), close(), rewind(), seek(), read(), etc.) -- which would permit, among other things, to use the Streaming library for easiness.

Code example:
Code:
Dataflash f;
int x;
f.scanf("%d\n", &x);
f.rewind();
f.printf(f, "Hello word\n");
f.close();

2. Use the sdtio library (fopen, fprintf, etc.) The nice thing with this is that it's pretty standard for C users. It would be "the way to go" if Arduino had been designed for C users, but it's not -- and that's a good thing most of the time -- so it means that it's not very standard for Arduino users.

Code example (I don't even know if we can do that, I'm not super familiar with the stdio library):
Code:
FILE* f = open_dataflash();
int x;
fscanf(f, "%d\n", &x);
rewind(f);
fprintf(f, "Hello word\n");
fclose(f);

Any ideas, suggestions, questions?
Logged

Left Coast, CA (USA)
Offline Offline
Brattain Member
*****
Karma: 361
Posts: 17263
Measurement changes behavior
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I would think that a standard API that can write serial data to either serial devices or memory devices would be very difficult, if even possible. Memory devices need to be told what address to write new data too, and somewhere something has to keep track of what address already have valid data stored in them, next free address, how to erase previous used memory location, etc.

I'm not a software guru at all, but what you are asking for seems darn near impossible, if not illogical?

Lefty

Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 212
Posts: 13531
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
The idea is to be able to use similar code wether you are writing to the uart, the EEPROM, a dataflash or an SD card.

It can be done but would it be worth the footprint it takes?? serious doubts.
It means for external EEPROM another layer e.g. around the I2C library costing extra scarce memory. For some applications it makes sense to do so, but not for all. Would it make (coding)  life easier? doubt it seriously

imho an EEPROM is more like one big array!  so why no array interface iso file interface? An SD card can have both an array interface (raw device) and a file interface.

my serial out cannot be rewind() so the 'file pattern' breaks quite fast.

I prefer every type of device its own interface, then I recognize them at least in the code smiley


Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Montreal
Offline Offline
Newbie
*
Karma: 0
Posts: 22
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

First, I agree that there is a difference between streams and files. Of course, it's impossible to rewind or fseek the uart for example. It doesn't make the thing impossible though: it could easily be solved by using a class hierarchy comparable to that of the C++ iostream lib; or, if you just want one top class, you can just make those methods (rewind / fseek) do nothing or been unspecified in the case of streams.

Second, I don't really see why there would be a huge memory footprint, except maybe if we use solution #2 which uses <stdio.h> which has in itself a huge footprint from what I have read. The top (abstract) class would take no more space than the Print.h class, the bridge classes probably would take some space, but not disproportionately.

I personnally don't want to have to reimplement a big bunch of my code at the risk of making mistakes just because I switched from saving my data to the EEPROM to saving it to a SD card or another device. Of course, there's a cost for abstraction in terms of memory usage; the advantage is faster development and less risks of errors. And if you really end up feeling short on space, you can always switch back to a lighter, low-level library.
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 212
Posts: 13531
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

still not convinced, sensors could also get a file interface, oven.scanf("%f", &temperature);  .... and IO pins.. Why not, reading from an IO pin just happens to be one bit at the time, analog pins read 2 bytes at a time. I can do digitalPin[12].print("1"); of analogPin[2].scanf("%d", &x);

but seriously build a prototype and suprise me .... The proof is allways in the pudding test !
Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Montreal
Offline Offline
Newbie
*
Karma: 0
Posts: 22
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I didn't know, but the last version of Arduino actually includes a Stream abstract superclass from which Serial derives.
http://code.google.com/p/arduino/source/browse/trunk/hardware/arduino/cores/arduino/Stream.h

(although, I think they should have included the peek() function to it).

So, the API is basically there for streams. What is missing is a subclass of Stream for files (or file-like mediums) that give extra methods (seek, tell, eof). While we're at it, I would also add functions to read/write blocks, and possibly to read/write directly at a specific address.
Logged

Montreal
Offline Offline
Newbie
*
Karma: 0
Posts: 22
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Yeah you're right I'll put something together and we'll see.
Logged

'round the world...
Offline Offline
Faraday Member
**
Karma: 42
Posts: 3225
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset


2. Use the sdtio library (fopen, fprintf, etc.) The nice thing with this is that it's pretty standard for C users. It would be "the way to go" if Arduino had been designed for C users, but it's not -- and that's a good thing most of the time -- so it means that it's not very standard for Arduino users.

Code example (I don't even know if we can do that, I'm not super familiar with the stdio library):
Code:
FILE* f = open_dataflash();
int x;
fscanf(f, "%d\n", &x);
rewind(f);
fprintf(f, "Hello word\n");
fclose(f);

Any ideas, suggestions, questions?

There is an stdio.h that you can use, here's the documentation http://www.nongnu.org/avr-libc/user-manual/group__avr__stdio.html. If you look up examples of it around the internet you'll see that as long as you define a function to "put char" and "get char", you can use printf to wherever you want. Although, it might only be available for one stream (file if you wish).
Although, I do remember thinking stdio.h was the sliced bread of microcontrollers until I saw the memory footprint it left. But then again, at the time I didn't bother looking up the definitions of the compiler regarding floating point conversions and such... so that may also have an impact.

But, give it a try. Even if it's not feasible, or limited, it isn't a waste of time, since I'm quite sure that the 2560 would benefit form such implementation, or maybe a 512kb memory device in the future.
Logged

Eu não sou o teu criado. Se respondo no fórum é para ajudar todos mediante a minha disponibilidade e disposição. Responder por mensagem pessoal iria contra o propósito do fórum e por isso evito-o.
Se realmente pretendes que eu te ajude por mensagem pessoal, então podemos chegar a um acordo e contrato onde me pagas pela ajuda que eu fornecer e poderás então definir os termos de confidencialidade do meu serviço. De forma contrária toda e qualquer ajuda que eu der tem de ser visível a todos os participantes do fórum (será boa ideia, veres o significado da palavra fórum).
Nota também que eu não me responsabilizo por parvoíces escritas neste espaço pelo que se vais seguir algo dito por mim, entende que o farás por tua conta e risco.

Dito isto, mensagens pessoais só se forem pessoais, ou seja, se já interagimos de alguma forma no passado ou se me pretendes convidar para uma churrascada com cerveja (paga por ti, obviamente).

Montreal
Offline Offline
Newbie
*
Karma: 0
Posts: 22
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I put something together in the form of the following "SeekableStream" library:
http://code.google.com/p/libinteract/source/browse/trunk/trunk/arduino/libraries/SeekableStream/SeekableStream.h

It's implemented as a subclass of Stream that adds seek/tell functionalities.

There's an issue with the flush() method that is unclear in Stream, in response I've included a flushInput() + flushOutput() (I believe it should just be included at some point in Stream).

I implemented such a SeekableStream for the ATD45DB161D library, here is the result:
http://code.google.com/p/libinteract/source/browse/trunk/trunk/arduino/libraries/ATD45DB161DStream

For the printf issue, I suggest using the Streaming library.

In the hope that it will be useful to others.
Logged

Pages: [1]   Go Up
Jump to: