Here's a few general notes that I've been meaning to make for a while.
One thing to remember is that uFat simply overwrites data that is pre-allocated on the card. This needs to be a contiguous file else wierdness will ensue. One way to ensure this is to format the card before writing your proxy file to it. Deleting files from the card can leave holes in the FAT which your operating system will use next time a file is created on the card leading to a fragmented file.
Unfortunately resizing a file isn't a simple case of adjusting a value in the directory structure. It entails re-linking the FAT table chains, which means a lot more code in uFat, which is counter to its design criteria. I think it's a reasonable trade-off. Actually, thinking about it, if people were happy with yet another trade off then we might be able to come up with a small extension that can work with one or two files on a card and set their lengths in a one-off manner... But I digress.
The length of the file indicated in the directory is the raw size of the original proxy file, not the number of bytes you've written. Its size remains constant no matter how you write to the space it occupies.
If you write binary data to the file then you'll no doubt have a client application that understands it.
If you write ascii data then things are a little (but only a tiny bit) harder. When the file is written to and later removed to the computer and viewed in a text editor, there may well be a warning that the file is corrupt (at best) to a refusal to load it at all (at worst). This is to do with non-ascii ranged bytes in the file, as explained in a previous post. Warnings can be ignored on the whole. It's best practice to empty the buffer before writing data to it. You could fill it with character 0x20 (ascii for 'space') but that would mean a bloated file when editing. You could fill with 0x0d (carriage returns), which might be better. I'd go with 0 myself, unless it was making editing hard.
Nachtwind, and those who want it, my PHP for converting to CSV:
<?php
$packet=array(3,2,2); //the number of bytes to read for each value of the 'packet' of data to be converted to a csv line.
$f=fopen('data.bin','r');
while(!feof($f)){ //repeat until end of file
$csv_line='';
foreach($packet AS $bytes){ // loop through the packet and generate the CSV line
if($csv_line) $csv_line.=","; //append a comma if needed
$csv_line.=getVal($f,$bytes); //append the next value
}
echo $csv_line."\n"; //echo the line
}
//reads $bytes bytes from file $f and convert them to a value
//does this by reading each bite as a character, and converting it to its ASCII value
function getVal($f,$bytes){
$val=0;
for($i=0;$i<$bytes;$i++){ // loop for however many bytes we require
$b=ord(fgetc($f)); // read a character and convert it to it's ASCII byte value with ord()
$val=($val<<8)+$b; // shift the current value left eight bits (8 bits == 1 byte) and then add the byte we just read to the value
}
return $val;
}
?>
You must adjust $packet to reflect the number of bytes used for each value. (This is currently configured to read the data generated from my example in SDWriter)
The magic of joining the bytes into larger values happens on this line: $val=($val<<8)+$b
this shifting of bits is how we join two bytes back into an int
this could also be achieved by multiplying by 256: $val=($val*256)+$b;
val
byte2
byte1
example values and how they would be stored in multiople bytes
255
255
as we probably know, 255 is the most one byte to contain
256
1
0
to store 256 or more we need another byte to store multiples of 256. (1*256) + 0 =256
261
1
5
(1*256) + 5 =261
65535
255
255
(255*256) + 255 = 65535 //max value of an int (2 bytes)
you could now add a third byte to store multiples of 65536
this can be done for as many bytes as you need to contain your range of numbers, with only a few more bytes you can store over billions!
sirmorris, I find the functionality of uFat is excellent and the trade-offs are not a problem, especially as it seems we are all really only wanting to log data, I can't really see the need for more functionality. If I wanted to do much more I think I would probably want a full FAT filesystem. But only the future can tell
Also your method of printing to the buffer looks really cool, I haven't got time to study it greatly now, but it looks like it's exactly what JB needs and wouldn't require my suggested writeString function. In fact I guess I could extend my SDWriter in the same way to add the print/println functionality to it
If I may I'd like to offer a couple of comments on the writer.
Having the write function which is used by the print() calls go through a general-purpose write function is less efficient than making things work the other way viz:
I've only had time this morning to quickly compile & try bobemoe's Print_Data example, but I can see that it's EXACTLY what I was after. I should be able to incorporate this code to 'format' the data stream in the most user-friendly way possible, which will make life so much easier for my workmates (and me!).
I'll be away from my computer today so I won't be putting much time into this, but in the next couple of days I'll let you know exactly how things have gone.
Charlie & bobemoe - you're work here will be extremely valuable to many others like myself I'm sure; we can't thank you guys enough.
Oh, one last thing - Charlie, did you have any thoughts on why my RTC data is being 'corrupted'?
I've looked at the code and there are a couple of things that I'd like to go over with you - as they're application specific maybe you should PM me and we'll take it offline via email. We can post final findings but the interim comms might be a little noisy for this topic.
Kind-of, and no. The library is intended for tight memory situations where you are willing to accept limitations in usage. uFat allows you to find the starting sector and length of a file on a fat12 or fat16 formatted device.
You can then read data from and write data to those sectors. It doesn't understand the filing system more than it needs to, which in this case is only reading the boot sector, partition table and directory.
A few posts ago bobemoe posted code to use the Arduino library print routines to output strings to an mmc card. I've spoken with him and used his code as a basis for DevicePrint. It has been modified to fit my usual criteria - small & tight with few (if any) dependencies.
Get it herehttp://arduinonut.blogspot.com/2009/02/libraries.html along with some of my usual written drivel about whatever and stuff.
Commentary and questioning welcomed as usual. Enjoy.
After numerous emails, pulling of hair & re-writes, sirmorris has helped to get my SD card logging dreams up & running. You can get the 'final' sketch here: http://docs.google.com/Doc?id=dqhc6fg_0gmk96kdd. It's certainly not the prettiest sketch, and I can't claim much of it at all as my own work, but it does what I hoped it would (although real-world testing hasn't happened yet).
Hi Sirmorris.. unfortunately i dont have much luck compiling your code - i am using 011 of the IDE with a 328 Chip..
Errors are:
In file included from C:\Documents and Settings\Administrator\Desktop\arduino-m328-win_current\arduino-0011-m328\hardware\cores\arduino/WProgram.h:6,
from DevicePrint.cpp:1:
c:/documents and settings/administrator/desktop/arduino-m328-win_current/arduino-0011-m328/hardware/tools/avr/lib/gcc/../../avr/include/avr/signal.h:36:2: warning: #warning "This header file is obsolete. Use <avr/interrupt.h>."
In file included from DevicePrint.cpp:3:
/DevicePrint.h:20: error: expected class-name before '{' token
23: error: microfat2.h: No such file or directory In function 'uint8_t proxyWriter(const uint8_t*, long unsigned int, uint8_t)':
In function 'void setup()':
It also looks from this error that the mmc, microfat and deviceprint folders haven't been copied to the folder
arduino-00XY\hardware\libraries
which is where they should live.
I hope the upgrade won't cause you too many problems, but there are issues with some sketches - the libraries changed significantly during the 11-12 transition. You can keep both versions side-by-side though, so if you have a problem sketch just compile it under 0011. For all new work, start out in the 0013 IDE.
If you do need to port code then the rule seems to be - if it's an .h file then add
well... i thought it had to do with 0011.. alright.. so i upgraded.. lets see if it works now ;0)
So far it compiles and runs - did i miss something or do you use the same wiring as on the first post shown? Either that has changed or my Card reader is not soldered properly ><
Hey guys, I've been following this very helpful thread but haven't made much progress. I've figured out it's not detecting the card is in since I get STA_NOINIT and STA_NODISK from the MMC init function. I'm using the 012 arduino software.
I've measured the pins with a multimeter and I suspect one of the ground pins isn't connected quite right. So I was wondering how you guys have it hooked up? I bought just the SD card holder and stuck a breakaway thing with the side with the long pins going in to the holder and wires soldered to the short pin side as shown below. I'm using a seeduino which has a 3v3 switch and it's hooked according to the diagram in the OP.
edit>oh right and the question: how do you guys have your card holder hooked up? Is there a better way of doing it? I could have gotten the card holder with a pcb soldered on from sparkfun but it was $15 but I want to use it in a final pcb design too. If nothing works I guess I'll have to suck it up and buy that sd holder board.