Go Down

Topic: A way to sort on SD. (Read 1 time) previous topic - next topic

GoForSmoke

If time is allowed, I see that a sort of a sort or other 'file manipulation' could be performed by building a link file of sorted addresses to the data in the original file.

Let's say I have an unsorted file of names and addresses, 1 CSV line each. My algorithm would find each record in turn and check the sort key against -every other record- to find out how many other records are lower in the final sort. For each record in turn a link is made in the link file; line number, address. The link file needs another, smaller link file to sort it.

And once the links are made, sorted access can be given to the original data.

I wonder how many of the Unix shell commands could be supported?




I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

Osgeld

seems heavy for something that can be done after stuff it logged at your leisure... what I am getting at is your using SD for some type of log... meaning you or something is eventually going to read this log, just have your multicore, multiGHZ, multiGB computer do a half second sort on the data when you need to look at it

unless you just want to do it all on a 8bit, no ram chip cause you have nothing better to do... then have fun!
http://arduino.cc/forum/index.php?action=unread;boards=2,3,4,5,67,6,7,8,9,10,11,66,12,13,15,14,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,86,87,89,1;ALL

GoForSmoke

For some things I may prefer my algorithms locked in the portable device.

I prefer the device be able to process the data as a capability but that does not preclude getting a card with data and links already there. The links serve as quick offsets to data, to make Q&A faster.
But how the data is related is contained on the MCU, what comes out is from the MCU. Maybe I don't want all the hows known (company secrets used) or just don't want to write that app for the PC/MAC and view the little stand-alone device in a task filler role that says "don't have to". 

I want it to be able to archive files and copy cards as well, anywhere. Information collection and dissemination made easy, so why not process on the time it takes to go from here to there? Links provide faster access than a sorted file and can be used by other links.

With data and links the MCU then can act as a server customized for some app. Hook it to a dumb terminal and run order entry or whatever.

We used to run business on far less computing power, now we can run at far less start-up cost.  The big power is needed for some things, like engineering and graphics, but what do you really need for AR, AP, and GL? Another few $600-$1000 PC's or a few far cheaper smart terminals?

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

fat16lib

You just need an external merge sort.  Invented by John von Neumann in 1945.

This was a key algorithm for tape based data processing and ran on computers with no disks like the IBM 1401.  Many 1401s had about 4 K (6-bit characters plus parity) of core.  I actually used one.

It can be implemented for an SD on Arduino and is optimal since it is O(n log n).

GoForSmoke

It all goes back to the card sorters... well, about 90% of business seems to.

External sort on an SD card? How many over-writes/scratch files?

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

fat16lib

There are hundreds of papers that have optimal algorithms for polyphase merge sorts.  Just google for disk examples.

You can use as few as three files.

GoForSmoke

3 files and how many record swaps?

I find it harder to express logic in English than in Code.
Sometimes an example says more than many times as many words.

fat16lib

Your question shows you don't understand the polyphase merge sort.  Find a good article about it on the web.



Go Up