Serious project PC to USB mass storage interface - not limited to Arduino

I need a micro controller with a USB slave and Host interface (i realise that is not on the Arduino) to act as a go between a PC USB port and a USB mass storage hard drive in both directions.

Is there a better choice of micro controller?

What interface chips can i use so as to work with the direct commands being issued between the PC and USB mass storage device.

Source code for mass storage devices , I believe the device is treated as just a mass of sectors and that all the format and structure is controlled within the OS and dependant on the file structure used (NTFS , FAT, FAT32, etc.) currently I don’t mind picking on one but will need technical knowledge on the others.

If there is an expert out there then interested in hearing from you as this is a project that will be supported with a grant so not looking for this all free of charge.

Will need to sign an NDA for much more info as this project is already going for a patent

Vivamort: I need a micro controller with a USB slave and Host interface (i realise that is not on the Arduino) to act as a go between a PC USB port and a USB mass storage hard drive in both directions.

Most Arduino boards don't have an innate ability to act as USB hosts. However, there are USB Host shields (like this one or that one) to enable them to act as USB hosts. Also I think the 2011 Arduino ADK board has the Host/Slave interface as a built-in feature.

Far-seeker: Also I think the 2011 Arduino ADK board has the Host/Slave interface as a built-in feature.

Yes thank you Far-seeker I was looking at the MAX3421E as it may be at least an experimental platform for experimentation.

If I could get the Arduino to look like a mass storage device to a pc by programming one of these as a slave device and understand the coding behind that then I would be a long way forward as I would also understand a lot more of the communication commands that I need to work with and support.

Excellent starting point much appreciated

I need a micro controller with a USB slave and Host interface (i realise that is not on the Arduino) to act as a go between a PC USB port and a USB mass storage hard drive in both directions.

Although you can build such a device with a Leonardo (has USB slave capabilities) with a USB Host Shield (providing the host part), you probably won't get happy with it because the Arduino will be the bottleneck. It's running with 16MHz, todays USB storage devices operate at 480Mbit/s transfer rates, completely out of reach for an Arduino to only transfer it, don't talk about doing something intelligent with it.

Will need to sign an NDA for much more info as this project is already going for a patent

Don't expect help on this forum if every participant have to sign non-disclosures. If you're already going for a patent, why do you need such basic help?

I need a micro controller with a USB slave and Host interface (i realise that is not on the Arduino) to act as a go between a PC USB port and a USB mass storage hard drive in both directions.

Although you can build such a device with a Leonardo (has USB slave capabilities) with a USB Host Shield (providing the host part), you probably won't get happy with it because the Arduino will be the bottleneck. It's running with 16MHz, todays USB storage devices operate at 480Mbit/s transfer rates, completely out of reach for an Arduino to only transfer it, don't talk about doing something intelligent with it.

A proof of principle even running slow will be enough to show the system works and then it can be built into silicon which is where it would finally be at home. I also said this is not limited to the Arduino but at the moment I am more interested in really understanding the mechanics and protocols behind mass storage over usb and to be able to interface at this level. per command level.

Don't expect help on this forum if every participant have to sign non-disclosures. If you're already going for a patent, why do you need such basic help?

I was not saying everyone has to be willing to sign a non-disclosure, and I am more than grateful for any advice especially the free type but I also wish to protect my investment and creativity of concept so do not wish to give the full project total disclosure to Joe Public!

I am however willing to enter a business relationship with the right expertise to get this "Simple and Basic" project as you think it is to hopefully one of them simple (why did I not think of that) commercial products that end up on the World market (but only time will tell if this ever happens and the right investment :-) I understand programming and micro controllers all to well but my expertise is not with the mass storage class and USB protocols and need a platform that may be to start from even if its not the finial product, but now that I had some useful directions to explore I will endeavour to learn it myself if required, however with an Expert in this field time to market would be "accelerated" and that can make the difference from a good idea to a million bucks I guess (here is hoping)

Vivamort: Will need to sign an NDA for much more info as this project is already going for a patent

I don't quite get why you are already getting a patent if you don't even have a design with components down. Much of the time I'll get a great idea, think its great, start working on it and then I'll realize why it hasn't been done, b/c the needed tech isn't there. If you weren't sure of what goes into make a PC USB interface on a design level, I don't understand why you would go ahead and apply for a patent.

funkyguy4000: I don't quite get why you are already getting a patent if you don't even have a design with components down. Much of the time I'll get a great idea, think its great, start working on it and then I'll realize why it hasn't been done, b/c the needed tech isn't there. If you weren't sure of what goes into make a PC USB interface on a design level, I don't understand why you would go ahead and apply for a patent.

For the finial product I have to have a chip made, this would cost tens of thousands, however as for knowing if the technology is there that is a given. what I need to do is to concept it and show the right people as a demonstration is like a picture it speaks for itself. When I have a working model even if not running at full speed its potential is then obvious rather than just conceptual, my patent is so my timing is protected, my idea is not in question , but to turn my idea into a reality needs to show people it works then the money issue is not a problem.

All I will say is ask yourself this if a royalty is paid of even pennies or cents or whatever your currency is on even only 1 % of the the worlds quantities of mass storage devices what does that come too, then you may understand why i feel its important to protect the project.

Its not the money that is the issue. Its the fact that you can actually do it. That is the question, and we are asking you to be realistic. Can you actually do it? Because you are here asking about USB host/slave interface and then you say you have to get a chip made custom. I really hope you know what goes into making a microchip. It isn't simple, and its nothing like ordering a custom circuit board. To be asking such trivial questions such as the USB thing would scare any investor.

So I'm not answering your original question; I just want to make sure that you aren't getting too hyped up about something that may be over your head. If you can't do it all by yourself, its best to get a team of people who can help you out. A team of people you trust. Its easy to keep everything to yourself and try and learn how to get all the knowledge so you can do it on your own. It is difficult to let somebody else in on your idea and let them do their thing with your baby. I just want to make sure you aren't going about this the wrong way.

I think personally I am being very realistic, I have a project which I know can be done, the technology is not the issue the implementation is. I wish to prototype it to demonstrate it rather than just say I have an Idea or concept as darn right nobody is going to invest in turning it into silicon at this point. I am willing to take it forward as much as I can, but I am also saying if someone is serious and has the skill sets to help out that I would welcome them on board the team with an NDA and yes depending on there input they get to share the rewards. I placed it here as there is a lot of very skilful people in all walks of life that love these sites and therefore I may get the right person interested. So your exactly right I am not just trying to solve this alone I am looking for a team / help, but guess what if it does not turn up I will do what I can and drive forward, my personal outlook is you only fail when you stop trying, until then your just learning how not to do it.

To give you starting points:

USB host driver for mass storage: https://github.com/felis/USB_Host_Shield_2.0/tree/master/

USB mass storage slave device (you need an Arduino Leonardo or a Teensy2 for this): http://elasticsheep.com/2010/04/teensy2-usb-mass-storage-with-an-sd-card/

You might be able to handle block access with this. Getting access to the filesystem layer of a modern filesystem (ext4, btrfs, zfs, HFS+, NTFS, etc.) is not possible with an Arduino because of the limited RAM capacity. But stuff like in-between compression/encryption is better handled at block level and not on the filesystem level.

The Arduino Mega ADK isn't suitable because the USB slave device is handled by the ATmega16U2 co-controller on the board which is only connected by a serial interface, so you haven't direct control over both USB interfaces.

pylon: To give you starting points:

USB host driver for mass storage: https://github.com/felis/USB_Host_Shield_2.0/tree/master/

USB mass storage slave device (you need an Arduino Leonardo or a Teensy2 for this): http://elasticsheep.com/2010/04/teensy2-usb-mass-storage-with-an-sd-card/

You might be able to handle block access with this. Getting access to the filesystem layer of a modern filesystem (ext4, btrfs, zfs, HFS+, NTFS, etc.) is not possible with an Arduino because of the limited RAM capacity. But stuff like in-between compression/encryption is better handled at block level and not on the filesystem level.

The Arduino Mega ADK isn't suitable because the USB slave device is handled by the ATmega16U2 co-controller on the board which is only connected by a serial interface, so you haven't direct control over both USB interfaces.

Thank you Pylon,

Currently doing some working through this beastly document USB Mass storage class

External hard drives alreadyu plug into USB ports http://www.bestbuy.com/site/Western+Digital+-+Caviar+Green+2TB+Internal+Serial+ATA+Hard+Drive/9234465.p?id=1218064150518&skuId=9234465&st=western digital&cp=1&lp=5 What will this interface in the middle do besides add a bottleneck to the process?

I'm currently working on an interface to mass storage, extending the project from Circuits At Home and using the USB Host Shield 2.

I can read a pen drive but I'm having problems to write. I keep getting NAK from the device. Someone out there with experience implementing the Bulk-Only protocol?