The Minimus AVR boards are also USB device only, so they're no alternative for the USB Host Shield.
Since I'm the guy who designed Teensy and wrote Teensyduino, my opinion is obviously biased, so take this with a grain of salt....
Minimus AVR will probably only be useful if you're already pretty knowledgeable with AVR development in C and you can use Dean Camera's LUFA library. Or in other words, you're the type who tends to feel like the Arduino IDE is a toy and you want a "real" programming environment like Eclipse or direct control of the Makefiles.
As a quick sanity check (which doesn't cost anything... do this before you buy), I'd suggest you download the LED and button test program for Minimus AVR, and try to compile it yourself. It's at the bottom of this page:
Just move the .hex file somewhere safe, and delete the .elf, .o and .d files. Then try to compile it and see if you can get a new .hex file that is the same. Don't worry if the new .elf and .o files are not perfectly identical, it's the .hex that matters.
You won't find any help in the Minimus instruction manual, as it only documents how to get the .hex file onto the board. In fact, if you look at the .c file, you'll notice in the comments that it was written for the Olimex AVR-USB-162 board, not Minimus AVR. That should give you some indication of the level of documentation, example code and support you'll find for actually using Minimus for any real development.
If you're able to easily figure out how to compile the example and get the same .hex file, then maybe Minimus might work for you? Well, you might also download and peek at Dean Camera's LUFA library before buying, since that is the only code which supports Minimus. But if you get stuck and/or frustrated just compiling the blink hex file, at least you can do so without paying for a Minimus board. Even at the fire sale price, if you can't compile a .hex file, the board will be worthless (other than running .hex files made by other people... which is all Minimus was ever really made for anyway). But if you have the skill to make Minimus work, it's very cheap.
There doesn't seem to be any support for using Minimus with the Arduino IDE. Dean Camera's LUFA library is your only reasonable choice on Minimus. I'm not sure how you'd get the USB Host Shield to work... the library is built on the Arduino platform and makes good use of C++ features. Porting it to a C-based LUFA project sans Arduino stuff seems like a huge task, but at least version 2 has a very nice self-contained abstraction layer built in. If I were to try, I'd definitely focus on version 2.
I'm going to refrain from making a sales pitch for Teensy 2.0 or Teensy++ 2.0. In fact, if you're planning to use the USB Host Shield, the Arduino Uno or Arduino Mini might be a more attractive alternative, since those shields are designed to directly line up. There aren't a lot of wires to connect, as you can see in the photos of how I connected Teensy 2.0 and Teensy++ 2.0, but if you're not handy with a soldering iron and small wire, shields that just plug in or only need headers soldered are a pretty nice way to go. One bad solder joint can make all the difference between a fun project and a terribly frustrating disappointment.
Likewise, what good will that Minimus plastic box do for your project if you're going to add the USB Host Shield, or just about any other electronics?