Well I don't know for sure what to try next. I got it working so in principle it should work. In your shoes I would be starting to swap things around. Do you have a Uno or similar you can hook the wires up to, to see if it works there? Or another programmer like a USBtinyISP? Or another chip?