The easiest way to control those matrices is probably to use four max7219. These cost around $10 each, but you can buy "clones" on eBay for around $2 each but there is a (small, it would seem) risk they won't work reliably.
I will be interested what solution you come up with for the WiFi part, this interests me too.
Paul