Well in a pinch, Radio Shack might do. If you are not familiar with them, they are a chain of stores that sell electronics parts and gizmos, such as the TRS-80 back in the 1970's. After an initial bankruptcy, they were bought by the Tandy Leather Company, which was always a strange pairing. The leather works have now been sold, and the Tandy company just operates the Radio Shack stores. Here is the corporate history: http://www.radioshackcatalogs.com/history.html.
While in general RS tends to sell pre-made consumer electronic products and cell phones, they generally do have an area in the back where they stock components and such. Recently, RS has seemed to rediscover the hobbiest electronics market, and perhaps has more variety now. Depending on the store, you still get a lot of clerks with the deer in the headlights look if you ask for a random part. Prices do tend to be on the high side, but if you need something immediately, they are plentiful.
Obviously, I would expect more specialized shops in the NYC area, but perhaps they aren't in Manhattan, but in Brooklyn or the other boroughs.