Let's put it this way ... say a court ordered that the police were to search every house in the land for evidence of some crime. Surely there would be huge outcry? Innocent until proven guilty! people would cry. Or, no search without probable cause. And, who knows? There might be resistance at the door. But to hand over a tape with everyone's search history on it?
And as for "being robbed" - I presume you mean, if they happen to have all this data, and it happens to be stolen, that's not their fault? I would suggest that it could be. I just read today about thousands of credit card numbers being stolen (yet again). I think we can presume that once large centralized caches of data exist, they will eventually be stolen. Given that, the mere keeping of such caches could be regarded as the real "crime" - since we presume it will eventually fall into the wrong hands.
I don't have a problem with Google keeping track of what pages link to other pages (via their crawling spider bots). After all, that's what basically makes it work. And to an extent, using your IP address to narrow down searches is probably OK too, since if I search for a plumber I probably want one here, and not in London.
But once they start to remember what I searched for, and trying to build up a commercial profile of me, with a view to selling that to advertisers, then I think it steps over the line. Put it like this: today they sell to advertisers, tomorrow: political parties.