BenEdelman writes "Professor Jonathan Zittrain and I are studying
exclusions from Google and have so far found some 113 sites
excluded, in whole or in part, from the French google.fr and German google.de.
Learn more about the situation and context, test
the exclusions for yourself, and submit further
sites suspected to be excluded.
UPDATE 0620 ET 24 OCT 2002
Declan McCullagh of C|Net News has written an article on the report (Google excluding controversial sites).
UPDATE 0807 ET 24 OCT 2002
Seth Finkelstein points out that some of the sites blocked were formerly racist sites, but the domain owner has changed. This is likely why the study is unclear why the sites are currently blocked. No one ever seems to update blacklists (Reason note for "Localized Google search result exclusions" report).
UPDATE 1045 ET 24 OCT 2002
Seth Finkelstein has done some more investigation, and has uncovered more about how Google is blocking sites (How-it-works note for "Localized Google search result exclusions" report).
Slashdot readers speak out (Google Complies with Law, Excludes 'controversial' Sites).
This work continues our study of Internet filtering in countries worldwide. To date, this work has focused on filtering in the context of affirmative actions by governments to restrict the sites viewed by their respective users. However, we are also interested in requests or demands to private parties that they assist in preventing particular citizens' exposure to locally unwanted or illicit data or activities. Search engines are increasingly the targets of such requests, and in our recent work we have attempted to begin to document some of the specific sites filtered -- reluctantly, we gather! -- by the French and German versions of Google.
As in our prior testing of Internet filtering China and Saudi Arabia, there exists no publicly available master list of blocked sites. To assemble something approaching such a list, we have found ways to remotely test "twenty questions" style, asking about thousands of individual web servers.
To help us broaden the number and types of servers tested and to provide the general public a means of finding out whether particular pages of interest are filtered, we have created a web site at which users can see whether a given site is excluded from French or German search results. We'll track all requests and report the results in our forthcoming report about exclusions from Google. We're eager to see whether such "open source research" will end up finding a substantial number of blocked sites that, despite our best efforts, we did not know about and therefore ask about ourselves.
We have also prepared a listing of 100+ sites filtered in France and/or Germany, as well as screen-shots and a system by which interested users can confirm, using their own browsers and their own Internet connections, the reported divergence between results on google.com versus google.fr and google.de.
See initial results, test additional sites, and contribute suggestions via
http://cyber.law.harvard.edu/filtering/google.
Ben Edelman
Berkman Center for Internet & Society
Harvard Law School"