Open Source and Search

Synopsis

My main interest is in open source software (OSS), and I will be speaking on lessons learned from OSS about privacy, truthfulness, independence, personalization, and cost. I will define the particular way in which I mean those words and how they re relevant to search engines, especially Google. I will also connect several ideas along the path(s) from OSS to search, including so-called Web 2.0 phenomena: blogs, wikis, RSS, tagging, BitTorrent, and the long tail of niche markets. Finally, I'll summarize how Google and open source are related and use this to speculate on what Google's future plans may be.

I am not a lawyer, so I will concentrate on the social issues above with regard to web search and how we can achieve them to varying degrees with technology. Technical detail will be kept to a minimum in order to make it clear why everyone should care about these issues, not just computer scientists.

Slides

• LaTeX source code, 870 KB
• PDF slides, 1.0 MB

Readings

I'll assume a little previous knowledge: that you use the web almost every day and have at least heard of the buzzwords above. If you want a little more background information than I can cover in a 30 minute talk, here are some useful readings:

• What Is Open Source?, a good summary by Dan Woods on O'Reilly's OnLamp.com.

• The Long Tail, a Wired article by Chris Anderson on how online businesses can make lots of money from niche markets.

References

I'll be summarizing the salient points of the references below in my talk, which are really too long and mostly too technical for a general audience. I only include the links here for completeness.

• The Cathedral and the Bazaar, the most famous essay on how Linux changed the open source model, by Eric Raymond. Unfortunately lots of jargon and acronyms are used, since programmers are the intended audience.

• Among the audience, a survey article by The Economist magazine that is extremely thorough and accessible. Only the first page is "necessary," although the other articles are also good reading.

• What Business Can Learn From Open Source, the economical arguments for open source software in the business world by Paul Graham.

• Why Open Source? Look at the Numbers!, a quantitative analysis by David Wheeler of open source and its economic/commercial advantages. It contains many useful links to news articles and other reports, helps dispels many common misconceptions, and is fairly recent (April 2006).

• Advice to Microsoft regarding commodity software, the good-bye letter of Dave Stutz, former head of the Rotor (.NET CLR) team, upon quitting his job at Microsoft.

• The Open Source Initiative, a non-profit organization devoted to promoting open source software through standard licenses, definitions, and facts to combat misinformation.
  • Peru's answer to Microsoft FUD. The law requiring all government software to use open standards was passed in Peru in September 2005.

Links

Useful links which contain too many resources or change too frequently for me to list as references above.

• Google Code, the official clearinghouse for Google's open source software and APIs.
  • Organizations in which Google participates, which are all effectively open source.
  • Summer of Code, Google's annual summer "externship" program which pays students stipends to develop open source software with mentoring organizations.

• Google Open Source Projects hosted on SourceForge.

• Search Engine Watch, news site and watchdog organization for all issues related to search engines.

• Delicious, website tagging and bookmark sharing.

• Technorati, a search engine for Blogs.

• Official Google Blog, hosted on Google-owned Blogger.com.