Much has been written about Google’s attempts to tweak their vaunted search algorithm recently, but the results might take awhile to become apparent. Ostensibly, Google changed the page rank code that is buried within their super-secret search architecture in order to make results more meritorious. More merit in a user’s search would be measured by the usefulness in the top-most (un-sponsored) links and supposedly less prone to the technological gerrymandering that we have collectively come to know as S.E.O.. While this is potentially good news for end-users, it is very bad news for firms that have invested time, money and intellectual capitol into manipulating search results. In the years since S.E.O. became the Holy Grail of online marketing, one class of content developer has emerged as the bête noire of search: the content farm.
Content farms are essentially clearing houses for cheaply created or “linked-out” content that are tagged and curated around popular search terms. The growth of metadata tagging (which will only grow more sophisticated in HTML 5) within web pages and discrete assets (like video) embedded in pages has made the gaming of search results very successful for some marketing strategists and virtually transparent to the end-user. The implications for small changes in search algorithms have potentially deep consequences, but the stakes are rarely discussed in technology and MBA programs that have invested in strategies aimed at the status quo. These issues go to the heart of much of what we think about how the Web works and what value web searches actually have.
Within a decade, the ubiquity of content online has significantly eroded the publishing industry’s longstanding print-based revenue model. Newspaper, print book, and magazine sales have all experienced steady declines as people turn to the internet for its surfeit of free and instant information. Internet technology expert Clay Shirky frames the dilemma like this: “The core problem publishing solves —the incredible difficulty, complexity, and expense of making something available to the public— has stopped being a problem.”
The question of how to profit from digital content has, so far, produced two general strategies: a digital restriction model requiring direct consumer payments, such as Apple’s iBooks app and The New York Times paywall, and the pay-per-click advertising model employed with great success by Google, whereby free content is accompanied by paid advertisements. In the former model, content for sale reflects traditional, curated models of publishing quality: known authors, established journalists. In the latter model, content is written by anyone (or in the case of Wikipedia and other aggregated sources of information, no one in particular). The authorship and quality of this free information is not of primary relevance as far as ad revenue is concerned. Content is simply the bait for display ads, and needs only to rank high in Google search.
Enter the content farm. Content farms are web sites that solicit search-optimized content from low-paid freelance writers and then display or license the content, accompanied by ads. Although content farms are controversial, they’ve done big business thus far: Yahoo bought the prolific content farm Associated Content for approximately $100 million in 2010, and Demand Media, the most successful content farm to date, reported revenue of $79.5 million in the first quarter of 2011, a nearly 50% increase over its revenue in 2010, and is currently valued at 1.5 billion dollars (a valuation placing the worth of Demand Media above that of the New York Times).
Demand Media’s content creation strategy, unlike that of traditional media companies, starts with data: an algorithm that analyzes the value of search terms, advertising keywords, and existing related content to determine content topics likely to drive search demand — in other words, what people want to know more about. These topics are then turned into videos and articles by a vast network of low-paid freelancers, and unleashed on the internet at the volume of 4,000+ pieces a day. (This infographic explains how it works.) Daniel Roth, in his 2010 profile on Demand Media in Wired Magazine, highlights why this strategy is working for Demand: “…what Demand has realized is that the Internet gets only half of the simplest economic formula right: It has the supply part down but ignores demand. Give a million monkeys a million WordPress accounts and you still might never get a seven-point tutorial on how to keep wasps away from a swimming pool. Yet that’s what people want to know.”
If you’ve used Google, the success of Demand Media’s data-driven approach appears routinely in your top search results: ehow, a Demand Media property, frequently lands above the fold in searches beginning with “how to.” (Try it now: How to stop wrinkles. How to jump-start a car. How to carve soap.) The 4000+ daily articles and videos published by Demand Media’s assembly-line system, however, are frequently accused of adding to internet information pollution due to poor writing and production quality, keyword-stuffing, and superficial subject matter. The low quality charge has potentially become the Achilles’ heel of the content farm model, and has drawn widespread criticism of Demand Media in particular.
The questionable quality of content produced by Demand Media and other content farms is, however, a logical consequence of the algorithmic authority that Google has enjoyed for most of this decade, described by Clay Shirky as “the decision to regard as authoritative an unmanaged process of extracting value from diverse, untrustworthy sources, without any human standing beside the result saying ‘Trust this because you trust me.'” Demand Media’s own algorithm is designed to produce content topics that will succeed according to the algorithmic authority of Google, and it works, without the need for any human readers to weigh in on the content’s actual value to people. This emphasis of machine over human wisdom was the subject of virtual reality pioneer Jaron Lanier’s book, You are Not a Gadget. “It is increasingly disheartening to write about any topic in depth these days,” says Lanier, “because people will only read what the first link from a search engine directs them to” — frequently an article from a content farm.
Google is well aware of the charge that Demand Media and other content farms have gamed its algorithm, and the company has taken several steps to respond. In early 2011, Google changed its algorithm (referred to as the Panda update) to, among other goals, de-prioritize “sites which are low-value add for users, copy content from other websites or sites that are just not very useful,” a move that negatively affected the search rankings of content farm properties like Suite 101, Mahalo, and Demand Media’s eHow. Perhaps resultingly, Demand Media’s stock dropped in value by 13% shortly thereafter.
Search engines are also moving toward incorporating human curation factors into search ranking results, a possible sign that the era of total algorithmic authority is on the wane. Google and Bing have recently incorporated social search features into their ranking results, meaning that search results are impacted by content that is liked, Tweeted, or shared on social networking sites. Search engine startup Blekko bans content farm results outright, acknowledging what Google does not: that human judgment plays a major role in assessing the quality of information online.
Possibly in response to these developments, Demand Media announced the launch of its Talent and Experts Network in June 2011, an effort to sign up contributors with name recognition or acknowledged expertise in particular subject areas. Published travel writer Rolf Potts is an early recruit. This is an interesting move toward the established expertise model relied upon by content providers like Apple and the New York Times, and raises the question of whether web content will eventually trend toward the traditional publishing model’s focus on highly curated name brand authors. Jimmy Wales, founder of Wikipedia, predicts that Demand Media, while currently occupying a successful niche, may have a limited future as “better quality models emerge.”