Cynics Unlimited

Part 1 of Cuil Runnings described the self-proclaimed Google-beating search engine and summarized industry’s initial response after launch. Their reception was … frosty, to say the least. Was the response justified? Several tests were performed using common search types to determine whether Cuil is fit to replace Google on discerning desktops.

Test 1: Music Group

For engineers, these ex-Googlers did a poor job of estimating the initial server load but let’s not be petty – it’s only day one. The search results matter most and in particular page 1 of the search results (which is where most people look). Total number of search results, while often used as a selling point, mean very little in practical terms since most surfers only look at the top 10 or 20.

My first test was done using a moderately popular music group, the British RnB / Drum n Bass outfit 4Hero. Cuil really showed its best side for this example: firstly, the results displayed in a three-column magazine style (two-column displays are selectable). To the right are a series of categories related to the group 4Hero, including albums from the group and related artists. If you were to click on the link titled “Goldie” (another Drum n Bass artist) Cuil would search for “4Hero Goldie”. This accessible drill down display will be indispensable for people doing actual research.

Test 2: Ancient Civilization

Inspired by the implications from the first test, I keyed in an ancient civilization –Nubia- to determine how easily research could be accomplished. Again, the category drill down was available on the right but this time a result filter was available above the search results. Since Nubia is a pretty broad topic, Cuil gave the option to filter the existing results on major subtopics like Ancient Nubia and Rhadopis of Nubia. Cuil’s interface is closer to an interactive encyclopedia than a straight search engine.

Test 3: Direct Website Reference

Cuil really fails when it comes to identifying name of actual websites. This is important because a worrying number of people still find websites by entering the English name in a search engine and clicking on the first link they see. I tried that with Jack’s NewsWatch and no link to the site root exists on the front page. In fact, the first link on the list is to Jack’s new-found nemesis at StageLeft. The average site-owner probably doesn’t want a large % of potential search traffic going to his detractors. A significant % of the search results were also coming from third-party services like HaloScan and blog aggregators. Conversely, Google’s first link went directly to the front of Jack’s site, with lesser quantities of the parasitic sites in later links. The same test was performed with other blog sites (Cynics Unlimited, Crux of the Matter, Blue Like You) and only Google linked to the root of the actual site within the first page. The first search result for Small Dead Animals links to a site attacking the blog owner. The first search for Blink 7 links to Blink 182’s band site.

Cuil was much better at identifying major websites like CNN, but that’s hardly an indication of a search engine’s ability to determine link relevance.

Test 4: Recent News Articles

Testing Cuil’s ability to retrieve the latest news stories involved writing two tests per article. First, the full title of the article entered verbatim into the browser to determine whether a link to the actual article or reprint of the article appeared on the first page. The second test was completed using key words. The tests are listed below (key words in brackets)

• Associated Press: Bush OKs execution of Army death row prisoner (keywords: Bush Ronald Gray Execution)
• Reuters: Zimbabwe crisis negotiations deadlocked (keywords: Zimbabwe negotiations)
• Canadian Press: Bell Canada to cut 2,500 jobs to lower operating costs ahead of takeover (keywords: Bell Canada job cuts)

Cuil returned no results for the title of the AP or CP articles. The Reuters subject line returned an excerpt from a Zimbabwe site unrelated to the article in question. Amazingly, Cuil had no results at all for they keywords related to the AP article, meaning not even info on traditional websites. The CP keywords returned several Wikipedia pages about Bell and one WSWS article about Bell cutting jobs … in 1999. Conversely, Google found all of the articles by title and keywords within the first page. Cuil may be indexing more pages than Google but surely aren’t doing so with great speed.

Conclusion

Cuil’s interface is beautiful and intuitive. General-purpose researchers and students will quickly take to its OLAP-style interface and numerous search refinement options. The search engine itself needs help, however. Cuil was not intuitive enough to recognize all by the most ubiquitous site names while third party sites and junk aggregators pushed actual site content out of the top listings. Cuil performed abysmally at retrieving current events or recently-updated sites, which is unacceptable in a 24-hour news environment. As of now, Google has little to fear.