[ Under Construction ]

Welcome
to
Albert's Parallel Computing
Home Page

[ Inktomi Company Logo ]

The topic of this page is the use of applications in a
parallel computing environment. The application I
chose to write about is Inktomi. It was originally a
CS 267 project here at UC Berkeley.

[ Little Inktomi Logo ]

A Question ...
 Have you ever wondered, how could it be possible to collect data from the
entire World Wide Web? Just imagine the logistics of tackling a problem
like that without the help of parallel computing.

What Problems ...
 The problem was to be able to organize and catalog over 50 million HTML
pages on the web with an aggregate size of about 0.5 terabytes. And the
problem wasn't simply size but also the lack of organization with regards
to data. Well the answer lies in Inktomi, the world's most effective search
engine.



The Solution ... Inktomi

"The Inktomi mission is to bring leading-edge parallel computing technology to the Internet using commodity building blocks that provide exceptional cost/performance, nearly unlimited scalability, and new levels of fault tolerance and availability. The HotBot search engine, developed with HotWired, is the first product on this path and represents the first such engine to exploit multi-machine parallel computing.

Inktomi technology can be viewed as the combination of four previously independent technologies into a coherent whole:


The use of Networks of Workstations (NOWs) to build large-scale computers out of commodity
workstations and high-speed local-area networks. This technology enables low-cost systems with
unlimited scalability: we can increase the performance or database size simply by adding more
commodity building blocks (machines, disks, or memory) to the collection.


The development of user-level networking and application software that enables a collection of
machines, disks, and network switches to mask the faults of individual components. The nature of
unlimited scale implies that the number of building blocks is large enough that some amount of
failure is inevitable; it is critical that application and networking software mask these faults from the
user. Inktomi's servers provide 100 percent operation even with multiple down machines, since the
remaining machines provide continuous backup.


The integration of traditional databases into scalable Web servers. Although the primary database
for the HotBot search engine is custom-built for high performance, we use an integrated
multi-machine Informix database for tracking user preference profiles and ad placement and
accounting. Informix provides multi-platform parallel database queries that fit well with the
building-block model used by Inktomi: each server has the full power of SQL transactions and we
replicate information to provide fault tolerance.


The pervasive use of dynamic HTML generation to allow every user to see a customized page. The
use of mass customization, in which we treat millions of users individually within one framework,
requires scalable computing resources and database integration, but also requires new tools and
technology. In particular, we have developed a new form of dynamic HTML that includes a
server-side scripting language that generates HTML on the fly based on the user profile and client
browser information. In addition to the obvious benefit of allowing users to customize their page, this
technology also enables more targeted advertising, and use of advanced HTML features such as
frames and tables for those browsers that can support them; we are not limited to some
"least-common denominator" subset of HTML (for example to support older browsers).


The Web server used by the HotBot search engine incorporates all four of these technologies. Further, these servers have applications far beyond search engines stretching into almost any Internet service that requires scalability, availability, better cost/performance, and the ability to treat users as individuals.

The Problem: Finding Information on the Web
The HotBot search engine exploits this base technology to achieve new levels of capacity, features and scalability in an Internet search engine. The World Wide Web ("the Web") is the graphically-oriented portion of the Internet that is currently experiencing exponential growth both in the number of users and the amount of available data. We estimate that there are over 50 million HTML pages on the Web with an aggregate size of about 0.5 terabytes. The numbers increase dramatically when including images, movies, and audio files.

By design, the Web is completely distributed both physically and in terms of administration. There are no rules on the organization of data or any acknowledged laws of common convention. Although this allows for creative freedom for individual publishers, it inevitably results in a mired environment where it is difficult, sometimes even impossible, to find relevant information.

The most adequate analogy for the Web might be an encyclopedia whose entries are not alphabetized; each entry, however, contains the page number of some related entries. To find something specific, you have few options: 1) Hunt around randomly until you find something that is vaguely related, and then follow its links until you locate the desired entry, 2) Keep your own list of favorite entries to use as starting points and search from there, or 3) Ask friends if they know the page numbers for similar pages. Applied to the Web, the first method outlined is essentially Web surfing; the second adds bookmarks to the user's favorite pages; and the third method of information-gathering is what most people do, except that in the case of the Web, they inquire of services instead of people. Given the above time-consuming scenario, encyclopedias would not have been quite so popular.

Current Solutions
One popular service is Yahoo!, a directory of Web pages. It provides a hierarchical organization of approximately 90,000 web pages that were assembled individually by the editors of Yahoo! In the encyclopedia analogy, Yahoo! would correspond to an incomplete table of contents: one that covers a good subset of information, but less than 1 percent of the total entries.

The other type of service is an index. After HotBot, powered by the Inktomi engine, the next most comprehensive index is Alta Vista from Digital Equipment Corporation, which covers about 30 million pages, or 60 percent of the Web. Alta Vista is a full-text index: it indexes every word in each document. Relating to an encyclopedia, Alta Vista is like a haphazard index: missing numerous entries, based on individual words instead of entire subjects, and occasionally the information provided is simply wrong. Nonetheless, because at 40% it offers the broadest available coverage, it is a popular avenue onto the Web.

Users typically choose Yahoo! when they search for a general topic, such as disk drives or American Indians. Alta Vista and other indices are used for specific searches, such as Seagate Barracuda disk drives, or Plains Indian mythological characters. The focus is either on quality or size: The directories (such as Yahoo!) are assembled by hand and tend to have links to very few high-quality pages; alternatively, indices (like Alta Vista) are assembled automatically (by programs called "robots"), and include a huge number of pages that cover a vast range of topics and quality, in addition to the pages listed in the directories. HotBot solves this dilemma with its powerful query language and SmartRelevance ranking system, uniquely allowing users the highest quality results even when sifting 50 million documents.

The HotBot search engine applies parallel computing technology in building an index that is not limited to a mere fraction of the Web. The only way to handle the exponential growth of the Web is to build a system that can grow with it. Fundamentally, this excludes the use of a either a single big computer (as in InfoSeek's recently announced UltraSeek) or a set of independent machines (as used by Alta Vista or Lycos) in which the additional machines allow more users, but do not help with managing the database size, since the data must be replicated on each machine. Like the single big machines, the HotBot engine has the flexibility of leveraging huge amounts of memory and disk for a single unified database, but it is not limited by the size of the single machine. Inktomi's underlying parallel processing provides a level of fault tolerance can be achieved that is inherently beyond a single large computer. To increase performance or expand a database, another commodity building block can be added; if a building block fails, the responsibility of that block falls to the others, which perform the necessary functions, unbeknownst to the user.

Network of Workstations
The key technology edge for Inktomi's engine is a Network of Workstations (NOW). Traditionally, large computing tasks were performed with mainframes, massive and extremely costly machines that were essentially just faster versions of a normal computer. The microprocessor revolution enabled competitive performance in small, low-cost computers. This new breed of machine, based on "Killer Micros", wiped out most of the mini and mainframe computer business.

The advent of cheap, powerful processors led to a new kind of fast but expensive machine: the supercomputer. Most current supercomputers perform using a large number of parallel microprocessors. Modern supercomputers, then, are not that far removed from a network of workstations or PCs, but there are five key differences:


Supercomputers cost two to five times as much per node (processor) as workstations, PCs, or
multi-processor PCs with a limited number of processors. This is because workstations and PCs are
commodity parts with huge volumes. The sheer volume reduces production costs and amortizes the
significant R&D costs over a larger number of units. The ratio of PCs to supercomputers is about
30,000:1. Unlike Pentium Pro boxes, which are available long before the large-scale machines that
will serve to combine them, commodity building blocks are available first. Given that PC
performance advances at a remarkable 50-80 percent per year, a delay of just six months results in
roughly a 30 percent performance penalty.


Supercomputers tend to have significant memory and I/O capability, which supports an information
system such as HotBot well. Fortunately, it is a simple task to set up workstations with significant
memory and disk space. Even though a single workstation has limits, a collection can handle nearly
unlimited amounts, and enables tremendous aggregate disk and memory bandwidth.


Supercomputers tend to have high-performance networks connecting the processors. Internal
bandwidth varies from 20-200 MB/sec/processor. In contrast, a normal Ethernet for four processors
is only 1 MB/sec/node, and a 100 Base-T switched ethernet might approach 10 MB/sec/processor.
In addition to bandwidth differences, there is an important difference in message overhead (the time
it takes to inject a message). Supercomputers often have injection times that are less than 10
microseconds, while local-area networks typically have injection times around 500 microseconds.
This speed difference is the primary limiting factor to using a NOW as a supercomputer. Inktomi, in
tandem with its custom software, is currently using a new network, called Myrinet, that has
bandwidth around 160 MB/sec/processor and an injection time of about 12 microseconds. We expect
to apply other high-speed networks as they become available.


Supercomputers tend to have unusual operating systems and strikingly few development tools. This
is essentially because of an insufficient market to justify the huge development effort required to
keep pace with commodity software. By using a NOW that uses a normal operating system, we can
exploit all of the latest tools and libraries, resulting in significantly reduced development time.


Supercomputers come as single units in fixed sizes. In contrast, a NOW can be incrementally scaled
by adding single nodes. More importantly, it is possible for a NOW to continue providing service
even if one of the nodes is broken. This flexibility will be the key to providing a fault-tolerant service
that essentially runs without interruption.


To summarize, a NOW is the correct base for a Web service because it is cost-effective, incrementally scalable, and fault-tolerant. We can tune the relative balance of CPU power, memory, and disk performance simply by picking the correct off-the-shelf components. Finally, we can use a leading-edge network to avoid the typical performance problems of a local-area network such as Ethernet.

Fault Tolerance and "Hot-Swap" Servers
A key advantage of using a large collection of commodity building blocks is that we can more easily build systems that tolerate the failure (hardware or software) of individual components. Each block is independent, and its failure has no fundamental bearing on the overall service provided by the site. For example, the HotBot engine can continue operation with full service (but linearly degraded performance) even while multiple machines and/or disks are out of operation. Inherently, this requires replicating data and interchangeable servers to mask failures. We hide disk failures with an on-the-fly switch to a second copy of the data, while fixing or replacing the failed disk. We currently use hundreds of 4GB disks that we can hot-swap, or quickly replace, while the system is running.

For hardware-oriented failures, the system is constructed so that users neither care nor even can determine which machine they are using, and thus do not recognize a (transparent) shift to a new machine. Since the machines are interchangeable, we can switch any user to any machine, avoiding all down machines. The Inktomi software automatically detects machine failures and reroutes users and queries as appropriate. The Inktomi servers can be viewed as a team of identical players that work together to provide comprehensive service to every user. Like a human team, the servers cover for each other when players are injured or absent.

A major advantage is the ability to swap building blocks, software or even parts of the database while the system is active, eliminating downtime for preventative maintenance or upgrades. For example, we can remove a few of the servers from the team and upgrade either their hardware, software, or a portion of the database. We then return them to the collection and upgrade other servers. Eventually, all of the servers will have been upgraded with no service interruption. This level of availability is critical as Internet services enjoy a more global following: The Internet does not sleep.

Audience1 Mass Customization and Database Integration
The era of static brochure-ware Web sites is quickly drawing to a close. In order to make the Web a useful marketing and customer service tool, the site must interact and react based on the user's needs and preferences. By incorporating a running knowledge of past interactions with a customer, the online marketer can optimize the efficiency and power of each contact for both parties. The Audience1 mass customization engine, which has been integrated into the Inktomi server, provides the tools to automate and scale the new generation of relationship marketing.

In today's competitive Web services market, it is simply not enough to provide bare-bones access to a database. A full-featured Internet service must deal with accounting, advertising, variations in client browser capabilities, and above all, must customize the service for individual users to ensure better overall service and build customer loyalty. To meet this new commitment to service, Inktomi has developed Audience1 (publishing to an audience of one), a Web server product which generates every page on-the-fly, tuned to a user's preferences, browser and platform. Audience1 represents the integration of three key technologies: off-the-shelf database management systems (DBMSs), innovative presentation services tuned to the Web and powerful publishing tools for managing these dynamic Web pages.

Database access lets Inktomi servers track customer preferences, user behavior (e.g. the most popular sites users visit from HotBot) and advertising accounts. Integration with commercial DBMSs allows Inktomi to harness off-the-shelf database design, administration and query tools. For example, these tools can be used to produce detailed advertiser reports and run complex data mining and decision support analysis.

The publishing system allows content developers to insert Dynamic Tags into standard HTML-based Web documents. These easy-to-use tags appear similar to normal HTML markup tags, but are evaluated at the server to generate normal, static content such as HTML or JavaScript for the user. Dynamic Tags provide a number of features:


Database access. Audience1 comes with Dynamic tags that can access a DBMS for arbitrary
persistent information and customize the HTML tracking, using either cookies or fat URLs. Unlike
other offerings, while Audience1 supports SQL, it does not require publishers to know SQL to
access the database. This allows Inktomi servers to store and recall a user's preferences for user
interface and query results presentation. More generally, Audience1 is ideal for allowing servers to
access pre-existing databases such as products, inventory, etc.


Browser targeting. Audience1 allows publishers to exploit leading-edge HTML features (such as
Netscape's frames and Java, and Microsoft's font changes and embedded audio tags), without
frustrating users who do not have those features. Audience1's browser targeting can be performed
at various levels of detail, ranging from tags that are easy to use, but don't provide a lot of publishing
control, to exposing the raw browser capabilities to the publisher. For example, advertisers on
HotBot are shown as progressive JPEG if the client browser supports it, otherwise they are shown
as JPEGs or GIFs for less-capable browsers. This allows Inktomi to make the most of each
browser, rather than resorting to a least-common denominator.


Access to high performance, scalable services. Dynamic Tags make it possible for publishers to
introduce new, high performance, scalable services, without requiring the publisher to understand
the intricacies of computing programming. For example, access to the Inktomi search engine is
encapsulated into a single Dynamic Tag, hiding the complexity of interfacing to a parallel program
such as Inktomi. In addition, Dynamic Tags can be multi-threaded, interleaving long-latency
operations such as Inktomi queries and customized content selection (i.e. targeted advertisements).
We know of no other Web-based publishing system with this capability and ease-of-use.
Publishing support hides the complexity of creating and managing sites of dynamic Web pages,
allowing sites with large amounts of content to control the publishing process. Unlike the
CGI-based tools that are emerging, Audience1's publishing support is fault tolerant, high
performance and scales to millions of users and millions of hits per day.


In summary, Audience1 and Dynamic Tags allow a customizable and sophisticated user-interface to Web
services such as search engine. HotBot's interface, including saved searches, personalization, and browser
targeting, would have been nearly impossible without the simplification provided by the Audience1 toolset.

Summary
The technology behind the HotBot search engine enables new levels of cost-effectiveness, scalability, fault tolerance, and mass customization. It represents the first use of parallel computing across multiple machines in an Internet server, and combines sophisticated support for fault-tolerance, database integration, and publishing of dynamic content.

For HotBot, this edge directly enables the largest Web index, but more importantly, it creates a unique system that can grow with the Web: Just add more commodity building blocks. The "team" aspect of the service leads to high availability even in the presence of hardware or software faults. It also lets HotBot staff hot-swap disks, machines, software, and even the database while the service is in active use. Finally, the cost-effective use of computing has led us to incorporate extensive support for dynamic content -- including database support, user customization, and browser independence -- via the Audience1 publishing tools" -from THE INKTOMI TECHNOLOGY BEHIND HOTBOT A White Paper


Here's a picture of Inktomi in Soda Hall.

[ Little Inktomi Logo ]

Another perspective on Inktomi by Thai Lam

[ BSD daemon ]

email me at goto@cs.berkeley.edu
Last Modified: Who knows