|The topic of this page is the use of applications in a
parallel computing environment. The application I
chose to write about is Inktomi. It was originally a
CS 267 project here at UC Berkeley.
|A Question ...
Have you ever wondered, how could it be possible to collect data from the
entire World Wide Web? Just imagine the logistics of tackling a problem
like that without the help of parallel computing.
What Problems ...
The problem was to be able to organize and catalog over 50 million HTML
pages on the web with an aggregate size of about 0.5 terabytes. And the
problem wasn't simply size but also the lack of organization with regards
to data. Well the answer lies in Inktomi, the world's most effective search
|The use of
Networks of Workstations (NOWs)
to build large-scale computers out of commodity
workstations and high-speed local-area networks. This technology enables low-cost systems with
unlimited scalability: we can increase the performance or database size simply by adding more
commodity building blocks (machines, disks, or memory) to the collection.
The development of user-level networking and application software that enables a collection of
machines, disks, and network switches to mask the faults of individual components. The nature of
unlimited scale implies that the number of building blocks is large enough that some amount of
failure is inevitable; it is critical that application and networking software mask these faults from the
user. Inktomi's servers provide 100 percent operation even with multiple down machines, since the
remaining machines provide continuous backup.
The integration of traditional databases into scalable Web servers. Although the primary database
for the HotBot search engine is custom-built for high performance, we use an integrated
multi-machine Informix database for tracking user preference profiles and ad placement and
accounting. Informix provides multi-platform parallel database queries that fit well with the
building-block model used by Inktomi: each server has the full power of SQL transactions and we
replicate information to provide fault tolerance.
The pervasive use of dynamic HTML generation to allow every user to see a customized page. The
use of mass customization, in which we treat millions of users individually within one framework,
requires scalable computing resources and database integration, but also requires new tools and
technology. In particular, we have developed a new form of dynamic HTML that includes a
server-side scripting language that generates HTML on the fly based on the user profile and client
browser information. In addition to the obvious benefit of allowing users to customize their page, this
technology also enables more targeted advertising, and use of advanced HTML features such as
frames and tables for those browsers that can support them; we are not limited to some
"least-common denominator" subset of HTML (for example to support older browsers).
|Supercomputers cost two to five times as much per node (processor) as workstations, PCs, or
multi-processor PCs with a limited number of processors. This is because workstations and PCs are
commodity parts with huge volumes. The sheer volume reduces production costs and amortizes the
significant R&D costs over a larger number of units. The ratio of PCs to supercomputers is about
30,000:1. Unlike Pentium Pro boxes, which are available long before the large-scale machines that
will serve to combine them, commodity building blocks are available first. Given that PC
performance advances at a remarkable 50-80 percent per year, a delay of just six months results in
roughly a 30 percent performance penalty.
Supercomputers tend to have significant memory and I/O capability, which supports an information
system such as HotBot well. Fortunately, it is a simple task to set up workstations with significant
memory and disk space. Even though a single workstation has limits, a collection can handle nearly
unlimited amounts, and enables tremendous aggregate disk and memory bandwidth.
Supercomputers tend to have high-performance networks connecting the processors. Internal
bandwidth varies from 20-200 MB/sec/processor. In contrast, a normal Ethernet for four processors
is only 1 MB/sec/node, and a 100 Base-T switched ethernet might approach 10 MB/sec/processor.
In addition to bandwidth differences, there is an important difference in message overhead (the time
it takes to inject a message). Supercomputers often have injection times that are less than 10
microseconds, while local-area networks typically have injection times around 500 microseconds.
This speed difference is the primary limiting factor to using a NOW as a supercomputer. Inktomi, in
tandem with its custom software, is currently using a new network, called Myrinet, that has
bandwidth around 160 MB/sec/processor and an injection time of about 12 microseconds. We expect
to apply other high-speed networks as they become available.
Supercomputers tend to have unusual operating systems and strikingly few development tools. This
is essentially because of an insufficient market to justify the huge development effort required to
keep pace with commodity software. By using a NOW that uses a normal operating system, we can
exploit all of the latest tools and libraries, resulting in significantly reduced development time.
Supercomputers come as single units in fixed sizes. In contrast, a NOW can be incrementally scaled
by adding single nodes. More importantly, it is possible for a NOW to continue providing service
even if one of the nodes is broken. This flexibility will be the key to providing a fault-tolerant service
that essentially runs without interruption.
|Database access. Audience1 comes with Dynamic tags that can access a DBMS for arbitrary
persistent information and customize the HTML tracking, using either cookies or fat URLs. Unlike
other offerings, while Audience1 supports SQL, it does not require publishers to know SQL to
access the database. This allows Inktomi servers to store and recall a user's preferences for user
interface and query results presentation. More generally, Audience1 is ideal for allowing servers to
access pre-existing databases such as products, inventory, etc.
Browser targeting. Audience1 allows publishers to exploit leading-edge HTML features (such as
Netscape's frames and Java, and Microsoft's font changes and embedded audio tags), without
frustrating users who do not have those features. Audience1's browser targeting can be performed
at various levels of detail, ranging from tags that are easy to use, but don't provide a lot of publishing
control, to exposing the raw browser capabilities to the publisher. For example, advertisers on
HotBot are shown as progressive JPEG if the client browser supports it, otherwise they are shown
as JPEGs or GIFs for less-capable browsers. This allows Inktomi to make the most of each
browser, rather than resorting to a least-common denominator.
Access to high performance, scalable services. Dynamic Tags make it possible for publishers to
introduce new, high performance, scalable services, without requiring the publisher to understand
the intricacies of computing programming. For example, access to the Inktomi search engine is
encapsulated into a single Dynamic Tag, hiding the complexity of interfacing to a parallel program
such as Inktomi. In addition, Dynamic Tags can be multi-threaded, interleaving long-latency
operations such as Inktomi queries and customized content selection (i.e. targeted advertisements).
We know of no other Web-based publishing system with this capability and ease-of-use.
Publishing support hides the complexity of creating and managing sites of dynamic Web pages,
allowing sites with large amounts of content to control the publishing process. Unlike the
CGI-based tools that are emerging, Audience1's publishing support is fault tolerant, high
performance and scales to millions of users and millions of hits per day.
Here's a picture of Inktomi in Soda Hall.