Press Archive - 2003-Jun, NEC My Point Of View

No. 5 - Power In Numbers

NEC, IT Solutions
June 6, 2003

Adam Beberg
Founder of the non-profit group- distributed.net
Founder, Chairman & Chief Technology Officer, Mithral Communications and Design.
Founder of Cosm- an open-source distributing computing project.

Whiz-kid computer scientist Adam Beberg is a pioneer in the new field of Internet Scale Operating Systems (ISOS). His 1997 project distributed.net became a sensation when it enlisted the help of personal computers around the globe to crack a "high- security" encryption code. In the future, he says, a large and flexible network of PCs will function as virtual supercomputers, solving complex problems for science and business.

Sharing the Work

AB: A multitude of computers or people can work together on the same project under the control of one central organization. This kind of "distributed computing" takes a complex task, breaks it down, and distributes parts of the work to many different computers. Each computer does a small amount of work and together they complete the bigger task - something that would be impossible for a single computer working alone to accomplish. The result is a virtual supercomputer many times faster, much more affordable and constantly updating itself as participants upgrade their computers.

Internet connectivity and ample bandwidth make this possible. The networks are finally fast enough to do what we've been trying to do for the last 30 years. Eventually this kind of distributed computing will transform the Internet from a document-based network of Web pages and e-mail into a dynamic network where specific components of information can be located and shared more efficiently. Computer scientists, researchers, corporations, universities, feature film animators, weather forecasters - any industry that requires massive amounts of data crunching will benefit from distributed computing. Ten years from now distributed computing will be an established commodity traded on an open market. Clients will be able to choose from among a variety of options and prices, and suppliers will be compensated for their contributions.

Making computers work the night shift.

Imagine walking into a design lab or an office after business hours and seeing banks of workstations sitting ready but idle.. Add far-flung computers situated in an organization's offices around the globe and what you have is a waste of resources.

By rewriting software, it is possible to chop structural analyses into manageable chunks and distribute them to workstations to process overnight. The content can run the gamut from graphics to risk analysis. Protectedbehind a firewall, large private corporations will take advantage of their existing computers' idle time and lash combined computational power into the network - creating a virtual supercomputer designed to support unbounded expansion. It's essentially free since the computers are already used during the workday. The future cost savings in hardware alone are potentially enormous.

Grid supercomputing will be much more controlled; redundancy and encryption are just the beginning of techniques assuring a company's data is consistently available to the corporation and inaccessible to anyone else. No node will act on a command or use data until it has verified that the command has come from a trusted source. Anything that is not from a trusted source will be discarded and measures taken to ignore further commands from that source.

Each company will want its own standards - its own way of doing things. Super computer guys, by whom I mean the large computing companies, will tailor the infrastructure to the client's needs.

Public and private networks: Teaming up on tough problems

In 1997 I was a founder of distributed.net which was really the first large public example of distributed computing and paved the way for future programs. The distributed.net program runs as a screensaver on the participating computers, and is only active when the computer is idle. Each chunk of data can be processed independently and the computer only needs to communicate with the central server. Using a brute force search distributed.net was able to crack a complex encryption contest from RSA Security Inc. By linking 26,000 computers we were able to check over five billion keys per second on average. We broke the code in 212 days. Using the same technology we also cracked two DES (Data Encryption Standard) and another RC5 contest. The success of distributed.net has been followed by many projects including dozens of start-ups.

A very recent example of distributed computing that I've been involved with is Folding@home - Stanford University's investigation of protein folding. Proteins are the workhorses of the biological world and can act as enzymes that drive biochemical reactions or antibodies that fight unwanted invaders. Proteins that fold improperly can result in diseases such as Alzheimer's, cystic fibrosis, Mad Cow disease and even some cancers - so you can see that this is important research.

But how do you simulate this process in a computer? Simple proteins assemble in about 10,000 nanoseconds and one 400MHz computer can simulate one nanosecond of assembly time in about one day. This means that 1,000 participants are needed to simulate a simple protein fold in 10 days. More complex proteins can take far longer. Currently almost 90,000 computer users have volunteered to put their computers to work on the problem, but the task is enormously complex.

[Screenshot of the Folding@home software, one of many internet distributed computing projects, showing one protein that is being studied by Stanford University.]

Folding@home was the first distributed computing project that resulted in published scientific results. This is pavingthe way for many more projects like Genome@home at Stanford which examines how proteins and the human genome relate to each other.

In the Folding@home and Genome@home type of computing, results are constantly double-checked but the client is not overly worried that someone might try to steal data or send fake computations. This type of distributed computing will continue to appeal to the non-profit public information client. But for private corporations and other security minded clients, it isn't an option. In order to get the desired level of control and security, these companies will have to do it on their own network or on a secure network that they hired. It's fundamental; you will never see, for example, multinationals or a drug company out there saying, we want to do this computation, "please help us." It won't work that way. So there will be proprietary networks with lots happening behind closed doors.

Tapping the potential, making it easier

As more computers link up to the Internet the potential for distributed computing is growing exponentially and completely changing the scale of what you think is possible. Soon companies will be able to instantly purchase extra processing power on demand. Already there are a handful of online exchanges creating portals that would allow commercial users to purchase idle computer power with one click.

But the problem for future users, like universities and businesses, is that up to now distributed computing solutions tend to be so problem-specific that no two projects really have much in common at all beyond the core set of needs. In order to solve this problem I'm refining Cosm - a distributed computing system built on top of traditional operating systems.

Cosm's aim is to provide the basic tools necessary to build any type of distributed application. Cosm offers a generic architecture that will give the user the ability to hook up even more computers in a project, and manage them efficiently. In essence, Cosm works across all platforms and it scales easily from handling systems on a local network to handling Internet scale operating systems. Cosm takes care of all the porting issues. The client can be "dumb". They can devote all their time to their project and leave the local system management to Cosm.

Cosm's tools will help develop services that make computer-processing power a commodity traded on the open market. It will be commercial but free for many users. We made significant contributions to projects like Folding@home by developing the tools (Mithral CS-SDK) necessary to build the client and server code, especially the networking and platform independence routines.

The future is simple

In the future, I'm looking forward to creating a single downloadable application that will oversee everything on your computer for you. This will be intelligent software that will manage your distributed computing tasks, manage encrypting instant messages and mail and things like that. At the moment there's no application out there that's easy to use that involves encryption. So right now you've got to deal with a lot of different pieces of software. I'm working toward a kind of a one-stop application that ties everything we deal with on a computer together. We don't have that right now. It doesn't exist.

The biggest challenge down the road is to take the complexity out of distributed computing and web services. The thing that really makes technology great is when you don't realize that it's there. My goal is to try and reduce it to one button - to make it easy. I think that the people who can do that are the ones that are really going to own the technology in the future.

Bio

In the early 1990's, while still in high school, Adam Beberg attended university half-time. He was fascinated with ray tracing used in creating 3D graphics, and running movies and graphics but the computers were too slow. So he devised a way to split the content into pieces and divide it among several computers to allow faster processing. That was his initial foray in to the world of distributed computing.

Beberg obtained a degree in Computer Engineering from the Illinois Institute of Technology and in 1995 founded Mithral Communications & Design. In 1997 he founded distributed.net . In April 1999 Beberg left distributed.net and returned to work at Mithral where he is the Chairman and Chief Technology officer.

MIT's Technology Review lists Beberg as one of the nations top 100 technology innovators. His work has appeared in many publications including Red Herring, Forbes, Scientific American and Fortune.