Sunday, July 26, 2009

New Technology

It may not seem like much, but to me the new class of memory devices - Solid State Drives - that are just entering the consumer market are very significant because they very much improve the efficiency of the home computer; and that means eventually, that new classes of applications may emerge.

In the modern processor, the fastest component is the CPU - the central processing unit that actually performs the computations. To get the instructions and data it needs to operate, the CPU relies on a hierarchy of memory systems. This is necessary because memory that is fast — near-CPU speed — is expensive.

The current hierarchy of memory subsystems is
L1 cache
L2 cache
L3 cache
RAM
Hard disk

Latency measures the time it takes between the CPU making a request for something from memory and starting to get the reply. Here is an excerpt from a table from Anandtech for Intel's latest processor, the Core i7 (Nehalem); I expect the numbers are typical for any processor running at a few gigahertz.

L1 latency - 4 CPU cycles
L2 latency - 11 CPU cycles
L3 latency - 39 CPU cycles
RAM latency - 107 CPU cycles
Hard disk latency - approx 10.5 million cycles.

( The average latency of a 7200 rpm hard disk is 4.2 milliseconds - the time for a half-rotation of the disk. A 2.5GHz processor has a cycle time of 0.4 nanoseconds. That is where the 10.5 million cycles comes from. There are faster disks and clever arrangements of disks one can make to reduce this somewhat, but the number remains of the order of millions of cycles. )

SSDs offer to fill the gaping hole that exists in the memory hierarchy. For instance, the just-released second generation of Intel main-stream (as opposed to enterprise class) SSDs, the Intel X25-M has a latency of 65 microseconds, or

SSD latency - approx 160,000 cycles - that is a hundred times faster than hard disk.

Apart from the speed of starting to fulfill the CPU request for data, we are interested also in getting a large throughput, because it would not do much for performance if the CPU had to wait for a long time for the request to complete. The Intel SSD has impressive performance numbers, but it appears that going through the standard processor-to-hard drive interface (SATA) robs a lot of performance potential. The Fusion ioextreme that uses the computer's PCI express bus instead, highlights that potential. See this review.

-----

In my opinion - if "intelligence" is in part speed of reaction, then the CPUs of today are greatly underperforming because of this memory performance wall they crash into. Even SSDs are a thousand-fold slower than RAM. Intelligence or its computer simulacrum is receiving input from the world, interpreting it on the basis of a model of the world, and taking actions accordingly. Any reasonably complete model of the world must be fairly large, and so to have affordable intelligent systems in the home, the processors of those systems cannot be spending most of their time twiddling their thumbs for bringing the representation of the model to the processor.

Our brains get over this problem with much slower components by being massively parallel. Human designers of computers have great trouble with even mild parallelism. Even to get computers to parallelize themselves on such a scale, we would (IMO) need fundamental breakthroughs in the state of the art.

-----

PS: to put the issue of latency into a human context - let us say humans effectively operate at the level of speech or writing at one cycle per second. Imagine two humans conversing. Then L1 cache is like a really slow conversation, taking 4 seconds to receive the response to a question. RAM latency is like waiting for two minutes. Hard disk latency is like waiting for a whole year!

PPS: In this analogy, the SSD latency is equivalent to roughly two days; a processor second is somewhere around 200-250 years.

2 comments:

Rajan P. Parrikar said...

I wonder why the capacity of laptop drives has been static the past year or so (capped at 500GB, as far as I know).

I am hoping the next big upgrade of the Macbook Pros will have sufficient capacity (i.e. 500GB or more) SSD drives as the default, at a reasonable price, of course.

Arun said...

SSDs are still pretty expensive. The first generation of Intel X25-M was retailing at around $350 for 80GB. The second generation just out a few days ago seems to be starting at $270 for 80GB. (It is not yet available anywhere and finding price points is hard). In comparison, an enterprise-class 1 TB Hitachi desktop hard drive can be had for $140 or less.

It will be a while before SSDs become a commodity.