1. SIX is better than EIGHT
We have confirmed through talking with engineers and through testing that 6 sticks in the 8-core system provide maximum triple channel memory performance. (NOTE: 4-core systems are have slower memory throughput even if you use 3 sticks. We measured a 72% drop in throughput with our memory benchmarks versus the 8-core. That's one of the "unadvertised consequences" of choosing the 4-core over the 8-core.)
Having said that, no matter how you populate your Nehalem Mac Pro's memory slots, since the vast majority of real world applications do not saturate the memory bandwidth, you may not see a drop in real world performance. At least you won't be able to measure the difference without a stopwatch. For example, when running 3D Games, the 4-core Nehalem was only two or three FPS slower than the 8-core running the same graphics card.
2. TWICE AS FAST AS 'early 2008' MAC PRO
Anandtech posted a fascinating piece on the 'Nehalem' processor comparing it to the 'Penryn' processor -- both running at the same clock speed (2.66GHz). Their tests show the memory transfer speed of the 'Nehalem' to be DOUBLE that of last year's 'Penryn' -- on which the 'early 2008' Mac Pro is based.
We confirmed this in our testing with DigLloydTools' stress test.
3. APPLE CHARGES TOO MUCH
We ordered our Mac Pro 'early 2009' with the minimum 6GB (6 x 1GB). We plan to upgrade it to 12GB using (6 x 2GB) modules from either OWC or TransIntl. Their prices are much more reasonable while their quality of construction equals or exceeds that of Apple's factory memory.
A 12GB upgrade costs $220 at OWC and $208 at TransIntl. The same upgrade from Apple is $300. A 16GB upgrade costs $290 from OWC and $282 from TransIntl. Apple wants $500. Plus Apple keeps the original modules which, in effect, drives the cost differential even higher. OWC and TransIntl let me keep the original factory memory. OWC will even give me a trade-in rebate if I don't want to keep the factory memory. I usually keep the factory memory in case I need to take the Mac Pro in for warranty repair. That way the techs can't blame my problem on third party memory.
HOW TO INSTALL MEMORY in '2009' MAC PRO
Apple has posted the '2009' Mac Pro manual that includes instructions on memory installation (pages 55-57).
Of special interest is this notation on non-matching memory modules (see page 57):
"Note: If you install different-size DIMMs in your Mac Pro, follow the order in the table. If the DIMM configuration you install doesnt provide optimized performance, the Memory Slot Utility appears onscreen and recommends an improved configuration."
Seems that the Memory Slot Utility contradicts the manual. When you put two modules in slots 1 and 2 (as suggested by manual), the Memory Slot Utility recommends slots 1 and 5 instead for optimal performance. (OWC discovered this and posted it on their blog.)
