Hadoop, Throwing Hardware, and Engineering for Results
January 7, 2010
We hear, “Let’s throw hardware at this problem.” Do you? Several years ago we did a SQL Server performance analysis for a major organization. The problem was not SQL Server; the problem was the suite of software that used SQL Server. We gently recommended a project to recode a couple of troublesome modules. You know what the client did? Yep, threw hardware at the problem. The system still ran like a slug, but the customer is king.
I enjoyed “The Mathematics behind Hadoop-Based Systems.” The goslings love Hadoop, a Google-blessed approach to stream processing. But this write up is more than a clever way to calculate the size of a Hadoop cluster. The author provides a case example of how to think about a problem.
Even if you suck at math, you will find the explanation, the logic, and the approach useful. Instead of following the advice of your hardware rep, why not sit down and do some thinking about what is being processed. And if you want to throw hardware at a problem, do the throwing with informed intent, not to make your IBM, Dell, or HP rep “like” you.
A most useful write up.
Stephen E. Arnold, January 7, 2010
I want to admit that I did not receive one bent penny tossed at my feet for this write up. Because of the literary allusion, I shall report this sad fact to the Library of Congress.
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
