Trev Harmon

  • Home
  • Business
  • HPC
  • Cloud
  • Big Data
You are here: Home / Cloud / Customizable vs. Bespoke – Why HPC Cloud Struggles

April 9, 2013 by Trev Harmon 2 Comments (REPOST)
April 2, 2013 at Adaptive Computing (ORIGINAL)

Customizable vs. Bespoke – Why HPC Cloud Struggles

It was the late 18th century, and man was harnessing the power of steam. The Industrial Revolution was underway, a movement that would forever change the face of the earth for both good and bad. In the middle of this cataclysmic change, an invention was born that would inspire Charles Babbage‘s Analytical Engine, the forerunner of modern day computers. This invention set in motion a chain of events leading to the Information Age, while changing the face of manufacturing forever.

This invention, first demonstrated in 1801, was the Jacquard loom.

Using a punch card system to program the loom, the weaver could instantly produce many different patterns. Though impressive as one of the first programming languages, the Jacquard loom did something else–something indicative of the entire Industrial Revolution. It ushered in mechanical production line processes. Faster, cheaper mechanical production meant the widespread use of handcraft to manufacture was at an end.

Interestingly enough, the Jacquard loom did offer something many of the other processes born in the steam-induced heat of the factory floor did not: customization. Regardless of what Henry Ford said regarding the color of his automobiles, customization had a way forward in this new world order.

But, is this a fair replacement for handcraft?

Consider the two following terms:

Customizable
to be modifiable according to individual or personal specifications or preference

Bespoke
made to individual order; custom-made

 
On the outset, they appear to mean almost exactly the same thing. But, there is a very important difference between the two. Namely, with customization one starts with an existing product and adapts or changes it. With bespoke, the product is created from scratch. In other words, with bespoke the product can be created with any given attribute, while customization only allows the modification of certain, predetermined attributes.

Cloud is customizable. HPC is bespoke.

The fact of the matter is when I go out to the cloud; I’m not really given

very many “low-level” decisions to make. In most cases, I’m only allowed to choose from some predetermined packages to which I may be able to make minor tweaks like determining the software stack. I likely won’t really know anything about the actual hardware—what CPU model, what memory types, bus speeds, interconnect, general networking, or another of myriad things.

And, really, that’s okay. I don’t want to care about those things. That’s not what I’m looking for from cloud. I’m optimizing for availability and ease-of-use.

HPC is different—especially at the high end. Here every hardware decision matters from top to bottom. Even the building where I store the machines needs to be engineered specifically for that purpose. Maximum performance requires me to care about memory configuration, networking hops, CPU parity, and the list goes on and on.

So, what do I do? I turn to a manufacturer to build me exactly what is needed. I match my workload to specific hardware specifications. I’m optimizing for utilization and speed.

The Bespoke Suit

Having a bespoke suit (or any other piece of clothing) made to order is a wonderful experience. Putting on the bespoke piece of clothing is… well… magical because it just fits. It feels good and moves perfectly in all of the right places. There is no stretching, straining or pulling. It was made specifically for you. It also has one ruinous effect—once one has experienced it, it is hard to go back to off-the-shelf clothing.

High-end HPC is no different. The supercomputer perfectly matched to its workload is unbeatable. It’s tuned so that each part works in perfect harmony with all the other parts. It is bespoke–perfectly created for its purpose. This is something cloud cannot yet offer.

It is true that some workload, like the types of jobs run by the BOINC network will work fine in a cloud environment. They are not so dependent on the hyper-tuning needed to squeeze every last possible compute cycle out of the processor cores. This can also be the case with many of the small- to mid-size clusters. If so, HPC in the cloud (or HPC Cloud as it is sometimes called) may be the way to go.

But, for the high-end in terms of size or throughput, the basic customizations available on the cloud and their inherent drawbacks aren’t currently able to provide what is needed. Perhaps in the future they will. But right now these HPC practitioners have tried on the “bespoke HPC suit,” and the fit is too perfect. Why would they want to go back to something off the shelf that is almost, but not entirely a good fit?

58987235_97b6877fa9_n

The clothes back in those days were made so much better than clothes are today. They actually took time to make clothes to fit a woman’s body. Today they make clothes that fit sizes, so it stretches to fit this and that.
~ Debbie Allen

While teaching at university, I frustrated countless students by giving the answer, “It depends.” Things are no different here.

Pick the tool right for the task. Just because you have a hammer, not everything is a nail. If you happen to fall into the magic middle ground where HPC cloud may work for you, consider tools like Adaptive Computing’s Moab that support you doing both.


Picture courtesy of Diego Torres Silvestre

Filed Under: Cloud, HPC Tagged With: Analytical Engine, bespoke, BOINC, Charles Babbage, Cloud, COTS, custom, Debbie Allen, Henry Ford, HPC, HPC Cloud, Industrial Revolution, Jacquard loom, Moab

Comments

  1. Daniel Hardman says

    April 4, 2013 at 21:42

    Excellent insight, Trev! I had never considered the bespoke vs. customizable analogy, but now that you explain it, I see its resonance in many places.

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

The Manifesto

Conscious Business Ethics

BusinessEthics300

Learn why it matters. Then sign it!

Get Updates!

Affiliate Disclosure

This site offers affiliate links to some online retailers such as Amazon.com in conjunction with hyper-linked books, movies, music and other such items. IF you click on these links and subsequently make a purchase, I will receive a small percentage of the transaction price

Twitter: trev_harmon

  • I've been working on a side project to help people contextualize and humanize the toll of #COVID19. It's finally do… https://t.co/Qg8ri7KSfJ 05:54:48 PM April 05, 2021 ReplyRetweetFavorite
  • RT @KungLab: Are you sad and angry about the surge in anti-Asian violence? Wondering if there’s something anti-racist you can do? One idea:… 07:58:56 AM March 19, 2021 ReplyRetweetFavorite
  • RT @saysthefox: I think everyone could use a lighthearted/happy story right now so here goes: At the beginning of the pandemic I went thro… 10:17:21 AM December 12, 2020 ReplyRetweetFavorite
  • #DoGoodRecklessly https://t.co/EjpxC0Mess 09:13:43 AM December 12, 2020 ReplyRetweetFavorite
  • 😁 https://t.co/bjYurTuVq0 11:20:50 PM December 11, 2020 ReplyRetweetFavorite
@trev_harmon

Writer, software architect, educator, blogger, photographer, would-be designer, and a believer in the power of simplicity and human-based design.

Other Blogs

Trev Harmon can also be read at:

  • Dream.Learn.Discover
    Primary Author -- This blog is about seeing the good in the world. With all the bad, evil and destruction, there are many, many people who are creating good in their sphere of influence. Some of these spheres are large and some are small. There is a time allotted to each one of us. It is with this time some decide to do remarkable things, though they may not believe them to be remarkable at the time.
  • Adaptive Computing
    Contributor -- The world of high-performance, cloud and supercomputing is opening the way for many new and exciting discoveries. As we push our quest for knowledge forward, technology will play a key role in supplementing our ability to learn and discover.

Copyright © 2021 · Executive Pro Theme on Genesis Framework · WordPress · Log in