Last month, I sat down for an exclusive interview with Facebook’s Vice President of Engineering Michael Schroepfer to talk about Facebook and his engineering organization. This is Part I of a multi-part series.

Michael Schroepfer is the key executive in charge of the growth as the VP of Engineering. He works directly with Mark Zuckerberg to extend Facebook’s leadership on the engineering side of their massive growth. Before joining Facebook Mike was previously the Vice President of Engineering at Mozilla where he headed up the engineering team responsible for Firefox. Before that he was the CTO of Sun Microsystem’s data center automation division.

Overall Mike is a really good guy and as they say “wicked smart and nice”. I enjoyed meeting him and getting his perspective.

Facebook user growth is skyrocketing and their product and platform continue to be highly available and reliable.

If there is any doubt that Facebook is scaling then look no further then their massive growth over the past three years. Facebook’s engineering group has assembled a unique team with a cutting edge approach. The Facebook formula allows them to scale at such a high rate that many old school folks are scratching their heads wondering how and why? Facebook is building their own cloud.

Facebook is a serious platform, and there is no doubt that it will be around for many more years to come. Facebook has proven itself. It has proven that they can scaled and continue to scale.

My big takeaway is that Facebook has a “homegrown” unique formula for ramping up their backend systems to scale at a rapid rate. This unique formula to Facebook’s success is grounded in three principles: open source software, unique talent for programmers, and an operating system software approach. In layman’s terms, they have designed their product that takes advantage of today’s computing environment and they have build in relevant software to deliver flexible and scalable user features (like today with the announcement of the redesign of Facebook Groups).

Others big players like Amazon, Google, Apple, Yahoo, and Microsoft are competing on scale. Others such as startups like Twitter and existing players like IBM are struggling to be competitive and build scale.

We all have watched Twitter grow like a weed in 2009. We have all seen Twitter’s FailWhale screen during their ascent to 10s of million registered users and 100s of millions of users. It’s been a struggle for Twitter. It can’t be said for Facebook. They are building scale from the ground up. Watching Facebook reminds me of the first 5 years of Google. During those Google’s growth years, they build scale from the ground up to handle their massive growth in search.

Can Facebook maintain their growth and scale? Where is their “FailBook” logo? We’ll see. Meanwhile, Google isn’t out for the count yet. They have scale from their legacy search business and are trying to find a way to leverage it – for App Engine, Wave, and other initiatives. Others like Microsoft and Yahoo are building up a cloud vision and hope to have something soon. Getting to scale and leveraging scale is hard. Developing it around or for a product is even harder.

Scale matters and it’s clear that Facebook will be the next Google in terms of market dominance and revenue.

Here is Part I of the Facebook Engineering Series

Operating At Scale – Part I: Exclusive Interview with Facebook’s VP Engineering Michael Schroepfer

John Furrier: Operating at scale is a major differentiator and Amazon and Google use scale as a key benefit of having such a large infrastructure in terms of iterating and innovating. The question is how does a new young and growing company like Facebook leverage their unique infrastructure and its growing scale?

Mike Schroepfer: For Facebook we look at this issue as a big deep system problem. If you look at the background of many of the engineers at Facebook, they are all deep computer scientists in a variety of disciplines related to systems whether it’s operating systems, networking, or storage. The mix of engineers consist of those that are right out of college with those that are 10-20 year industry veterans from other companies that have seen large scale systems such as Netapp, VMWare, Google, and others.

We have seen big scale system before and have learned lessons in the past. We don’t want our engineers implementing their first file system on your first day at huge scale. There is just a lot to learn. So we have the deep technologists that you need to do this combined with boldness to try new things.

Our goal is to move fast. The other half of our mission is to break stuff. We have a willingness to try stuff out, and see what happens. This includes the system side where engineering is difficult. We have a real willingness to build a new piece of hardware or to build a new brand new piece of software that’s in turn is a new storage system unique to Facebook. Throw it in production and see what happens. Test it out with real load and see if your environment scales.

The willingness to build stuff and try it by fire and not assume that everyone else has figured it out the problem is critical and part of our attitude of the engineering organization. For example, if you look at Haystack storage system which is how they store all of their photos is seeing a tremendous amount of traffic and usage with over 1 billion uploads a month.

People don’t realize it but photos are critical to Facebook. It not just storing the photos and sticking them in a mountain of disk and maybe they’ll get them in a week. We also serve a tremendous volume of photos to users. Every time you load the home page you see lots of photos. Page views on photos.php is very high and the people are clicking on lots photo all the time. Haystack is an integrated storage and serving system – 2u boxes that store the photos and serve them back directly to users at a high rate. It’s a high density storage and high throughput in terms of rates.

Next up Tomorrow: Part II: Facebook’s Engineering Vision