Craig Weissman, Chief Software Architect (and just announced new CTO) of Salesforce.com, gave a presentation on “The Magic of Multitenancy: Under the Covers of the Data Architecture of Force.com” at the SDForum Software Architecture and Modeling Special Interest Group in Palo Alto on March 25th, 2009. This talk shared a lot of content with an earlier presentation at PARC Forum given by Todd McKinnon, ex SVP for Software Development at Salesforce. This is because both presentations derive from an earlier presentation given by Craig at Dreamforce 2008. However, Craig went into a bit more depth and detail in part because he was in a better position to do so, having invented some of the technology, and in part because of the relatively large number of software architects present and because of the interactive nature of SDForum SIG meetings.
Some of Salesforce’s actions and strategies have been creative and innovative (even daring!). In particular, their approaches to multitenancy at the business level and at the level of implementation are new. At the business level, they followed a nearly religious conviction and refused to break out any individual customer for an on-site or customized implementation. At the implementation level, they created an innovative way of housing multiple tenants in their database by using metadata and effectively building a multitenant database within their conventional database. Both of these help them achieve economies of scale and help them reduce the amount of hardware and software (especially database) infrastructure required.
Craig’s talk highlighted huge advantages of Salesforce’s approach. In their Platform as a Service (PaaS) approach, their clients are shielded from a lot of issues that their clients would normally have to deal with. Issues made invisible and taken care of by Salesforce include:
- multitenancy;
- high availability;
- load balancing;
- scalability;
- backups;
- security; and
- operating system patches
Another huge advantage Salesforce has is not having to deal with different possible hardware and software underlying their own software since they own the platform. They can code to Oracle and not have to worry about other databases and so on.
Some of the interesting details Craig provided include the following:
- Salesforce is strongly committed to Oracle in part for historical reasons (early employees and founders were from Oracle or were “Oracle people”) and in part because Oracle gets a lot of things right that they didn’t want to have to write for themselves;
- they get a large number of clients (>50k companies) on a surprisingly small number of servers (500 or 1k with geographic redundancy);
- most tenants are small but some have tens of millions of rows;
- they cache crucial data such as metadata and use memcached;
- they use other open source including a customized version of Lucene;
- they use F5 hardware for one layer of load balancing;
- they run their databases on Sun servers and the rest on Dell;
- Craig invented Apex (originally naming it PL/SOQL as a take-off on Oracle’s PL/SQL)
Salesforce’s “build versus buy” decisions are interesting. They chose to “buy” and implement on top of Oracle in part because it has advanced features and it supports higher performance that they felt were not present in open source alternatives like MySQL instead of attempting to write those low database level features themselves. They chose to “build” and implemented a database on top of or within the Oracle database as part of implementing multitenancy because no database vendor was providing the kinds of things Salesforce needed at the time.
Salesforce chose to “build” almost all of their application server. They use Resin, a Java application server, but they only use the servlet engine. They wrote all the rest of their application server code including relatively basic things like their connection pool manager. This seems somewhat inconsistent with the thinking behind the purchase of Oracle for the database layer. But it is consistent with other large web applications like eBay. (eBay also implemented their own connection manager and so on.) I wondered why companies don’t depend on a company that is expert in application servers (like IBM or Oracle) to specialize in app servers and provide high quality generic infrastructure enabling companies that use them to focus more on their business applications. I asked Rob Woolen at Salesforce, formerly at BEA Weblogic, about this and he noted that some people simply feel that off-the-shelf software can’t handle their application (e.g., at eBay and on Wall Street). Rob also noted that engineers like to build rather than buy and companies like eBay and Salesforce have a lot of talented engineers who can build infrastructure quickly.
Later, I asked Craig for more about the big picture and he listed some of the things Salesforce is focusing on moving forward.
