One Month with Sitecore 7.5 Part 4: Sclability Options, New and Old

• Part 1: Milwaukee Sitecore Developer’s Meetup Presentation (video)
• Part 2: The Experience Profile and Experience Database
• Part 3: xDB Setup, Contact Identification and Shared Sessions
• Part 4: xDB Deployment, On Premise vs Cloud
• Part 5: Persisting Contact Data to xDB
• Part 6: Extending Report Data via Aggregation

It’s actually going on two months with Sitecore 7.5 at this point, and I’ve obviously gotten myself in over my head with this blog series, but I’m battling on. Here in Part 4, we’ll take a look at deployment options for Sitecore 7.5, both new and existing.

A more in-depth reviewing of existing Sitecore hosting architecture considerations can be found on Aware Web’s blog.

Minimal Deployment

Despite the additional complexity Sitecore has added to the DMS architecture in 7.5, it’s still possible to run full Sitecore functionality in a single instance of the software. This obviously simplifies things for developers, and means that even small deployments can take advantage of the xDB. However, unless you are utilizing the xDB Cloud Service, running MongoDB is necessary. With the addition of xDB Cloud and Sitecore’s “enhanced” SQL Sessions (to support Session_End), you can avoid that requirement.

Scaling via Server Role

In 7.5, Sitecore continues to provide new options for offloading server roles onto dedicated hardware (or virtual hardware). This does allow you to vertically scale (in the traditional use of the term) individual servers according to the needs of the service they are running. Let’s review all of our available options…

Content Management and Content Delivery

The longstanding scaling option in Sitecore, splitting your Content Management (CM) and Content Delivery (CD) servers is typically your first step in growing a Sitecore environment. This is sometimes called “content staging” and can be done for performance, availability, and security reasons.

Publishing

This has essentially been an option since Sitecore 6.3 (someone please correct me if I’m wrong on that), but since the publishing process was limited to a single thread, there was not much benefit to splitting publishing responsibility from your CM server. Sitecore 7.2 changed this however, and multi-threaded publishing can now easily consume CPU resources on a multi-core server. Large instances with many items and frequent content changes will benefit greatly from a dedicated publishing server.

Content Search

Using either the Solr or Coveo providers for Sitecore.ContentSearch, it’s possible to offload content indexing and searching onto a dedicated search server. Useful for deployments which are highly dependent on content search, or which need to handle searching of massive amounts of content.

SQL Server

Even the most minimal of Sitecore installations should likely have an independent SQL Server. In addition to the standard Sitecore databases (core, master, web), in 7.5 this would house the reporting database and potentially a session database. Depending on the amount of analytics data, you may want to take this a step further by splitting your reporting into a separate SQL Server. This is a must if your hosting architecture is geo-distributed, since you will need a central reporting database in which to aggregate data from your data centers.

MongoDB

Unless you are using xDB Cloud (which we’ll discuss shortly) and using SQL Server for your sessions, to use analytics in 7.5 you will need to run MongoDB. For low traffic sites, it may be possible to run it on the same hardware as SQL Server, but given the low cost of virtualized Linux servers, adding a dedicated MongoDB install is an easy scaling option. You might also consider separating the MongoDB session database onto its own server, again a must if you have a geo-distributed architecture.

Processing / Data Aggregation

For high traffic sites, splitting off processing and data aggregation responsibilities to separate hardware will decrease impact on content authoring, and speed up data processing. Another option might be to dual-purpose your CD servers, and take advantage of distributed processing using your existing cluster.

Reporting Services

High traffic sites, or those making heavy use of reporting, may also benefit from splitting responsibility for the Reporting Services, which run queries and combine data from the collection and reporting databases in order to support reporting UIs such as the Executive Insight Dashboard and Engagement Analytics.

Scaling via Clustering

Many aspects of the Sitecore deployment architecture can also “scale out,” so that as your needs increase, you can add additional servers for load balancing and failover.

Content Delivery

Almost always your first need for scaling out. Adding additional content delivery servers is the primary mechanism by which you can improve your site’s performance and availability. With the publishing improvements in 7.2, and the new analytics architecture in 7.5, it’s now also much more practical to deploy a geo-distributed architecture, with multiple clusters of content delivery servers. By utilizing the SQL Server or MongoDB session state providers in Sitecore 7.5, it’s also possible to implement non-sticky load balancing within each webfarm, which gives a better load distribution between the servers, and increases the reliability / failover capability of your webfarm.

Content Management

For organizations with a large number of content authors, adding additional content management servers allows scaling of the authoring environment. Since you can only have a single master database, all content management servers must be local to the master database.

MongoDB

One of the primary benefits of MongoDB over SQL Server and other relational databases is its ability to scale horizontally. Sharding allows MongoDB to distribute data across multiple servers using a shard key. Replication sets mirror data between servers for failover. Combining the two gives you a sharded cluster. This allows MongoDB to scale horizontally for huge data sets, on low cost Linux servers. This is not without complexity however. See the MongoDB guide to Sharded Cluster Architectures.

Processing Servers

The new Sitecore 7.5 data processing/aggregation services utilize worker processes which read from a processing pool and feed them to “aggregators” which process the data for reporting purposes.. This makes it possible for deployments which are processing huge amounts of data to run multiple processing servers, all of which can process data from the Collection database and aggregate it to the Reporting database.

Between versions 7 (content search), 7.2 (publishing), and 7.5 (analytics), Sitecore has made many improvements on the ability of the software to scale to massive amounts of content and analytics data. But with it has come additional deployment complexity. To mitigate this, and make it possible for any Sitecore deployment to easily take advantage of the xDB, Sitecore plans to offer xDB Cloud.

xDB Cloud Service

Complimenting the release of Sitecore 7.5, xDB Cloud will allow you to take full advantage of the Experience Database and all its reporting without having to collect, store, or process any analytics data locally. 100% managed by Sitecore, this means you can get away without having to run MongoDB in your environment — provided you are using SQL Server for session data.

Sitecore plans to offer xDB Cloud at a “low” cost based on the number of contacts stored in the xDB. They will also be offering non-production access at a lower cost (with a corresponding lower SLA) for use in development and testing.

What’s most exciting about this new offering is that Sitecore is removing barriers for all their customers to better utilize the Digital Marketing System. Large customers with huge amounts of data can take advantage of the new, highly scalable architecture. Smaller and larger customers alike also have the option of outsourcing their analytics infrastructure to Sitecore with a service that they know will grow with their data needs.

Configuration and Use

Sitecore was kind enough to grant me access to a preview of xDB Cloud for evaluation with Active Commerce. After requesting an instance from Sitecore, a step which will be replaced with an easy App Center purchase in the future, I was given a Deployment ID. I enabled the Sitecore.Cloud.Xdb.config, filled in my Deployment ID within this file, and… that was it. I admittedly did not spent a lot of time testing this service, but it certainly appears that they have achieved their goal of making xDB easy to use via this SaaS offering.

Limitations

One of the most exciting aspects of xDB, which I will be investigating in the last part of this series, is the extension of reporting data. The data aggregation and reporting architecture in 7.5 makes it possible to extend the data collection and analysis performed by the DMS. However, in the initial release, it will not be possible to extend data aggregation in the xDB Cloud offering. Sitecore does plan on addressing this in a future release, and all appearances are that it is a high priority item for them.

MongoDB Cloud Hosting

This article was going to end here, but I was inspired to try one last experiment with Sitecore 7.5 this morning — utilizing ObjectRocket’s hosted MongoDB service for the collection database and the other Sitecore 7.5 MongoDB databases. ObjectRocket, owned by Rackspace, has a very nice, scalable cloud offering for MongoDB. They have several data centers which are local to both AWS and Rackspace data centers, making it a very attractive option particularly for those who are already hosting with Amazon or Rackspace.

I created all the needed MongoDB databases in an ObjectRocket instance and configured my connection strings to utilize them. This included a MongoDB session database. In a production scenario, this may or may not be practical, depending on your latency to ObjectRocket, as the session database would presumably be more sensitive to latency. But with just myself as a single user browsing my development site, there did not appear to be any performance degradation. Even the Experience Profile report, utilizes the collection database, seemed to perform reasonably.

Using a service such as ObjectRocket with Sitecore 7.5 would potentially give you the ability to scale to massive amounts of data, without maintaining your own MongoDB infrastructure. However unlike Sitecore’s xDB Cloud offering, you would still have the complexity of maintaining your own processing and reporting services. And you would certainly want to do additional performance testing of both the site and data aggregation!

That’s it for Sitecore 7.5 deployment architectures. In Part 5 of this series, we’ll get back into some code, and look at how you can extend the Contact data which is persisted to the xDB.

-Nick