The SharePoint Maturity Model (SMM)

MOSS has been the most successful server product Microsoft ever released. Sales are growing much faster than Microsoft ever expected and apparently the UK is outstripping worldwide growth (for more see this). Unfortunately this quick growth is also highlighting one of the major problems that everyone seems to be struggling with - deployment. I'm a SharePoint consultant and I've been working with MOSS since Beta 2. I've worked across several companies in the UK and I have also debated this issue with other colleagues and we are all in agreement: deployment is the biggest pain on any SharePoint project. It's one of the areas that will give you more problems and cost you more money. What is curious is that all companies adopting SharePoint seem to go through the same evolution path. Finding a way to measure where my customers are on this path gives me a good idea on the challenges that I will be facing when moving their projects forward. The kind of measure that I'm talking about it's called a Maturity Model so I called it the SharePoint Maturity Model (SMM).

Like the Capability Maturity Model (CMM) I divided the SMM is divided into 5 levels (except for level 4, I have used the same names has the CMM because they are a good match):

1. Level 1 – Ad hoc (Chaos): Companies in level one normally do all development directly in the production system. There is no development or system test environment because no one knows how to move the site to a different server. These companies are normally just starting using MOSS and they do not have experienced MOSS resources.
   
2. Level 2 – Repeatable (Backup/Restore): Level 2 companies have found that SharePoint content and configuration can be moved across environments using backup and restore. As such they have a repeatable deployment process. Unfortunately these companies soon realise that using backup and restore will cause a series of problems that are not immediately noticeable (like the publishing pages still pointing to page layouts on the old environment or exceptions being raised when trying to edit publishing page settings). I was once called to a customer that was having a series of problems with their live environment; it was running too slow with frequent crashes, also they could not understand why the problem seemed to become worse when they turned off their development server (which was on the same network as live). It turned out to be the typical backup\restore syndrome that had to be fixed by copying the content of the publishing pages to new pages created in the live environment (we gave them a small C# script to do that).
   
3. Level 3 – Defined: Companies normally get to level 3 after learning with their mistakes and spending a lot of time and effort going through levels 1 and 2. In level 3 companies have learned that there is a better way to deploy their applications by using CAML in Features, Solution Packages and Site Definitions. Unfortunately there are quite a few configuration artefacts (like security, search, etc) that cannot be deployed via CAML. To overcome this problem companies develop a mixed automated\manual deployment process where some artefacts are deployed via CAML scripts and others by following a long list of manual steps (see Deployment Pete Syndrome below).
   
4. Level 4 – Automated: At this level companies have managed to fully automate their deployment process by using a mixture of CAML and .Net code. This normally comes at great cost because not only developers will need to learn CAML but they will also need to learn the SharePoint API so that they can automate the deployment of configuration artefacts not covered by CAML.
   
5. Level 5 - Optimised: This is the SharePoint deployment Nirvana. At this level companies not only have their deployment process fully optimised and automated but they have also started to manage those processes has shared Intellectual Property (IP) across projects. At the end of every project there is an IP harvesting process that brings any new deployment code into a central location so that it can be reused and shared by future projects. The time required by these companies to setup a new SharePoint project is minimal and quality of their deployment scripts is high because it's based on tested code used several times before.
   

The Deployment Pete Syndrome (DPS):

One of the constant complaints that I seem to find when meeting new customers is about the exceeding amount of time that it takes to move their SharePoint application between environments. Especially when they have been promised -normally by a sales person more interested in hitting their sales target than in providing a fit for purpose solution- that most of their requirements could be accomplished with out-of-the-box (OOB) functionality provided by SharePoint (I'll leave my OOB ramblings for another post). During my first meeting with these customers I'm normally introduced to their deployment expert which, for the purpose of this post, I will call Pete in honour of a great ex-colleague of mine that used to spend a lot of is time deploying SharePoint applications. In further talks with Pete I normally discover that he is given a huge list (sometimes several dozens of pages long) of manual steps that he needs to follow in order to move each SharePoint application between environments. I call this the "Deployment Pete Syndrome" (in lack of a better name) and unfortunately it seems to be a constant of most companies that I go to (especially those in levels 2 and 3 of the SMM).

In my opinion one of the ways to help companies adopting SharePoint and ease the pain of reaching the SMM level 5 (if they wish to do so) is to offer them a mixture of advice and knowledge transfer together with a series of packaged tools that have been designed and developed based on the experience past projects. This is one of the main objectives of a new UK based company called Collaboris (we are still finishing the web site so please bear with us) that I've started together with my colleague Mark Jones.

How to free your Data View Web Part (DVWP) from those nasty GUIDs

Ever want to use the same page layout containing a DVWP, and bind it to a local site list in each site of your site collection ?

Ever want to be able to deploy a page with a DVWP to a different server?

If you ever tried any of the above using the SharePoint Designer DataViewWebPart you would have found problems because, by default, SharePointDesigner binds the control to the list instance using the list instance GUID. To resolve this we need to replace the GUIDs by the list name. The steps to do this are:

1) On the attributes of the DataFormWebPart element replace the attribute ListName="{GUID}" by ListName="LIST_NAME" where LIST_NAME is the name of the list that you are binding to.

2) Go through all of the DataFormParameter elements and replace:    

    <WebPartPages:DataFormParameter Name="ListID" ParameterKey="ListID" PropertyName="ParameterValues" DefaultValue="{GUID}"/>

With:

     <WebPartPages:DataFormParameter Name="ListName" ParameterKey="ListName" PropertyName="ParameterValues" DefaultValue="LIST_NAME"/>

3) Go to the ParameterBindings element and replace

    <ParameterBinding Name="ListID" Location="None" DefaultValue="{GUID}"/>

With:

    <ParameterBinding Name="ListName" Location="None" DefaultValue="LIST_NAME"/>    

This should give you a GUID free DVWP that can be placed on a page layout used by multiple sites on your site collection (as long as the name of the list is the same on all sites).

Sharepoint list deployment

As we all know by now (unfortunately) deployment is one of the big "pain areas" in SharePoint 2007, specially the deployment of custom lists and document libraries. There are already several blog posts out there covering this topic but it seems that they all miss something (i.e. some cover out to deploy document libraries but not mention how to configure the library to use custom content types). The Microsoft documentation is, as usual, very thin on this area as well.

To cover this area I'm planing to post a blog called "One Stop Shop for creating Sharepoint libraries via features" that will cover:
1) How to deploy custom lists via features
2) How to deploy custom document libraries via features
3) How to use a custom content types in a list definition
4) How to deploy views.


In the mean time here are the pointers to some useful blog posts about custom lists deployment:

Creating a Custom Document Library Feature in WSS V3 / MOSS 2007
Add a custom list definition to SharePoint
Using a SharePoint 2007 content type in a List definition

SharePoint Applications – Design Lessons Learned (so far)

I can't believe that it has been 18 months since my last blog entry. The main reason for this absence is that I've been busy designing and implementing MOSS enterprise projects since June 2006 and, like they say, time flies when you are having fun (well sort of). I've decided to return mainly because I needed to create a list of "tricks" and lessons learned as a "reminder to self" so that I can apply them in future projects. If I can help other people on the way then even better.

In this blog post I'm going to highlight some of the design and architecture lessons that I've learned so far. In future posts I'll be drilling down in some of the areas and present concrete examples.

So here is my list so far:

1. Do not forget standard/proven design and architecture patterns just because you are using SharePoint. SharePoint should be seen as another layer in your technology stack that your code will interact with. But just because you are using this layer you should not forget to follow good and proven design practices. The following points describe some of the principles that sometimes seem to be forgotten when using SharePoint.
   
   1. Create a data access layer to access your lists. After all you would be doing this if you were programming against a database table so why not do the same for a SharePoint list. The last thing you want is to have several components of your application accessing the same list in several different ways. If the list schema, name or location changes then you would need to change the code in every component that uses that list (this could be acceptable for small proof of concept applications but not for enterprise applications). In a future post I'll be giving an example of designing and coding a simple SharePoint data access layer.
      
   2. Use SharePoint lists to store configuration data. Create a configuration site, on your site collection, containing all the configuration lists. Before SharePoint all configuration needed by you applications resided in a data store (normally a database). You then needed to build a UI to allow power users and/or administrators to change that information. You also had to implement security mechanisms to make sure that only the users with the correct permissions were allowed to change or access that data. With SharePoint you can accomplish all of that with out of the box functionality without writing a single line of code. You can also adopt the same principle for structured content. For example let's say that you want to create a very simple top navigation menu (like the one on the top right corner of this page) and allow your users to add new links or edit existing ones. You could create a TopNavs list on a "/config" site under your site collection with "Name" and "URL" columns. You could then create the menu using a simple ASP custom control and bind it to the TopNavs list. I'll be showing how to do exactly that in a future post.
      
   3. Do not forget about databases. Just because you are using SharePoint and have all of those nice lists that you can easily create and configure it doesn't mean that you should stop using databases (disregarding the fact that SharePoint is actually storing all of the list data on a database). You should always ask yourself the reasons why you should use a SharePoint list instead of a database table. Remember - SharePoint is a service layer - so if you are not using the services provided by that layer then perhaps you should be using a database table instead. SharePoint lists also have greater limitations than database tables (see this Microsoft document for more information on those limitations).
      
   4. Empower business users but try to keep them in their comfort zone. I know that this sounds like a sales cliché but it's actually one of the main SharePoint selling points. You should ear this sentence echoing in the back of your mind every time you are talking to a customer. This will help you making better decisions and keep your customers happy. Here is an example:
      
      1. You are responsible for the architecture and design of a brochureware site for a major financial institution. One of the main reasons for them to adopt SharePoint was to give more control to business users. You decided to base the site on the out of the box publishing site definition and you are just about to test your first content page (which is using the out of the box Rich HTML editing control).You look at the generated html code and suddenly freeze. Your heart starts pumping faster and you cannot believe on what your eyes are telling you; the HTML code generated by the control is appalling. Your professional pride kicks in and tells you that you cannot be responsible for code like this. What do you do?
         
         1. Talk to the customer about the problem and suggest replacing the control with a "professional quality" third party editing control (they would need to agree the extra costs).
            
         2. Tell the customer that you cannot be responsible about the quality of the code generated by the control and ask them to make a decision. You can point out that, although the quality of the generated code is far from ideal, it can be a viable option as long as they are not concerned with accessibility issues.
            
         3. Tell the customer to continue to use the control and teach them how to improve the code by showing them the control's HTML view.
            

      Option 3 is the least appealing one because it will require your business users to step out of their comfort zone and learn something that they shouldn't need to for their day to day jobs (unless they are technical minded and do not mind the learning curve).