Friday, April 10, 2009

Import user profiles using Active Directory / Business Data Catalog

It has been a while since I blogged, I was on a large enterprise project and was pretty busy until recently. I concluded the project at a client where we had a lot of user information in a legacy system that used SQL Server as the backend. We were building the new site on MOSS 2007 and the user profiles were needed in the SharePoint application - for searching and surfacing these user profiles for over 120,000 users. We were also implementing Active Directory as part of this release so we needed to bring all the user information into AD for authentication to the new MOSS site.

To implement this complex requirement, we broke the challenge into 3 tasks.

1. Moving the user accounts from the legacy database to AD to enable user authentication against AD.

Then we had to pull the same information into the SharePoint profile store. Here is how we broke this up.

2. We moved the user account information from AD to the SharePoint profiles using the built in SharePoint connector to AD. Though the user profile in the legacy system had a lot of fields, we only brought over a handful of fields to AD to minimize the impact to the Active Directory. 

SharePoint 2007 profiles have a concept of master and non-master connections. AD or a LDAP source is a master connection. BDC on the contrary, is a non-master connection. What this means that the BDC cannot be used to create a new profile record in the profile store. Only AD or a LDAP connection can do that. BDC can only supplement user information in the SharePoint profile store.

3. We supplemented the rest of the information from the custom legacy store to the SharePoint profile store using the Business Data Catalog in SharePoint. This allowed us to bring 50-60 additional fields over directly into the SharePoint profile store without needing to move them all to AD and bring them from there.

I was getting ready to write many more pages about the details of this solution, but I just noticed that my friend Todd Baginski also concluded a similar project a few weeks ago, so I am going to point you his way for all the details. You can view his post here.

The way my solution differed from Todd's is that we did not use this information in the MySites. In fact, we did not even enable MySite. The reason being that we were pulling a 120,000 users or so - and that would be nearing the 150,000 site collections per Web application upper limit recommendation on technet. We did contemplate splitting up the MySites in a couple of Web Apps, but the requirements for that user interface were also very different than what comes OOB in SharePoint so a custom page served us much better, instead of the overhead of having 120,000 site collections. Also we did not want to give users control of editing on the site.

The other notables to this piece were 
1. A handy script I wrote to create 70 profile attributes in the user profile store in SharePoint and also map those attributes to either the master connection on the BDC connection depending on the application file settings.

2. A wholly custom search implementation that used the SharePoint Enterprise Full Text search to search the user profiles and display each individual profile on the custom page that we developed and deployed to SharePoint.

I will document both these pieces shortly. Stay tuned!!