Fully automated deployment of SharePoint farms in Microsoft Azure

There a plenty of blogs out there about scripting SharePoint, so I wont fill up more space about all the troubles and joys regarding that. I will how ever point you in the direction of AutoSpInstaller who have an awesome script for slip streaming SharePoint installations and better yet. Automating the deployment of SharePoint 2010 and 2013.

I’ve seen a few blog post about installing SharePoint in Microsoft Azure, and I wanted to add my two cent about that. I just finished up doing a Proof of Concept showing a FULLY automated installation of multiple both single and multiple server, SharePoint farm, inside Microsoft Azure. This is how I did it.

I’m developing a Provisioning Engine that supports running in many sites and distribute servers and services across these sites seamlessly. I order to start facilitating Azure we need a subscription and some Persistent VMs deployed inside that subscription. I originally wanted to do everything “offsite” from each subscription, so a new customer could start from scratch, but after a while it became clear I needed some more granulated control doing deployment and needed to deploy my provisioning engine inside the subscription. Microsoft does not have an API for creating subscriptions (that I know off) so we manually sign up for a new Azure Subscription. Next we sign up for a Trial Account in our Provisioning Site. Create a certificate and upload it to Azure and Add Subscription ID and certificate to my Provisioning API in order to make the provisioning engine able to facilitate Azure for that subscription.

We do that by adding a Provisioning Site,

Next, we need to make sure all servers can see each other. We can install everything on one server. We can add all servers to the same Azure Hosted Service. We can “link” different roles to a Azure Hosted Service (I have not tested this) or we can create a Virtual Network. The latter is most appealing, so we add a Workspace

( at the time of writing, I have not automated the creating of the Virtual Network inside Azure, so we need to add it manually inside Azure and fill out the values in the Workspace plan )

Once this is done, manually Add a server to Azure, and deploy our provisioning engine. (this could later be a custom image, or pushed from another site, but for proof of concept, doing this manually  is ok). We link the server to the provisioning site, so the new server have access to all services, and we are now ready to automated everything else.

Next we add an Microsoft SQL server. This can also be done while adding the SharePoint Farm

Let customize the SharePoint installation so we control Server names etc. Create a Hosted Service for our new SharePoint Farm

Next add 1 or more servers depending on how big you want the farm to be

Make sure to update relation to point to your new Azure Hosted Service if you got more than one. Lastly we need to add a SharePoint farm on top of the Hosted Service.

Fill out desired SMTP/Email settings if needed, and hit deploy. This will automatically add a new Hosted Service and add an SQL Server inside that Service. It will add a Plan for each of the (up to) 22 (default is 16) databases  needed to run the farm, once all have been setup it will deploy SharePoint to each of the servers you added in the above steps, it will create a new Farm, a default SharePoint site and My Site. Configure most of the normal features of a SharePoint farm ( but any features supported by AutoSpInstaller is supported here too)

So what happens behind the scene? Well, AutoSpInstaller supports deploying SharePoint remotely on one or more servers, but it uses PS Exec from PS Tools to do it. ( if you look in the PowerShell scripts it looks like the author also tried making it work using WSMan ( invoke-computer / new-pssession )) but a new blank server doesn’t allow connect with either, so we need a way to get in contact with the server in the first place. If you deploy the standard Windows 2003 image, it allows you to talk with it, using WMI but not WSMan, so you could fire off a WMI command to run the command to configure WSMan ( winrm quickconfig ) but if you deploy a Windows 2012 server, your just out of luck, neither WMI or WSMan will work. You can how ever, tell Windows Azure to automatically join new servers to a domain, and when we are talking about SharePoint you need the servers to be domain joined. Once a Windows server 2012 is part of a domain, you can connect using WSMan and execute PowerShell on the machine (but not WMI, hence PS Exec will not work )

Sure, you could just fire up all servers to be part of a domain, create a group policy to configure the firewall (and maybe there is a Policy for doing something like a winrm quickconfig  too?) but I wanted something generic. Granted, I never figured out a way to “cheat” a fresh Windows 2012 that is not a domain member, I guess I will need to create a custom Image for those later, but for now, having them domain joined was good enough.

On all servers I deploy, I automatically I push out a small windows service. It handles certificates, joining/disjoining domains, installing software, remote management and is also used on Remote Desktop and Citrix XenApp servers to manage user sessions. Its perfect for handling the SharePoint installation and with above I can now push out this service though WMI and/or WSMan and kick of the installation. So the service handles installing an SQL server. well tested and Known technology and easy to automate, next we can now kick off the SharePoint installation.

AutoSpInstaller uses Start-Transcript doing installation, and I haven't figured out how to “support” that in a custom hosted PowerShell host, so first I had to strip that out. Next it does an insane amount of “am waiting lets do a write a “.” dot each second” thing, and is annoying as hell in the console logs, so had to strip those out too. Lastly you need patience … A LOT of it … So now when looking at the Provisioning Request Log is more clean.

For some reason the first server you install, that will also handle setting up the farm, and configuring services, will take from 2½ to 4 hours depending on the amount of services you choose. The following servers can take up to 30 minutes to install binaries, and 20 to 40 minutes joining the farm and configuring services. Not sure why really (both CPU/memory and Disk queue length is way below any warning signs on all servers doing the installation)

Once AutoSpInstaller is done, its easy and well documented how to connect to one of the servers and configuring new SharePoint sites/Site Collections and adding/removing users and groups, as part of my provisioning engine.

When using AutoSpInstaller it worth taking 2 minute extra reading the comments in the AutoSPInstallerInput.xml file. Stuff like how database prefix's, limits on characters' in password, or knowing that what worked on your “single” server install will not work in farms ( using the default http://localhost as WebApp name’s or not limiting the QueryServer to a single server will take a lot longer to troubleshoot that just reading the damn documentation )