Johan's Web Portal

In the first part of this blog we looked at the theory and some examples. In this blog we will continue with configuring and managin the DAG.

In this case we have an environment which consists out of two multirole Exchange 2013 servers. Because this is not an odd amount we will need to use a file share witness. Ideally this fileshare witness is placed on another Exchange server (for example your CAS) but it can also be located on an generic server.

First step is to prepare our FSW. It is important the Exchange Trusted Subsystem is a member of the local administrators group of the server. This because Exchange uses this account under the hood to perform the needed actions.

If the server doesn’t contain the File Server feature add it by using the following Powershell cmdlet:

Add-WindowsFeature FS-FileServer

Once these actions have been performed we can continue with configuring our DAG.

If you are familiar with DAG’s from Exchange 2010 pay attention to the next step. You will need to pre-stage the Cluster Name Object (CNO). Open ADUC and create a computeraccount which matches the name you want to give to the DAG, for example DAG01. After creating the object disable the account and get the properties of the object. Select the security tab and add the computeraccount of the computer which will be the first member of the DAG. Repeat this last step for the Exchange Trusted Subsystem security group.

Of course you can also script the creation of this object. Fellow The UC Architects member Michel de Rooij has created a script which can be found here.

Next step is to create the DAG, we will use the Exchange Management Shell for this:

New-DatabaseAvailabilityGroup -Name Exchange_DAG01 -WitnessServer FS01 -WitnessDirectory C:\Exchange_DAG01  -DatabaseAvailabilityGroupIpAddresses 192.168.1.90

By using the cmdlet above we will create a DAG which has the name DAG01 and assign FS01 as the witness server. On the FS01 a directory will be created called Exchange_DAG01. As last step we will assign an IP address to the DAG 192.168.1.90.

Now we have created the DAG it’s time to add the mailbox servers to it. This can be done by using the Add-DatabaseAvailabilityGroupServer cmdlet:

Add-DatabaseAvailabilityGroupServer -Identity DAG01 -MailboxServer EX01

Remark: If the Windows Failover Clustering components are missing the cmdlet will install then automatically. Keep in mind that this might require a restart of your Exchange Server. The cmdlet will in that case only install the components and will not add the server to the DAG. So if the server is rebooted run the cmdlet again to add the server to the DAG. However it may not be required to reboot the server in that case the server will be added to the DAG directly.

Once the first server is completed repeat the same step for the other server:

Add-DatabaseAvailabilityGroupServer -Identity DAG01 -MailboxServer EX02

To finish the configuration of the DAG we will need to add the additional copies of the databases. For adding a copy you will need to use the Add-MailboxDatabaseCopy cmdlet:

Add-MailboxDatabaseCopy -Identity MBDB01 -MailboxServer EX02 -ActivationPreference 2

After the copy is added you will need to restart the Microsoft Exchange Information Store service on the target server. This has to do with the “Managed Store” as introduced in Exchange 2013. Collegue Exchange MVP Tony Redmond published a nice blog. So if you want to know the indept details I recommend to visit this page.

This will add a copy of the database called MBDB01 to mailboxserver EX02 and will set the activation preference to 2.

Repeat the step for the database hosted by EX02:

Add-MailboxDatabaseCopy -Identity MBDB02 -MailboxServer EX01 -ActivationPreference 2

Once the cmdlets have been ran you can use the Get-MailboxDatabaseCopyStatus cmdlet to verify the status of the databases and its copies:

In the example above the first copy is mounted and the content index state is healthy. However for the second copy of the database the content index is FailedAndSuspended. In case of a *over this results in users who will start to have issues when searching their mailbox.

To fix this issue we need to run the Update-MailboxDatabaseCopy cmdlet like this:

Update-MailboxDatabaseCopy -Identity MBDB01\EX02 –CatalogOnly

After the confirmation the catalog will be updated on the second copy:

Let’s look at another issue:

In this case the second copy failed completely so we do have to update both the database and the catalog:

Update-MailboxDatabaseCopy -Identity MBDB02\EX01

After confirming the reseed is performed and both the database and catalog should be healthy again.

If we want to perform maintenance on the DAG members we will need to put the member in maintenance mode. This can be done by using the StartDagServerMaintenance script which can be found in the default script directory of Exchange.

By adding the Server parameter we can specify a server which we would like to put in maintenance. Once this is done it will check if the PAM role and which databases are currently activated on the server and will try to move them to another DAG member.

Once we have performed some maintenance on EX02 we will need to take it back in production. This can be done by using the StopDagServerMaintenance script:

.\StopDagServerMaitenance.ps 1 –Server EX02

Once this script has been executed verify if the databases are healthy using the Get-MailboxDatabaseCopyStatus cmdlet. The script will not re-balance the databases so we will need to do this manually. To move the active database we need to run the Move-ActiveDatabase cmdlet:

Move-ActiveMailboxDatabase MBDB02 –ActivateOnServer EX02

After confirming the activation on the other server the database will be moved to EX02.

Here ends the second part of the Exchange Alphabet about Database Availability Groups. In this part we had a look how to create a DAG and perform several operations

In an earlier blog we had a brief look at the Database availability Group (DAG) of Exchange 2013. In this blog we will have a closer look how it works and how you can create it.

The DAG functionality was introduced in Exchange 2010 to replace all the other cluster methods Exchange 2007 contained. Using a DAG it is possible to create multiple copies of a database within the DAG. This is limited to 16 copies per database which I personally have never seen during implementations.

Database Copies can have several statuses which are displayed in the table below:

When looking under the hood of the DAG you will discover that is uses windows clustering. These components are automatically installed when adding a server to a DAG. One import remark that must be made is that this requires an enterprise edition when deployed on Windows 2008 R2.  For Windows 2012 is doesn’t matter anymore because the failover feature is available in both standard and datacenter. Although it uses the failover feature of the OS it is not recommend to make changes via the failover cluster manager.

Within the DAG one server has the primary active manager (PAM). This server will make the decisions which copies are active and passive. When an issue occurs with the active copy the PAM is responsible for getting the topology change notifications and reacting to the failure. The server which hosts the PAM is always the owner of the cluster quorum group. If the server hosting the PAM will fail it will automatically failover to one of the DAG members who has survived.

Besides the PAM role there is an additional role called the standalone active manager (SAM). This role is active on all servers within the DAG but on the server which contains the PAM role the PAM will perform the actions of the SAM. The SAM role is responsible for detecting local database and local Information Store failures and acting on actions initiated by the PAM.

As you may have noticed we talked about the quorum. This is pretty important to understand. If you are familiar with clustering this must sound familiar to you. Because the DAG is a kind of clustering it also uses quorum to prevent issues like split-brain. In a DAG there are two options for this. Either implement an odd amount of DAG members, for example 3. Or decide to implement a Failover Witness Share. In this last case a shared folder is created on another server which is not a part of the DAG. For example if you have two DAG members we will need to add the failover witness share to get quorum.

To calculate how many at least need to be online to we can use the following formula (n / 2) + 1 where n is the number of DAG nodes within the DAG. For example we have 3 DAG members which makes (3/2) +1=2.5. Since halves do not count we need to round this down to 2. This means that at least 2 members will need to be online to leave the DAG running.  Members can be either the mailbox servers or a mix of one mailbox server and the server containing the FSW. Another example, let’s say we have 6 DAG members which makes (6/2)+1=4 which is equal. In this case at least 4 members of the DAG will need to be online.

We already talked about the multiple copies of a database but how does this work? Well each copy of a database has an activation preference. The initial database gets an activation preference of 1. All additional copies will get a unique ID assigned. For example the second copy gets 2 as activation preference etc. This preference is used as one of the parameters when the PAM needs to select a copy which needs to be activated if a failover is initiated.

Some examples

OK enough theory let’s have a look at some examples. The easiest example is a DAG within a datacenter which contains 3 multirole Exchange 2013 servers. Each server contains a copy of each database. The preferences of the databases have been configured in a way that during normal operation each server will only host one active copy:

Let’s assume one of the nodes has an issue and would need to be taken down for maintenance. No problem just put the node in maintenance and the PAM will ensure the database will be moved to another server.

After the maintenance has been performed on the server just stop the maintenance. The other servers will see that the node is back and will ensure the copies hosted on the server will be brought up-to-date. A manual action however is needed to move the database originally active on this server.

In the next example we will have our DAG members spread across two different datacenters. Each datacenter is defined as an AD site. We are still having 3 multirole servers in our Exchange 2013 environment. The only difference with the first example is that two of them are located in the primary datacenter and the third one in the secondary datacenter. The second data center is used for disaster recovery purposes:

Let’s assume the WAN link goes down in this scenario. Think of the formula what do you think will happen? Right the users can access their mailbox. Let’s use the formula (3/2)+1 = 2.5 which can be rounded down to 2. Which means at least two mailbox servers of a combination of one FSW and one mailbox server should be online. But if the complete primary data center burns down we will need to follow the procedure as described next.

A DAG can be either created within one AD Site or across two AD sites. In this last case it is pretty important in which site you locate the file share witness (FSW). In most cases this will be the site which you want to survive which might be primary site. In this case if the WAN link between the two sites goes down the site containing the quorum will survive. To bring the DR site online again we will need to introduce an alternate FSW and then bring the DAG online again.

Here ends the first part of the blog about database availability groups. In this article we looked at the theory behind the DAG and looked at some example configurations. In part two we will start building a DAG.