Following our last article, introducing the new Exchange Administration Center (EAC), we follow up on how the new EAC makes it really easy to manage multi-forest Exchange deployments.

As described in Deploy Exchange 2010 in a Cross-Forest Topology, Exchange Server 2010 can be deployed in a cross-forest or resource forest topology. For the purpose of this article, we will employ the same approach to deploy a resource forest topology with The New Exchange.

First, let’s setup a resource forest environment as shown below, with a one-way forest trust setup so that Forest B trusts Forest A.

Figure 1: Layout of a resource-forest Exchange topology

Forest A is the account-forest, where all user accounts are provisioned. Forest A does not have any Exchange servers installed. Forest B is the resource forest – with Exchange Server 2013 installed and with linked mailboxes for each of Forest A’s user accounts. Forest B has no user accounts provisioned.

In a lab environment, we have a user, John Doe, setup in Forest A. We now want John Doe to be an Exchange administrator. To do this, we need to ensure that he has the appropriate Role-Based Access Control (RBAC) privileges in place in the Exchange resource forest. Since we are dealing with a multi-forest environment here, there are two ways of configuring RBAC, and we will try both:

1. Using linked mailboxes
2. Using linked role groups

You may already have users setup in an account forest. Configuring these users as linked mailboxes establishes their presence in the Exchange resource forest. By adding these linked mailboxes to existing RBAC role groups, the associated users thus effectively become Exchange administrators.

Using linked mailboxes

In the figure below, John Doe’s user account is setup in the accounts forest. Using EAC, an Exchange administrator can then create a linked mailbox for him, as shown below.

Figure 2: John Doe has been setup as a linked mailbox.

John’s mailbox can then be added to any of the built-in RBAC administrative role groups, thus making him an Exchange administrator. In the screenshot below, John has been made a member of the built-in Organization Management role group.

Figure 3: John Doe has been added as a member of the Organization Management role group.

Now, let’s try logging into EAC as John himself. John should now have Exchange administrative privileges, and that is what we see below.

Figure 4: Logging into EAC as John Doe, configured as a linked mailbox, and as a member of the Organization Management role group.

Using linked role groups

Linked role groups are essentially role groups, but linked via a Universal Security Group (USG) across a forest boundary. Thus members of this USG effectively become members of the linked role groups. This means that if John Doe is a member of such a USG, he automatically gets all RBAC permissions in an Exchange resource forest via the linked role groups. Linked role groups thus make it possible for a small set of USGs in one account forest to manage complex, multiple Exchange forest topologies.

To demonstrate the above, we will do the following:

1. In the account forest, using Active Directory Users and Computers (ADUC) management tool, we will create a USG called AdminSG. AdminSG will represent a group of Exchange administrators. We will then make John Doe a member of AdminSG
2. In the resource forest, we will setup a single linked role group, based on the built-in role group “Organization Management”, mapped to AdminSG, using the following Exchange Management Shell calls:

$accountForestCredential = Get-Credential

$OrgMgmt = Get-RoleGroup "Organization Management"

New-RoleGroup "Organization Management - Linked" -LinkedForeignGroup "AdminSG" -LinkedDomainController bjex062.extest.microsoft.com -LinkedCredential $accountForestCredential -Roles $OrgMgmt.Roles

Get-ManagementRoleAssignment -RoleAssignee "Organization Management - Linked" -Role My* | Remove-ManagementRoleAssignment

Get-ManagementRole | New-ManagementRoleAssignment -SecurityGroup "Organization Management - Linked" -Delegating

With the above steps, John Doe now has all RBAC privileges in the Exchange resource forest even though his user account is in the account forest.

We will now attempt to login to EAC as John Doe. We launch EAC from any desktop, and point it to a CAS server in the resource forest, and we are logged in. John has standard RBAC privileges as defined in the Organization Management RBAC role group and is able to manage Exchange in the resource forest accordingly.

Figure 5: Logging into EAC as John Doe, via the linked Organization Management role group.

Try EAC!

As you can see, EAC makes it a snap to manage multi-forest Exchange deployments. You don’t need to remote into dedicated management machines to manage a specific Exchange forest, nor do you need to create special administrative accounts in every Exchange resource forest. You can do all this right from your own desktop with EAC, with your existing user credentials!

EAC is available in the latest Exchange 2013 Preview download bits as well as via the Office 365 Customer Preview offering. We will continue to write more about EAC in the upcoming weeks. In the meantime, do try the new EAC, and give us your feedback!

The Exchange Manageability Team

The changes in the updated Exchange 2010 Monitoring Management Pack include:

• Fixes to Alerts – at times two events were logged for the same event.
• Ensure Correct Values are read.
• Log the event on local server where the TS is running.

Here is a location of the updated Management Pack: http://www.microsoft.com/en-us/download/details.aspx?id=692 (short: http://aka.ms/ex2010mp)

Note: When you upgrade from the previous Management Pack and you have a Language Pack installed there is no need to update the Language Pack. If you want to update the Language Pack you need to uninstall the previous Language Pack and install the new one.

Thanks,

Exchange Team

A question that has come up several times in recent months is: How do I create RBAC roles that present only very limited ActiveSync management functionality?

Before we dive into the answer let us do a quick review of what RBAC (Role Based Access Control) is.

Prior to Exchange 2010 permissions were defined through tools like DSACLS and ADSIEdit. This let you specify what objects a user or group could touch and what they could do to the object as a whole. If a user needed write access to one specific property of an object, but not the other properties, there was no easy way to handle this. RBAC does not define permissions on the object; instead it defines permissions on the PowerShell cmdlets that can modify the object. PowerShell cmdlets get added to a role and a user or group is assigned to the role. If the cmdlet and parameters you need are part of a role you participate in, then you will be able to run the cmdlet.

In the Exchange Management Shell you can run (get-excommand).count to see how many Exchange cmdlets you currently have access to. In the Exchange Control Panel (ECP) and the Exchange Management Console the cmdlets you have access to determine what options are displayed. Therefore if a window in either GUI requires you have specific PowerShell cmdlets in your role, but you do not have them one of two results is possible:

• The window will not be displayed (in fact options that lead to it are unlikely to be offered)
• The window will display, but all content will be disabled (this is typical if you have the relevant Get- cmdlets, but lack one or more of Set, New, Add, etc.)

For more detail around RBAC, please start with the following:

• RBAC and the Triangle of Power http://blogs.technet.com/b/exchange/archive/2009/11/16/3408825.aspx
• Understanding Role Based Access Control http://technet.microsoft.com/en-us/library/dd298183.aspx

Exchange 2010 SP2 contains 71 RBAC roles. However, none of them offer limited subsets of ActiveSync commands that can be assigned to help desk staff. To create such roles you have to tackle them yourself. If your helpdesk works with PowerShell this is relatively straightforward because you can create a role that has only the PowerShell cmdlet and parameters they require. If the users operate exclusively through the ECP this process is more complex, because the user must be able to navigate to the point where the operation is conducted.

In this example the customer needed helpdesk staff to be able to wipe ActiveSync devices through the Exchange Control Panel. The problem was that only members of Organization Management were able to navigate to the proper window in the ECP and carry out a device wipe. Adding the HelpDesk users to the Organization Management role group was out of the question – it simply has too many rights.

The Administrator tried creating an RBAC role that contained only the Clear-ActiveSyncDevice cmdlet. The problem was that the new role did not allow them to access the functionality through the ECP. The ECP did not allow the users to navigate to the place where the “Wipe Device” button is displayed because the button is nested a few levels down. The user needs to be able to list the users in the Organization, see their Phone and Voice properties and be able to open the dialogs along the way. A role that contains only the Clear-ActiveSyncDevice cmdlet does not include the other cmdlets that are required by these other steps in the ECP. What we have to do from here is figure out what we have to add to this role so we can navigate to the window and click the “Wipe Device” button.

The first step in this process is to look at what roles the Organization Management role group is comprised of. To do this we run:

[PS] C:\>$FormatEnumerationLimit=999
[PS] C:\>get-rolegroup "organization management" | fl roles

Roles : {Active Directory Permissions, Address Lists, ApplicationImpersonation, Audit Logs, Cmdlet Extension Agents, Database Availability Groups, Database Copies, Databases, Disaster Recovery, Distribution Groups, Edge Subscriptions, E-Mail Address Policies, Exchange Connectors, Exchange Server Certificates, Exchange Servers, Exchange Virtual Directories, Federated Sharing, Information Rights Management, Journaling, Legal Hold, Mail Enabled Public Folders, Mail Recipient Creation, Mail Recipients, Mail Tips, Mailbox Import Export, Mailbox Search, Message Tracking, Migration, Monitoring, Move Mailboxes, Organization Client Access, Organization Configuration, Organization Transport Settings, POP3 And IMAP4 Protocols, Public Folder Replication, Public Folders, Receive Connectors, Recipient Policies, Remote and Accepted Domains, Retention Management, Role Management, Security Group Creation and Membership, Send Connectors, Support Diagnostics, Transport Agents, Transport Hygiene, Transport Queues, Transport Rules, UM Mailboxes, UM Prompts, UnScoped Role Management, Unified Messaging, User Options, View-Only Audit Logs, View-Only Configuration, View-Only Recipients, MyBaseOptions, MyContactInformation, MyDiagnostics, MyDistributionGroupMembership, MyDistributionGroups, MyProfileInformation, RetentionPolicies, MyTextMessaging, MyVoiceMail, MyMailboxDelegation}

From here we need to discard roles that are not going to be directly related to wiping a device. Some of the roles like “Remote and Accepted Domains” are obviously not going to contain related cmdlets. For the potentially interesting Roles our next step is to find out what they contain. To do that we run a cmdlet like this one:

[PS] C:\storage>Get-ManagementRoleEntry "Mail Recipients\*"

Name Role Parameters
---- ---- ----------
Write-AdminAuditLog Mail Recipients {Comment, Confirm, Debug, DomainController, ErrorAction, Er...
Update-Recipient Mail Recipients {Confirm, Credential, Debug, DomainController, ErrorAction,...
Test-MAPIConnectivity Mail Recipients {Archive, Confirm, Debug, ErrorAction, ErrorVariable, Ident...
Set-User Mail Recipients {AssistantName, CertificateSubject, City, Company, Confirm,...

I have cut the output short here and included only a few lines as a demonstration (the actual cmdlet produces 96 results).

From here the next step is to create new child roles of the roles that we believe are useful for our work. Here is an example:

new-managementrole -parent "Mail Recipients" -name StrictlyRecipActiveSyncDeviceWipe

The role we just created is a child of Mail Recipients, thus it has all the permissions of its parent. Why not use the Mail Recipients role itself? As we progress I intend to remove pieces from this role so that we can achieve the lowest level of permissions possible. Never modify the built-in roles like Mail Recipients because it will break functionality. Another point to raise: a child of a role can only be assigned the functionality of its parent; it cannot be given any rights or access to functionality that is not already included in the parent.

Once we have created our role we need to create a Role Group. This will make it easier to assign the role to users in the future. For our testing we are using an account called WipeTest. Once testing is complete, replace WipeTest with the account(s) or group(s) desired.

New-RoleGroup -Name "OnlyActiveSyncDeviceWipe" -Roles "StrictlyRecipActiveSyncDeviceWipe " -members WipeTest

WipeTest can log into OWA and access the ECP. We find that we can navigate most of the way to the option we want in the ECP, but not all the way. This indicates an additional role is required.

If you look in the Exchange directory (by default c:\Program Files\Microsoft\Exchange) you will find the following path “ClientAccess\ecp\PhoneVoice”. Looking through the files in here will give you some clues as to what you might want to add next. The Web.Config file has numerous mentions of OrganizationConfig, but that isn’t specific enough. We can look through the folder to see if any of the files mention specific cmdlets we might be able to use and that are not in the role we have already defined. One that stands out is Set-CasMailbox (it is in QuarantinedDevices.ASCX). We then checked on where that cmdlet resides:

[PS] C:\>Get-ManagementRoleEntry "*\set-casmailbox" | fl name,role,parameters

Name : Set-CASMailbox
Role : Mail Recipients
Parameters : {ActiveSyncDebugLogging, ActiveSyncEnabled, ActiveSyncMailboxPolicy, Confirm, Debug, DisplayName, DomainController, ECPEnabled, EmailAddresses, ErrorAction, ErrorVariable, EwsAllowEntourage, EwsAllowList, EwsAllowMacOutlook, EwsAllowOutlook, EwsApplicationAccessPolicy, EwsBlockList, EwsEnabled, HasActiveSyncDevicePartnership, Identity, IgnoreDefaultScope, ImapEnabled, ImapEnableExactRFC822Size, ImapMessagesRetrievalMimeFormat, ImapSuppressReadReceipt, ImapUseProtocolDefaults, MAPIBlockOutlookNonCachedMode, MAPIBlockOutlookRpcHttp, MAPIBlockOutlookVersions, MAPIEnabled, Name, OutBuffer, OutVariable, OWAEnabled, OwaMailboxPolicy, PopEnabled, PopEnableExactRFC822Size, PopMessagesRetrievalMimeFormat, PopSuppressReadReceipt, PopUseProtocolDefaults, PrimarySmtpAddress, SamAccountName, ShowGalAsDefaultView, Verbose, WarningAction, WarningVariable, WhatIf}

Name : Set-CASMailbox
Role : Organization Client Access
Parameters : {ActiveSyncAllowedDeviceIDs, ActiveSyncBlockedDeviceIDs, Confirm, Debug, DomainController, ErrorAction, ErrorVariable, Identity, OutBuffer, OutVariable, Verbose, WarningAction, WarningVariable, WhatIf}

Name : Set-CASMailbox
Role : User Options
Parameters : {ActiveSyncDebugLogging, Confirm, ErrorAction, ErrorVariable, Identity, ImapMessagesRetrievalMimeFormat, ImapSuppressReadReceipt, ImapUseProtocolDefaults, OutBuffer, OutVariable, PopMessagesRetrievalMimeFormat, PopSuppressReadReceipt, PopUseProtocolDefaults, ShowGalAsDefaultView, WarningAction, WarningVariable, WhatIf}

Name : Set-CASMailbox
Role : MyBaseOptions
Parameters : {ActiveSyncDebugLogging, Confirm, ErrorAction, ErrorVariable, Identity, ImapMessagesRetrievalMimeFormat, ImapSuppressReadReceipt, ImapUseProtocolDefaults, OutBuffer, OutVariable, PopMessagesRetrievalMimeFormat, PopSuppressReadReceipt, PopUseProtocolDefaults, ShowGalAsDefaultView, WarningAction, WarningVariable, WhatIf}

It is interesting to note the parameter differences. We can exclude MyBaseOptions and UserOptions because their parameters would not help us wipe a device. We have already tried a child role of Mail Recipients and that role did not work. That leaves “Organization Client Access” as a role we could try using. You will notice it has access to the ActiveSyncAllowedDeviceIDs and ActiveSyncBlockedDeviceIDs parameters and they are not present in the Mail Recipients role.

Since we have seen Set-CasMailbox listed in the files we can try creating a new child role of “Organization Client Access” and strip out everything except Set-CasMailbox.

new-managementrole -parent "organization client access" -name OrgClientAccessWipeDeviceOnly
get-managementroleentry "OrgClientAccessWipeDeviceOnly\*" |where{$_.name -notlike "*set-casm*"}| Remove-ManagementRoleEntry

Now we add this new role to the Role Group we created earlier:

New-ManagementRoleAssignment -Name "OCA Child ActiveSyncDevice Wipe" -SecurityGroup "OnlyActiveSyncDeviceWipe" -Role OrgClientAccessWipeDeviceOnly

Now when WipeTest tries to wipe the device they can access the button in the ECP. One problem remains: StrictlyRecipActiveSyncDeviceWipe (a child of “Mail Recipients”) still has too many rights. From here we remove most of the cmdlets from StrictlyRecipActiveSyncDeviceWipe using the following cmdlet:

get-managementroleentry "StrictlyRecipActiveSyncDeviceWipe\*" |where{$_.name -notlike "*activesync*"}| Remove-ManagementRoleEntry

The next stage involves adding cmdlets back in to the role one at a time until we can navigate to the proper location in the ECP and conduct a device wipe. The routine is simple: add a cmdlet, try the operation. If it fails log out of OWA, add another cmdlet and try again. When we get it to work we can either stop or go back and remove individual cmdlets to try and achieve a minimum level of permissions. Below are some examples of the cmdlets we can use.

To add a cmdlet back in we need to do this:

add-ManagementRoleEntry "Mail Recipients\get-mailbox" -role StrictlyRecipActiveSyncDeviceWipe

/Aside

If you wanted to put back multiple cmdlets you can use the syntax from the example below:

Get-ManagementRoleEntry "Mail Recipients\*" | where{$_.name -like "get-m*"} | add-ManagementRoleEntry -role StrictlyRecipActiveSyncDeviceWipe

In this blog I only want to insert the cmdlets one at a time, but I included this for illustration.

/End Aside

If we want to remove an individual cmdlet from the role we can use this:

Remove-ManagementRoleEntry "StrictlyRecipActiveSyncDeviceWipe\Get-Mailbox"

(When the “Are You Sure?” prompt appears, select Yes or Yes to All)

In this blog I have included many of the details regarding how I journeyed to the final set of cmdlets. I have left out most of the tedious trial and error, but hopefully included enough detail you can go through this process yourself.

Below I have pasted in a summary of all the cmdlets for those that want to see everything together in a single, concise presentation:

new-managementrole -parent "Mail Recipients" -name StrictlyRecipActiveSyncDeviceWipe
new-managementrole -parent "organization client access" -name OrgClientAccessWipeDeviceOnly
get-managementroleentry "OrgClientAccessWipeDeviceOnly\*" |where{$_.name -notlike "*set-casm*"}| Remove-ManagementRoleEntry

(When the “Are you sure?" prompt is presented, select A)

get-managementroleentry "StrictlyRecipActiveSyncDeviceWipe\*" |where{$_.name -notlike "*activesync*"}| Remove-ManagementRoleEntry

(When the “Are you sure?” prompt is presented, select A)

add-ManagementRoleEntry "Mail Recipients\get-mailbox" -role StrictlyRecipActiveSyncDeviceWipe
add-ManagementRoleEntry "Mail Recipients\get-user" -role StrictlyRecipActiveSyncDeviceWipe
add-ManagementRoleEntry "Mail Recipients\get-recipient" -role StrictlyRecipActiveSyncDeviceWipe
add-ManagementRoleEntry "Mail Recipients\get-casmailbox" -role StrictlyRecipActiveSyncDeviceWipe
New-RoleGroup -Name "OnlyActiveSyncDeviceWipe" -Roles StrictlyRecipActiveSyncDeviceWipe,OrgClientAccessWipeDeviceOnly -members WipeTest

Once these cmdlets are run the WipeTest user (or the user or group you specify) will be able to open OWA, navigate to the ECP and open each of the dialogs needed to reach the point where they can wipe a device.

Thanks to Matt Byrd and Brad Hughes as contributors as I was writing this post.

Chris Pollitt

A brief history of hybrid management

The release of Exchange Server 2010 brought with it a revolutionary new Cloud based service which we all know today as Exchange Online for Office 365. In the early days of the Cloud lots of time and energy was devoted to rapid migration of user and organizational data with some focus on the deployment and support of coexistence between an on-premises Exchange enterprise deployment with a web-based tenant. However, for larger organizations (LORGs) the strong feedback was that in addition to data migration efficiency there is a great need to richly support a longer state of on-premises and Cloud coexistence. The LORGs have spoken and they’ve asked for a full-fidelity, no-compromises management experience for mailboxes based on-premises and in the Cloud.

Exchange Server 2010 was the first to deliver an on-premises plus Exchange Online cross-premises coexistence solution by extending the existing functionality provided by the Exchange Management Console (EMC). EMC is a Windows MMC based client originally geared toward managing large scale deployments of enterprise servers which later evolved into the de facto Exchange Server console that we refer to today simply as “hybrid.” EMC facilitated efficient data migration to the Cloud (Office 365), provided cross-premises recipient management including bulk editing and facilitated the management of most organizational level policies and objects.

EMC is typically used on individual x64 servers hosting Exchange Server roles or separately on workstations via the “Management tools only” installation. It’s a Windows only management tool which means it depends upon local services like WinRM to communicate with remote servers via the PowerShell protocol.

Figure 1: Hybrid Management with Exchange Server 2010

For hybrid management, EMC provides admins the ability to add a second “tree” to the console pane in order to view all recipients, perform mailbox migrations and manage organizational objects related to the Exchange Online tenant. A purely Exchange Online only deployment does not need to apply EMC for management, instead the simplified “Exchange Control Panel” (ECP) console is used which was also new in Exchange Server 2010.

So, what is hybrid?

Hybrid should not be imagined as a unique offering of an Exchange server itself, but rather as a state of coexistence which implies an on-premises server is interacting in cooperation with an Internet-based service. The concept of hybrid is not unique to the Exchange product line and may also be found across the wider family of Microsoft Office servers such as SharePoint and Lync.

In the case of Exchange hybrid, typically it implies a set of mailboxes and policies are distributed across on-premises and Office 365 to act as one “virtual” organization. In previous blog entries on hybrid setup and deployment, you learned that from the perspective of the two premises this state of coexistence is viewed as an internal deployment e.g. certificates are added automatically during cross-premises mail flow from Office 365 tenants to on-premises recipients resulting in the delivered mail being trusted as having originated from within the organization itself although actually arriving via the Internet using dedicated hybrid mail connectors.

There are many possibilities and permutations of how to organize recipients across on-premises and the Office 365 service to suit the needs of your organization. For example, one organization may plan to move 100% of their on-premises recipients to the tenant within two years, another will opt to add all their resource room mailboxes to their Office 365 tenant immediately while yet another chooses to use their online tenant to securely host their online archive mailboxes. The point is we’re delivering flexibility with hybrid designed to meet the needs of your business.

Introducing hybrid management for the new Exchange

This blog post picks up where “Cloud on your terms – Part I” left off by focusing primarily on the “steady state” of hybrid management particularly around the most common scenarios.

For the purposes of this post we’ll assume these pre-requisites have already been satisfied:

• At least one Exchange Server 2013 Client Access Server (CAS15) and Mailbox Server (MBX15) role has been installed. One of each type separately or both residing on a single server e.g. in an interoperability environment featuring a previous edition of Exchange Server.
• The Office 365 tenant version must be 15.0.000.0 or greater to configure a hybrid deployment with Exchange Server 2013. Consult the Office 365 site for latest details on licensing and pricing plans.
• You’ve enabled coexistence via the Office 365 admin portal and performed all other pre-requisite steps as noted including proving coexistence domain ownership.
• You have access to the “tenant administrator” Microsoft Account credentials which are necessary to access the Office 365 tenant.
• All on-premises dependencies are in place e.g. Active Directory Synchronization installed.
• The Hybrid Configuration wizard has been run successfully. This may have included upgrading your previous Hybrid Configuration settings if you were already using hybrid with Exchange 2010 and Office 365.
• You are not using the built-in “administrator” recipient account. Read on for details on why this isn’t supported for hybrid management.

Figure 2: Hybrid Configuration wizard for the new Exchange as reviewed in a previous posting from Ben Appleby

The new Exchange Administration Center hybrid console

To start things off here’s an annotated guide to the new hybrid management console via Exchange Administrator Center (EAC) for the new Exchange:

A) “ENTERPRISE” and “OFFICE 365” pivots – use these to toggle between your on-premises deployment and your online Office 365 tenant

B) A single consolidated central list where all your notifications will appear regardless of where they originated or which pivot you’re currently using e.g. for tracking mailbox migrations from on-premises to Office 365.

C) A single list view containing all recipients from both premises

D) “Details pane” for remote (Office 365) hosted mailboxes

E) Mailbox migration entry point via navigation tab

What’s new in hybrid management for Exchange

The philosophy of hybrid for the new Exchange is dead simple: to provide you, the administrator, with a single familiar console that you can use from nearly anywhere to manage your all-up cross-premises organization.

Here’s a short list of what’s new:

1) We’re taking advantage of having the new full featured browser-based console for Exchange administrators, which means lower maintenance costs required to keep hybrid “Management Tools” installations up-to-date. Updating just your Exchange Server 2013 Mailbox servers will keep all your EAC admins up-to-date.

2) Depending upon the security configuration of the ECP (the protocol name for EAC) IIS virtual directory on the Mailbox servers you can choose to allow both external and internal administrator access or exclusively internal for domain joined machines.

3) From one browser tab you can control of all your recipients and organizational objects (e.g. address lists and policies).

4) A single consolidated set of Exchange Notifications across all premises.

5) Support for Single Sign-On via ADFS 2.0 – there will be future topics presented soon devoted to deploying and managing Single Sign On. More information on preparing to use Single Sign-On is available on Office 365.

Figure 4: ADFS Single Sign-On module for Hybrid mode

6) Manage “Another User …” cross-premises – to perform a help-desk scenario such as setting the out-of-office message on behalf of another user on vacation simply use the “Another user …” option next to your Display Name in the upper right hand corner of the console to view a complete list of recipients across all premises.

Figure 5: Managing "Another user  ..." merged Recipients view

Start using hybrid management mode for Exchange Server 2013

If you’ve already run the Hybrid Configuration wizard (HCW) or the Update-HybridConfiguration cmdlet directly in PowerShell then you are already using EAC’s hybrid mode. In fact, from the “Hybrid” tab in EAC once you click “enable” you’ll be asked for your Microsoft Account credentials and after your tenant is found you’ll be sent right back to EAC but this time running in hybrid mode.

Later on, there’s nothing special you need to do to enter hybrid mode after running the HCW successfully. This is because in addition to enabling key scenarios like mail flow and data sharing (aka free/busy) services cross-premises the wizard creates artifacts on-premises that when present will automatically enable hybrid mode for EAC. Specifically, Update-HybridConfiguration (via HCW) will create a special Remote-Domain object which, when EAC detects a –TargetDeliveryDomain (TDD) property, will start EAC hybrid mode automatically.

One of the biggest difference you’ll notice when using hybrid mode is that when you click the “Office 365” tab it will prompt you to sign into your online tenant via either your Microsoft Account or ADFS credentials. Note that for performance reasons EAC caches whether to use hybrid mode to avoid checking at every logon (the cache state is automatically refreshed every 30 minutes). If you’ve manually created a Remote Domain with a TDD and immediately need to start using hybrid mode you should restart IIS on your Exchange Server 2013 Mailbox servers.

Figure 6: Hybrid Configuration wizard entry point

Managing hybrid in an on-premises interoperability deployment

There are few fine points to consider when managing Hybrid in an on-premises interoperability environment where Exchange 2010 and/or Exchange 2007 servers exist.

In an interoperability deployment, be aware that there’s added functionality you should use with the URI you’re using to access your deployment when logging on if your admin mailbox isn’t on the new Exchange. These URI keys will not be added by default for you in most cases, but stay tuned for more information on future releases which may feature pre-built links. We highly recommend bookmarking URIs with any needed key/value pairings for easy reference.

Reminder that the use of the built-in “administrator” account in Exchange Server should not be used with hybrid management:

Managing your recipients in hybrid mode

A complete list of all recipients is available in the “ENTERPRISE” pivot under the “mailboxes” tab. There are few items you should be aware of when creating and managing new recipients in hybrid mode.

The simple rule to follow when provisioning or modifying any recipient with hybrid is to always use the on-premises “ENTERPRISE” side. This is due to mostly one-way synchronization nature of MSO Directory Synchronization services and ensures that both on-premises and the tenant have the same copies of all recipient Active Directory details.

Figure 7: On-premises mailboxes noted as Mail Users in an Office 365 tenant

Provisioning new mailboxes in Office 365

To provision a new Office 365 mailbox in hybrid mode begin in the on-premises “mailbox” tab then select the “Office 365” mailbox type in the dropdown list from the new icon. Creating a new mailbox in your tenant may impact your available client licenses – view available licenses and plans from the Office 365 portal using your Microsoft Account credentials.

Figure 8: Creating an Office 365 mailbox

You’ll notice that on the “Office 365” service side there is no option to create a mailbox from EAC in hybrid mode. This ensures that all new mailboxes are provisioned from the on-premises side for complete recipient copies. It may be possible for you to directly provision a new Office 365 mailbox from the Office 365 Portal directly but this should not be used in Hybrid deployments since this mailbox will not be “back-synced” from Office 365 to on-premises and mail flow problems may occur.

Modifying recipients is also not recommend or permitted in hybrid mode from the “Office 365” side as this would result in the recipient being out-of-date on one side of the cross-premises deployment. In fact, this operation is blocked by Role-Based Access Control (RBAC) runtime validation rules to prevent divergence of copies (see the error message below).

Figure 9: Edit to recipient in service side blocked to avoid divergence

Much more to come soon!

We hope that you’re as excited as we are about the new hybrid mode for the Exchange Administration Center! Look for more articles covering topics such as Migration, Single Sign-On via ADFS, and debugging for hybrid soon.

Warren Johnson

You may have seen the introduction to the new Exchange Administration Center (EAC). The EAC is a unified Web-based portal for both on-premises and online Exchange deployments. Managing high availability (HA) is one of the key scenarios for on-premises customers, and the EAC delivers a brand new experience of managing HA. With EAC, the HA management tools are put together with a new modern look and feel.

Managing Exchange HA involves different operations like database switchovers, server switchovers, adding database copies, reseeding, etc. In previous versions of Exchange, there were UI gaps in the management consoles that required you to use both the console and the shell for some management tasks. For example, configuring lagged database copies. In previous versions of Exchange, you had to create a lagged database copy using the shell. In Exchange 2013, you can do this using EAC.

When you use the EAC to manage an on-premises environment, you will see a feature pane called “Servers.” This is where the Mailbox server-related HA features are managed. An example of this is shown below in Figure 1.

Figure 1. Where to Manage HA with the EAC

In this feature area, you will see 5 tabs (servers, databases, database availability groups, virtual directories, and certificates). The first 3 tabs are used to manage mailbox server-related HA features.

Database Availability Group creation and configuration

Let’s start by setting up a new DAG. As shown in the figure below, you can quickly create a DAG using the EAC.

Figure 2: new Database Availability Group

Then you can add Mailbox servers to the DAG, as shown in Figure 3.

Figure 3: managing DAG membership

Database and database copies management

Now it’s time for you to switch to database management to configure mailbox databases and deploy database copies on DAG members.

Continuing from where we left off, we switch to the databases tab. As you can see, there is an option called “Add database copy”, as shown in the figure below.

Figure 4: add mailbox database copies

All database copies are shown in the database details pane, as shown in Figure 5.

Figure 5: database copies in details pane

By drilling down to the mailbox database details pane, you can see the status of the selected database and its copies. You can also see important information like copy queue length and content index state. For the passive copies, you can do different operations like suspend and activate based on their current status.

After you have created database copies for a database, you can easily switch to the other databases from the main database list view to create copies of them. As you see, admins can manage database and database copies in one view without switching to another UI. Very handy and straightforward!

Server Switchovers

As mentioned before, besides managing HA at the database level, you can also perform switchovers at the server level. The EAC provides a more comprehensive way of managing servers.

For example, for a variety of reasons, you may need to take some a DAG member offline. The first step in doing this will always be to perform a server switchover; that is, to move all of the active copies currently hosted on that server to other DAG members, as shown below.

Figure 6: Server Switchover

As with Exchange 2010, when performing a switchover, you can specify the switchover target or perform a targetless switchover, as shown below Figure 7.

Figure 7: Two choices for Server Switchover

Conclusion

The handy and improved UI brings you a brand new experience in managing HA in the EAC. You don’t need to toggle between console and shell anymore. And more importantly, you can easily access it from anywhere.

Go and try it out, we are looking forward to hearing from you!

Bin Sun

OAB History

Offline Address Books, fondly referred to as OABs, are a critical component in Exchange infrastructure for a long time now. An OAB is used by Microsoft Outlook clients in Cached Exchange Mode for address book lookups when offline. OABs are also critical in reducing the workload on Exchange servers as cached mode Outlook clients will always query the local OAB first.

The OAB has evolved over Exchange releases. The last major overhaul of OAB architecture was in Exchange Server 2007, where we introduced web-distribution of OAB along with CAS server role taking major responsibility of distributing the OAB. But the OAB generation process itself hasn't changed much.

Until now.

With the change in the server role architecture introduced in Exchange Server 2013, we have also changed the way OABs are generated and distributed to clients. Let’s explore the new OAB in Exchange 2013 by comparing it to its predecessors.

Changes in OAB generation

Which Server will generate the OAB?

In all previous Exchange releases, OAB generation was bound to a specific Exchange server by the Server property. When you install the first Exchange mailbox server, setup designates it as the OAB generation server. You can create new OABs as needed. When creating a new OAB, the OAB generation server has to be specified.

OAB in Exchange Server 2010:

Get-OfflineAddressBook "Default Offline Address Book" | fl name,server
 
Name : Default Offline Address Book
Server : MBX1

The disadvantage with this approach was that only one server was configured for OAB generation, and it was a single point of failure. If this server was unavailable for a long period, the OAB generation was affected.

In Exchange 2013, the OAB is generated by each Exchange 2013 Mailbox server(s) that hosts a special type of arbitration mailbox, called organization mailbox. OAB generation is not bound by the Server parameter anymore.

OAB in Exchange Server 2013:

Get-OfflineAddressBook "Default Offline Address Book (Ex2012)" | fl name,server
 
Name : Default Offline Address Book (Ex2012)
Server :

The unbinding of OAB from a specific server allows the same OAB to be generated by multiple Mailbox servers. This new architecture provides greater resiliency in OAB generation.

Which component will generate the OAB?

The Microsoft Exchange System Attendant service was the workhorse responsible for OAB generation in previous Exchange versions. The OAB generation was a scheduled process, i.e. OAB generation would start at the scheduled time configured on the OAB property, irrespective of the work load on the server.

In Exchange 2013, the OABGeneratorAssistant, a mailbox assistant running under the Microsoft Exchange Mailbox Assistants service, generates the OAB. Like most other mailbox assitants, the OABGEnerationAssistant is a throttled process – it runs or pauses according to the workload on the server.

Where are the OAB files stored?

In previous Exchange versions, the OAB generated by the Mailbox server was located in the %ExchangeInstallPath%\ExchangeOAB folder. The folder was shared so the CAS could retrieve the OAB files for distribution to Outlook clients.

In Exchange 2013, the OAB files are generated and stored in the Organization Mailbox first and later copied to the %ExchangeInstallPath%\ClientAccess\OAB\ folder.

Changes in OAB distribution

Exchange 2007 and 2010 supported two methods of OAB distribution: web distribution and Public Folder distribution. Exchange 2013 supports only the web distribution method, so let’s explore the changes in web-distribution method.

The Exchange 2007/2010 CAS pulled the OAB files generated on the respective Mailbox server and stored them locally. The Microsoft Exchange File Distribution Service on the CAS role did the task of pulling OAB files.

This was the flow OAB download from client side:

1. Outlook receives OAB URL from Autodiscover and reaches a CAS server.
2. The CAS authenticates the user and serves OAB files from local disk.

Couple of disadvantage with this method:

1. The OAB download fails if the CAS doesn't have the OAB files locally.
2. If the File Distribution Service on CAS isn't working, clients will receive stale OAB files or, in other words will not receive the updates.

In Exchange 2013, OAB files are not stored locally on the CAS. CAS 2013 proxies all OAB download requests to the appropriate Exchange 2013 Mailbox server. With this change in the architecture, the Microsoft Exchange File Distribution Service is removed from the CAS role.

In Exchange 2013, this is the flow of OAB download:

1. Outlook receives OAB URL from Autodiscover and reaches designated CAS 2013 through OAB URL.

The CAS server performs the following actions:

1. Performs initial authentication for OAB.
2. Queries Active Directory and determines the closest Organization Mailbox for the requesting user.

3. Queries Active Directory again to determine the mailbox database hosting the Organization Mailbox.

4. Queries the Active Manager to determine the mailbox server where the mailbox database is active (mounted).
5. Proxies the request to the Mailbox server identified in step 4.
6. Retrieves OAB files and passes them to the client.

This new workflow overcomes the disadvantages of legacy OAB download workflow.

The Organization Mailbox

The Organization Mailbox is a new type of arbitration mailbox introduced with Exchange 2013. The arbitration mailbox with persisted capability OrganizationCapabilityOABGen is referred to as Organization Mailbox. It plays a crucial role in OAB generation, storage and distribution.

Each Exchange Server 2013 mailbox role hosting an Organization Mailbox will generate all Exchange 2013 OAB’s defined in the environment. The OAB is generated in the Organization Mailbox first and later copied to the disk.

Use the following command to identify the Organization mailbox:

Get-Mailbox -Arbitration | where {$_.PersistedCapabilities -like "*oab*"}

Example:

Get-Mailbox -Arbitration | where {$_.PersistedCapabilities -like "*oab*"}
 
Name Alias ServerName ProhibitSendQuota
---- ----- ---------- -----------------
SystemMailbox{bb558c35... SystemMailbox{bb5... mbx1 Unlimited

Storing the OAB files in the Organization Mailbox makes the OAB files more resilient.

Putting it together: A real-life scenario:

The following scenario puts together the critical points we learned so far:

1. MBX1 and MBX2 are Exchange 2013 Mailbox servers and member of a DAG. CAS1 is an Exchange 2013 CAS.
2. The organization mailbox is present on mailbox database DB1. DB1 has copies on MBX1 and MBX2.
3. DB1 is currently active on MBX1.
4. The Microsoft Exchange Mailbox Assistants service on MBX1 will generate the OAB.
5. The OAB will be first generated in the organization mailbox and later copied to disk of MBX1. At this point, MBX2 is not playing any role in OAB generation.
6. An Outlook client tries to download OAB, and reaches CAS1 through OAB URL.
7. CAS1 queries Active Manager and finds out database hosting organization mailbox (DB1) is active on MBX1.
8. CAS1 proxies the OAB download request to MBX1 and serves the files back to the client.
9. At this point, MBX1 goes down due to power failure and DB1 is activated on the server MBX2.
10. CAS1 receives another request for OAB download, queries the Active Manager again and this time proxies the request to MBX2, as DB1 is now active on MBX2.
11. MBX2 extracts OAB files present in the organization mailbox to the disk, to ensure latest files are served to the client.
12. MBX1 comes back online, but DB1 remains active on MBX2.
13. At next OAB generation work cycle, the Microsoft Exchange Mailbox Assistants service on MBX2 will generate the OAB.

The next article in this series will talk about how to manage the new OAB in Exchange 2013.

Bhalchandra Atre

Many organizations have chosen to configure a hybrid deployment with Exchange Online to take advantage of different features such as rich mailbox moves and cross-premises calendar free/busy sharing. This includes Exchange 2003, Exchange 2007 and Exchange 2010 organizations that require a long-term hybrid configuration with Exchange Online and organizations that are using a hybrid deployment as a stepping stone to migrating fully to Exchange Online. So, at what point should these organizations decide to get rid of their on-premises Exchange servers used for the hybrid deployment? What if they have moved all of the on-premises mailboxes to Exchange Online? Is there a benefit to keeping on-premises Exchange servers? While it may seem like a no-brainer, the decision to get rid of the on-premises Exchange servers is not simple and definitely not trivial.

Mailbox Management

Organizations that have configured a hybrid deployment for mailbox management and hybrid feature support have also configured Office 365 Active Directory synchronization (DirSync) for user and identity management. For organizations intending on keeping DirSync in place and continuing to manage user accounts from the on-premises organization, we recommend not removing the last Exchange 2010 server from the on-premises organization. If the last Exchange server is removed, you cannot make changes to the mailbox object in Exchange Online because the source of authority is defined as on-premises. The source of authority refers to the location where Active Directory directory service objects, such as users and groups, are mastered (an original source that defines copies of an object) in a hybrid deployment. If you needed to edit most mailbox settings, you would have to be sure the Active Directory schema was extended on-premises and use unsupported tools such as Active Directory Service Interfaces Editor (ADSI Edit) for common administrative tasks. For example, adding a proxy address or putting a mailbox on litigation hold when there isn’t an Exchange Management Console (EMC) or Exchange Management Shell (Shell) on-premises becomes difficult and these simple (and other more complex) tasks cannot be done in a supported way.

Note: A hybrid deployment is not required in order to manage Exchange objects from an on-premises organization. You can effectively manage Exchange objects with an on-premises Exchange server even if you do not have an organization relationship, Federation Trust, and third-party certificate in place. This Exchange server gives you a supported method for creating and managing your Exchange recipient objects. It is recommended to use Exchange Server 2010 for management tasks since this will give you the option to create objects such as remote mailboxes with the New-RemoteMailbox cmdlet. The server role needed is at least a Client Access Server (CAS) role, for management tools to work properly.

Online Organizations without On-Premises Exchange Servers

Some Exchange Online organizations may have removed all Exchange servers from their on-premises organization and have felt the user management pain mentioned above first hand. Each situation is unique, but in many cases an Exchange 2010 server can simply be added back to the organization to simplify the management process. These organizations will need to ensure that a mail-enabled user is in place for all Exchange Online mailboxes in order to properly configure the mailboxes. Assuming DirSync is still deployed in the on-premises organization, duplicate object issues shouldn’t be a problem.

Managing Users from the On-Premises Organization when Source of Authority is Online

There are some organizations that have created an Office 365 service tenant and started to use Exchange Online only to realize they want to consolidate the user management tasks. There are also some organizations that came from hosted environments or migrated from Business Productivity Online Services (BPOS) where they did not manage their users from an on-premises organization. Now that they are in Office 365 and using Exchange Online, they want to simplify the user management process. In either case, if you have DirSync deployed and you are using Exchange Online, you should have an on-premises Exchange server for user management purposes.

The process for changing the source of authority after the users are created in Office 365 would be to use the DirSync “soft match” process outlined here. This will allow organizations to manage the user account and Exchange Online mailboxes from the on-premises organization. Organizations need to verify that there was a mail-enabled user in the on-premises directory for the corresponding Exchange Online mailboxes. Organizations that haven’t had an Exchange server deployed previously will need to install an Exchange 2010 server. Office 365 for enterprises customers can obtain an Exchange Server 2010 license at no charge by contacting customer support. This license has limitations and doesn’t support hosting on-premises mailboxes.

Removing the HybridConfiguration Object created by the Hybrid Configuration Wizard

When a hybrid deployment is created using the Hybrid Configuration Wizard, the wizard creates the HybridConfiguration Active Directory object in the on-premises organization. The HybridConfiguration object is created when the New-HybridConfiguration cmdlet is called by the Hybrid Configuration Wizard. The object stores the hybrid configuration information so that the Update-HybridConfiguration cmdlet can read the settings stored in the object and use them to provision the hybrid configuration settings.

Removing the HybridConfiguration object isn’t supported in Exchange Server 2010. There isn’t a cmdlet that will remove the HybridConfiguration object and the object can reside in Active Directory without adverse effects as long as the Hybrid Configuration Wizard isn’t run again.

However, removing the HybridConfiguration object is supported in Exchange Server 2013. The new Remove-HybridConfiguration cmdlet will remove the HybridConfiguration object from the configuration container, however it will not disable or remove any existing hybrid deployment configuration settings.

Although many people want to remove the HybridConfiguration object as part of their Exchange decommissioning plan, it isn’t critical and is optional.

Removing a Hybrid Deployment

The proper way to remove a hybrid deployment is to disable it manually. The following actions should be performed to remove the objects created and configured by the Hybrid Configuration Wizard:

1. Re-point your organization’s MX record to the Office 365 service if it is pointing to the on-premises organization. If you are removing Exchange and don’t point the MX record to Office 365, inbound Internet mail flow won’t function.

2. Using the Shell in the on-premises organization, run the following commands:

Remove-OrganizationRelationship –Identity “On Premises to Exchange Online Organization Relationship”
Remove-FederationTrust –Identity “Microsoft Federation Gateway”
Remove-SendConnector “Outbound to Office 365”

3. Using EMC, you can also remove the <your organization domain>.mail.onmicrosoft.com domain that was added as part of the email address policy for your organization.

4. OPTIONAL - Remove the remote domains created by the Hybrid Configuration wizard in the Exchange Online organization. From the EMC, select the Hub Transport in the Exchange Online forest node and remove all remote domains starting with “Hybrid Domain” shown below:

5. Remove the organization relationship from the Exchange Online organization with the following command. You must use Remote PowerShell to connect to Exchange Online connected to Exchange Online. For detailed steps, see Connect Windows PowerShell

Remove-OrganizationRelationship –Identity “Exchange Online to On Premises Organization Relationship”

6. OPTIONAL - Disable the Inbound and Outbound Forefront Online Protection for Exchange (FOPE) connectors created by the Hybrid Configuration Wizard. These connectors can be disabled using the FOPE Administration Console and the release option shown below:

Note: Removing or modifying objects with ADSIEDIT isn’t supported.

Conclusion

Most of the time the reason for most organizations that have configured a hybrid deployment, removing the last Exchange server from the on-premises environment will have adverse effects. In most cases, we recommend that you leave at least one Exchange 2010 Server on-premises for mailbox management unless you are getting rid of the on-premises messaging and identity management dependencies all together.

Timothy Heeney

In Exchange Online and Exchange Server 2013, you can use In-Place Hold to preserve mailbox content for litigation or investigations. Many organizations also need to preserve mailbox data for users who are no longer in the organization.

In on-premises Exchange deployments, this has typically been done by disabling the Active Directory user account and performing actions such as removing it from distribution groups, preventing inbound/outbound email to and from the mailbox (including setting delivery restrictions and configuring message size limits), hiding the mailbox from the Global Address List (GAL), and also setting an account expiration date on the user account in Active Direcory. Licensing costs are not a concern in this scenario, because you do not need a Client Access License (CAL) for a mailbox that’s no longer active.

In Exchange Online, admins remove mailboxes for departed users. However, once you remove a mailbox, it can no longer be included in eDiscovery searches (using Multi-Mailbox Search in the previous version). Additionally, 30 days after you remove a mailbox, it is permanently deleted from Exchange Online and can no longer be recovered. Multi-Mailbox Search requires that the mailbox be active, which means an Exchange Online or Office 365 plan is required for the mailbox for as long as you want to preserve data for eDiscovery.

Note: You can preserve mailbox data offline by exporting it to a PST file using Microsoft Outlook and then remove the mailbox. However, if you need to perform an eDiscovery search, you would need to inject it back to an Exchange Online mailbox.

Inactive Mailboxes

In the new Exchange Online, we’ve introduced the concept of inactive mailboxes to handle departed users. When a user leaves the organization and you need to retain their mailbox data for some time to facilitate eDiscovery (or meet retention or business requirements), you can place the mailbox on In-Place Hold before removing the Office 365 user. This preserves the mailbox, but prevents it from sending/receiving messages, hides it from users so it's no longer visible in the GAL and other recipient lists. You can add inactive mailboxes to In-Place eDiscovery searches. Inactive mailboxes do not require an Exchange Online or Office 365 plan.

When your eDiscovery, retention or other business requirements are met and you no longer need to preserve the mailbox content, you can remove the mailbox from In-Place Hold. After you remove hold, the normal mailbox removal behavior of Exchange Online will resume for the mailbox - if the mailbox was removed more than 30 days ago, it will be permanently deleted. If it was removed less than 30 days ago, it will be permanently deleted after 30 days of removal.

For more details, see Managing Inactive Mailboxes (short url: aka.ms/inactivembx) in Exchange Online documentation.

Inactive mailboxes are available in March 2013 in the E3, E4, A3, A4, G and Exchange Online P2 plans.

Bharat Suneja

In Exchange 2010, when a database availability group (DAG) member needs service, it can be placed into maintenance mode. Exchange 2010 includes scripts the StartDagServerMaintenance.ps1 and StopDagServerMaintenace.ps1 scripts to place/remove a DAG member from maintenance mode. For a summary of what these scripts do, see this post.

Within a DAG, it is not uncommon to have one or more databases or servers blocked from automatic activation by the system. Some customers configure entire servers to be blocked from activation, some block individual copies, and some do a combination of both, based on their business requirements. Administrators using the maintenance mode scripts will find their configured activation blocks reset to the unblocked. This behavior is not a problem with the scripts; in fact, the scripts are working as designed.

Here is an example of a database copy that has had activation suspended:

[PS] C:\>Get-MailboxDatabaseCopyStatus DAG-DB0\MBX-2 | fl name,activationSuspended

Name                             : DAG-DB0\MBX-2
ActivationSuspended              : True

Here is an example of a server that has activation blocked:

[PS] C:\>Get-MailboxServer MBX-2 | fl name,databasecopyautoactivationpolicy

Name                             : MBX-2
DatabaseCopyAutoActivationPolicy : Blocked

When the administrator executes the stopDagServerMaintenance.ps1 script, these states are reset back to their defaults. Here is an example of the states post StopDagServerMaintenance.ps1:

[PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>Get-MailboxDatabaseCopyStatus DAG-DB0\MBX-2 | fl name,activationSuspended

Name                             : DAG-DB0\MBX-2
ActivationSuspended              : False

[PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>Get-MailboxServer MBX-2 | fl name,databasecopyautoactivationpolicy

Name                             : MBX-2
DatabaseCopyAutoActivationPolicy : Unrestricted

Although the maintenance scripts behavior is by design, it can lead to undesirable scenarios, such as lagged database copies being activated. Of course, to eliminate these issues, an administrator can record database and server settings before and after maintenance and reconfigure any reset settings as needed.

To help with this, here is a sample script I created that records database and server activation settings into a CSV file which can then be used with the maintenance scripts to adjust the states automatically.

What the script does

When you run the script, it creates two CSV files (on the server you run it on) along with a transcript that contains the results of the command executed. The first CSV file contains all database copies assigned to the server and their activation suspension status. Here's an example:

#TYPE Selected.Microsoft.Exchange.Management.SystemConfigurationTasks.DatabaseCopyStatusEntry
"Name","ActivationSuspended"
"DAG-DB0\DAG-3","True"
"DAG-DB1\DAG-3","True"

The second CSV file contains the database copy auto activation policy of the server. For example:

#TYPE Selected.Microsoft.Exchange.Data.Directory.Management.MailboxServer
"Name","DatabaseCopyAutoActivationPolicy"
"DAG-3","Blocked"

Using maintenanceWrapper.ps1 to start and stop maintenance

Because this scipt is unsigned, you'll need to relax the execution policy on the server to allow for unsigned scripts.

IMPORTANT: Allowing unsigned PowerShell scripts to execute is a security risk. For details, see Running Windows PowerShell Scripts. If this option does not meet your organization's security policy, you can sign the script (requires a code-signing certificate).

This command sets the execution policy on a server to allow unsigned PowerShell scripts to execute:

Set-ExecutionPolicy UNSIGNED

You can use the maintenanceWrapper.ps1 script to start and stop maintenance procedure on a DAG member.

1. Use this command to start the maintenance procedure:

   maintenanceWrapper.ps1 –server <SERVERNAME> –action START

   The command creates the CSV files containing the original database states and then invokes the StartDagServerMaintenance.ps1 script to place the DAG member in maintenance mode.

2. After maintenance is completed, you can stop the maintenance procedure using this command:

   maintenanceWrapper.ps1 –server <SERVERNAME> –action STOP

   The command calls the StopDagServerMaintenance.ps1 script to remove the DAG member from maintenance mode and then resets the database and server activation states from the states recorded in the CSV file.

Give the script a try and see if it makes maintenance mode for activation-blocked servers and databases easier for you. I hope you find this useful, and I welcome any and all feedback.

*Special thanks to Scott Schnoll and Abram Jackson for reviewing this content and David Spooner for validating the script.

Tim McMichael

Update 7/15/2013: we have made a few updates to the below blog post to adjust the instructions for the release of the version 2 of the script.

Prior to Exchange 2007, there were two primary methods of implementing automated resource scheduling – Direct Booking and the AutoAccept Agent(a store event sink released as a web download for Exchange 2003). In Exchange 2007, we changed how automated resource scheduling is implemented. The AutoAccept Agent is no longer supported, and the Direct Booking method, technically an Outlook function, has been replaced with server-side calendar booking function called the Resource Booking Attendant.

Note: There are various terms associated with this new Resource Booking function, such as: Calendar Processing, Automatic Resource Booking, Calendar Attendant Processing, Automated Processing and Resource Booking Assistant. We will be using the “Resource Booking Attendant” nomenclature for this article.

While the Direct Booking method for resource scheduling can indeed work on Exchange Server 2007/2010/2013, we strongly recommend that you disable Direct Booking for resource mailboxes and use the Resource Booking Attendant instead. Specifically, we are referring to the “AutoAccept” Automated Processing feature of the Resource Booking Attendant, which can be enabled for a mailbox after it has been migrated to Exchange 2007 or later and upgraded to a Resource Mailbox.

This conversion process to resource mailboxes utilizing the Resource Booking Attendant is sometimes an honest oversight or even deliberately ignored when migrating away from Exchange 2003 due to Direct Booking’s ability to continue to work with newer versions of Exchange, even Exchange Online. This will often result in resource mailboxes (or even user mailboxes!) with Direct Booking functionality remaining in place long after Exchange 2003 is ancient history in the environment.

Why not just leave Direct Booking enabled?

There are issues that can arise from leaving Direct Booking enabled, from simple administrative burden scenarios all the way to major calendaring issues. Additionally, Resource Booking Attendant offers advantages over Direct Booking functionality:

1. Direct Booking capabilities, technically an Outlook function, has been deprecated from the product as of Outlook 2013. It was already on the deprecation list in Outlook 2010 and required a registry modification to reintroduce the functionality.
2. Direct Booking and Resource Booking Attendant are conflicting technologies, and if simultaneously enabled, unexpected behavior in calendar processing and item consistency can occur.
3. Outlook Web App (as well as any non-MAPI clients, like Exchange ActiveSync (EAS) devices) cannot use Direct Booking for automated resource scheduling. This is especially relevant for Outlook Web App-only environments where the users do not have Microsoft Outlook as a mail client.
4. The Resource Booking Attendant AutoAccept functionality is a server-side solution, eliminating the need for client-side logic in order to automatically process meeting requests.

How do I check which mailboxes have Direct Booking Enabled?

How does one validate if Direct Booking settings are enabled on mailboxes in the organization, especially if mailboxes had previously been hosted on Exchange 2003?

Figure 1: Checking Direct Booking settings in Microsoft Outlook 2010

Unfortunately, the manual steps involve assigning permissions to all mailboxes, creating MAPI profiles for each mailbox, logging into each mailbox, checking Tools> Options> Calendar> Resource Scheduling, note which of the three Direct Booking checkboxes are checked, click OK/Cancel a few times, log out of mailbox. Whew! That can be a major undertaking even for a small to midsize company that has more than a handful of mailboxes! Having staff perform this type of activity manually can be a costly and tedious endeavor. Once you have discovered which mailboxes have the Direct Booking settings enabled, you would then have to repeat this entire process to disable these settings unless you removed them at the time of discovery.

Having an automated method to discover, track, and even disable Direct Booking settings would be nice right?

Look no further, we have the solution for you!

Using Exchange Web Services (EWS) and PowerShell, we can automate the discovery of Direct Booking settings that are enabled, track the results, and even disable them! We wrote Remove-DirectBooking.ps1, a sample script, to do exactly that and even more to aid in automating this manual effort.

After you've downloaded it, rename the file and remove the .txt extension.

IMPORTANT:  The previously uploaded script had the last line truncated to Stop-Tran (instead of Stop-Transcript). We've uploaded an updated version to TechNet Gallery. If you downloaded the previous version of the script, please download the updated version. Alternatively, you can open the previously downloaded version in Notepad or other text editor and correct the last line to Stop-Transcript.

Let’s break down the major tasks the PowerShell script does:

1. Uses EWS Application Impersonation to tap into a mailbox (or set of mailboxes) and read the three MAPI properties where the Direct Booking settings are stored. It does this by accessing the localfreebusy item sitting in the NON_IPM_SUBTREE\FreeBusy Data folder, which resides in the root of the Information Store in the mailbox. The three MAPI properties and their equivalent Outlook settings the script looks at are:

   • 0x686d Automatically accept meeting requests and remove canceled meetings
   • 0x686f Automatically decline meeting requests that conflict with an existing appointment or meeting
   • 0x686e Automatically decline recurring meeting requests

   These three properties contain Boolean values mirroring the Resource Scheduling checkboxes found in Outlook (see Figure 1 above).

2. For each mailbox processed, the script attempts to locate the corresponding free/busy message stored in the ‘Schedule+ Free/Busy’ system Public Folder representing the user.  This item must be updated just like the user’s local mailbox item – the two items must be consistent in their settings. We need to do this because Outlook only considers the settings in the Public Folder free/busy item when a user attempts to Direct Book a resource.  Therefore, it is critical that the script checks for the Public Folder item’s existence and its settings are in sync with the localfreebusy item stored in the mailbox itself.
3. For mailboxes where Direct Booking settings were detected, it checks for conflicts by determining if the mailbox also has Resource Booking Attendant enabled with AutomateProcessing set to AutoAccept.
4. Optionally, disables any enabled Direct Booking settings encountered.

   Note: It is important to understand that by default the script runs in a read-only mode. Additional command line switches are available to run the script to disable Direct Booking settings.

5. Writes a detailed runtime processing log to console and log file.
6. Creates a simple output text file containing a list of mailboxes that can be later leveraged as an input file to feed the script for disabling the Direct Booking functionality.
7. Creates a CSV file containing statistics of the list of mailboxes processed with detailed information, such as what was discovered, any errors encountered, and optionally what was disabled. This is useful for performing analysis in the discovery phase and can also be used as another source to create an input file to feed into the script for disabling the Direct Booking functionality.

Example Scenarios

Here are a couple of example scenarios that illustrate how to use the script to discover and remove enabled Direct Booking settings.

Scenario 1

You've recently migrated from Exchange 2003 to Exchange 2010 and would like to disable Direct Booking for your company’s conference room mailboxes as well as any user mailboxes that may have Direct Booking settings enabled. The administrator’s logged in account has Application Impersonation rights and the View-Only RecipientsRBAC role assigned.

1. On a machine that has the Exchange management tools & the Exchange Web Services API 1.2 or greater installed, open the Exchange Management Shell, navigate to the folder containing the script, and run the script using the following syntax:

   .\Remove-DirectBooking.ps1 –identity * -UseDefaultCredentials

2. The script will process all mailboxes in the organization with detailed logging sent to the shell on the console. Note, depending the number of mailboxes in the org, this may take some time to complete
3. When the script completes, open the Remove-DirectBooking_<timestamp>.txt file in Notepad, which will contain list of mailboxes that have Direct Booking enabled:

   
   Figure 2: Output file containing list of mailboxes with Direct Booking enabled

4. After reviewing the list, rerun the script with the InputFile parameter and the RemoveDirectBookingswitch:

   .\Remove-DirectBooking.ps1 –InputFile ‘.\Remove-DirectBooking_<timestamp>.txt’ –UseDefaultCredentials -RemoveDirectBooking

5. The script will process all the mailboxes listed in the input file with detailed logging sent to the shell on the console. Because you specified the RemoveDirectBooking switch, it does not run in read-only mode and disables all currently enabled Direct Booking settings encountered.
6. When the script completes, you can check the status of the removal operation by checking the Remove-DirectBooking_<timestamp>.csv file. A column called Direct Booking Removed? will record if the removal was successful. You can also check the runtime processing log file RemoveDirectBooking_<timestamp>.logas well.

   
   Figure 3: Reviewing runtime log file in Excel

Note The Direct Booking Removed? column now shows Yes where applicable, but the three Direct Booking settings columns still show their various values as “Yes”; this is because we record those three values pre-removal. If you were to run the script again in read-only mode against the same input file, those columns would reflect a value of N/A since there would no longer be any Direct Booking settings enabled. The Resource Room?, AutoAccept Enabled?, and Conflict Detected all have a value of N/A regardless because they are not relevant when disabling the Direct Booking settings.

Scenario 2

You're an administrator who's new to an organization. You know that they migrated from Exchange 2003 to Exchange 2007 in the distant past and are currently in the process of implementing Exchange 2010, having already migrated some users to Exchange 2010. You have no idea what resources mailboxes or even user mailboxes may be using Direct Booking and would like to discover who has what Direct Booking settings enabled. You would then like to selectively choose which mailboxes to pilot for Direct Booking removal before taking action on the majority of found mailboxes.

Here's how you would accomplish this using the Remove-DirectBooking.ps1 script:

1. Obtain a service account that has Application Impersonation rights for all mailboxes in the org.
2. Ensure service account has at least Exchange View-Only Administrator role (2007) and at least have an RBAC Role Assignment of View Only Recipients (2010/2013).
3. On a machine that has the Exchange management tools & the Exchange Web Services API 1.2 or greater installed, preferably an Exchange 2010 server, open the Exchange Management Shell, navigate to the folder containing the script, and run the script using the following syntax:

   .\Remove-DirectBooking.ps1 –Identity *

4. The script will prompt you for the domain credentials of the account you wish to use because no credentials were specified. Enter the service account’s credentials.
5. The script will process all mailboxes in the organization with detailed logging sent to the shell on the console. Note, depending the number of mailboxes in the org, this may take some time to complete.
6. When the script completes, open the Remove-DirectBooking_<timestamp>.csv in Excel, which will looks something like:

   
   
   Figure 4: Reviewing the Remove-DirectBooking_<timestamp>.csv in Excel

7. Filter or sort the table by the Direct Booking Enabled? column. This will provide a list that can be scrutinized to determine which mailboxes are to be piloted with Direct Booking removal, such as those that have conflicts with already having the Resource Booking Attendant’s Automated Processing set to AutoAccept (which you can also filter on using the AutoAccept Enabled? column).
8. Once the list has been reviewed and the targeted mailboxes isolated, simply copy their email addresses into a text file (one address per line), save the text file, and use it as the input source for the running the script to disable the Direct Booking settings:

   .\Remove-DirectBooking.ps1 –InputFile ‘.\’ -RemoveDirectBooking

9. As before, the script will prompt you for the domain credentials of the account you wish to use. Enter the service account’s credentials.
10. The script will process all the mailboxes listed in the input file with detailed logging sent to the shell on the console. It will disable all enabled Direct Booking settings encountered.
11. Use the same validation steps at the end of the previous example to verify the removal was successful.

Script Options and Caveats

Please see the script’s help section (via “get-help .\remove-DirectBooking.ps1 -full”) for full information on all the available parameters. Here are some additional options that may be useful in certain scenarios:

1. EWSURL switch parameter. By default, the script will attempt to retrieve the EWS URL for each mailbox via AutoDiscover. This is preferred, especially in complex multi-datacenter or hybrid Exchange Online/On-premises environments where different EWS URLs may be in play for any given mailbox depending on where it resides in the org. However, there may be times where one would want to supply an EWS URL manually, such as when AutoDiscover is having “issues”, or the response time for AutoDiscover requests is introducing delays in overall script execution (think very large quantity of number of mailbox identities to churn through) and the EWS URL is the same across the org, etc. In these situations, one can use the EWSURL parameter to feed the script a static EWS URL.
2. UseDefaultCredentials If the current user is the service account or perhaps simply has both the Impersonation and the necessary Exchange Admin rights per the script’s requirements and they don’t wish to be prompted to type in a credential (another great example is scheduling the script to run as a job for instance), you can use the UseDefaultCredentials to run the script under that security context.
3. RemoveDirectBooking By default, the script runs in read-only mode. In order to make changes and disable Direct Booking settings on the mailbox, you mus specify the RemoveDirectBooking switch.

The script does have several prerequisites and caveats to ensure proper operation and meaningful results:

1. Application Impersonation rights and minimum Exchange Admin rights must be used
   • For Exchange 2007, see Configuring Exchange Impersonation (Exchange Web Services)
   • For Exchange 2010/2013, see Configuring Exchange Impersonation (uses RBAC)
2. Exchange Web Services Managed API 1.2 or later must be installed on the machine running the script
3. Exchange management tools must be installed on the machine running the script
4. Script must be executed from within the Exchange Management Shell
5. The Shell session must have the appropriate execution policy to allow the script to be executed (by default, you can't execute unsigned scripts).
6. AutoDiscover must be configured correctly (unless the EWS URL is entered manually)
7. Exchange 2003-based mailboxes cannot be targeted due to lack of EWS capabilities
8. In an Exchange 2010/2013 environment that also has Exchange 2007 mailboxes present, the script should be executed from a machine running Exchange 2010/2013 management tools due to changes in the cmdlets in those versions
9. Due to limitations in the EWS architecture, the Schedule+ Free/Busy System Folder subfolders must contain a replica on each version of Exchange that users are being processed for; additionally, the user's Exchange mailbox database 'Default public folder database' property must be set to a Public Folder database that is on the same version of Exchange as the mailbox database.

Summary

The discovery and removal of Direct Booking settings can be a tedious and costly process to perform manually, but you can avoid and automate it using current functions and features via PowerShell and EWS in Microsoft Exchange Server 2007, 2010, & 2013. With careful use, the Remove-DirectBooking.ps1 script can be a valuable tool to aid Exchange administrators in maintaining automated resource scheduling capabilities in their Microsoft Exchange environments.

Your feedback and comments are welcome.

Thank you to Brian Day and Nino Bilic for their guidance in content review, and to our customers (you know who you are) for piloting the script.

Seth Brandes& Dan Smith

A question that is often asked of Support in regard to legacy Public Folders is whether they're replicating and how much progress they're making.  The most common scenario arises when the administrator is adding a new Public Folder database to the organization and replicating a large amount of data to it.  What commonly happens is that the administrator calls Support and says:

The database on the old server is 300GB, but the new database is only 150GB!  How can I tell what still needs to be replicated?  Is it still progressing??

You can raise diagnostic logging for public folders, but reading the events to see which folders are replicating is tedious.  Most administrators want a more detailed way of estimating the progress of replication than comparing file sizes.  They also want to avoid checking all the individual replication events.

There are a number of ways to monitor replication progress so that one can make an educated guess as to how long a particular environment will take to complete an operation.  In this post, I'm going to provide a detailed example of one approach to estimating the progress of replication by comparing item counts between different public folder stores.

Getting Public Folder item counts

To get the item counts in an Exchange 2003 Public folder database you can use PFDAVAdmin.  The process is outlined in this previous EHLO blog post.  For what we're doing below, you'll need the DisplayName, Folderpath and the total number of items in the folder. The rest of the fields aren't necessary.

To get the item counts on an Exchange 2007 server, use (remember there is only one Pub per server):

Get-PublicFolderStatistics -Server <servername> | Export-Csv c:\file1.txt

To get the item counts on an Exchange 2010 server, you use:

Get-PublicFolderStatistics -Server <servername> -ResultSize unlimited | Export-Csv c:\file1.txt

Comparing item counts

There are some very important caveats to this whole procedure.  The things you need to watch out for are:

• We're only checking item counts.  If you delete 10 items and add 10 items between executions of the statistics data, gathering this type of query will not reveal whether they have replicated.  Therefore, having the same number on both sides is not necessarily an assurance that the folders are in sync. 
• If you're comparing folders that contain recurring meetings, the item counts can be different on Exchange 2007 and older because of the way WebDAV interacts with those items.
• I've seen many administrators try to compare the size of one Public Folder database to the size of another.  Such an approach to checking on replication does not take into account space for deleted items, overhead and unused space.  Checking item counts is more reliable than simply comparing item sizes
• The two databases might be at very different stages of processing replication messages.  It is unlikely that both pubs will present the same numbers of items if the folders are continuously active.  Even if the folders are seeing relatively low activity levels, it's not uncommon for the item count to be off by one or two items because the replication cycle (which defaults to every 15 minutes) simply hasn’t gotten to the latest post
• If you really want to know if two replicas are in sync, try to remove one.  If Exchange lets you remove the instance, you know Exchange believes the folders are in sync.  If Exchange cannot confirm the folders are in sync, it'll keep the instance until it can complete the backfill from it.  In most cases, the administrators I have spoken with are not in a position where they can use this approach.

For the actual comparison you can use any number of products.  For this blog I have chosen Microsoft Access for demonstrating the process of comparing the CSV files from the different servers.  To keep things simple I am going to use the Access database.  There are some limitations to my approach:

• Access databases have a maximum file size of 2GB. If your public folder infrastructure is particularly large (i.e.  your CSV files are over 500MB) you may have to switch to using Microsoft SQL.
• I am not going to compare public folders with a Folder path greater than 254 characters because the Jet database engine that ships with Access cannot join memo fields in a query.  Working around the join limitation by splitting the path across multiple text fields is beyond the scope of this blog.
• I am going to look at folders that exist in both CSV files.   If the instance has not been created and its data exported into the CSV file the folder will not be listed.

An outline of the process is:

1. Export the item counts from the two servers you wish to compare
2. Import the resulting text files
3. Clean up the data for the final query
4. Run a query to list the item counts for all folders that are in Both files and the difference in the item counts between the originally imported files

Assumptions for the steps below:

• You have exported the public folder statistics with the PowerShell commands presented above
• You have fields named FolderPath, ItemCount and Name in the CSV file

If your file is different than expected you will have to modify the steps as you go along

Here are the steps for conducting the comparison:

1. Create a new blank Microsoft Access database in a location that has more than double the size of your CSV files available as free space.

2. By default, the Export-Csv cmdlet includes the .NET type information in the first line of the CSV output. Because this line will interfere with the import, we'll need to remove it.  Open each CSV file in notepad (this can take a while for larger files) and remove the line highlighted below.  In this example the line starting with “AdminDisplayName” would become the topmost line of the file.  Once the top line is deleted close and save the file.

Figure 1

TIP You can avoid this step by including the -NoTypeInformation switch when using the Export-CSV cmdlet, which filters out the .NET object type information from the CSV output. For details, see Using the Export-Csv cmdlet on TechNet. (Thanks to #MSExchange MVP @SteveGoodman for the tip!)

3. Import the CSV file to a new table:

• Click on the External Data tab as highlighted in Figure 2
• Browse to the CSV file and select it (or type in its path and name directly)
• Make sure the “Import the source data into a new table in the current database’ option is selected
• Click OK

Figure 2

4. In the wizard that starts specify the file is delimited as shown and then click Next.

Figure 3

5. Tell the wizard that the text qualifier is the double quote (character 34 in ASCII), the delimiter is the comma and that the “First Row Contains Field Names” as shown in Figure 4.

Note:  It is possible that you will receive a warning when you click “First Row Contains Field Names”.  If any of the field names violate the rules for a field name Access will display a warning.  Don’t panic.  Access will replace the non-conforming names with ones it considers appropriate (typically Field1, Field2, etc.).  You can change the names if you wish on the Advanced screen.

Figure 4

6. Switch to Advanced view (click the Advanced button highlighted in Figure 4) so that we can change the data type of the FolderPath field.  In Access 2010 and older the data type needs to be changed from Text to Memo.  In Access 2013 it needs to be changed from Short Text to Long Text.  While we are in this window you have the option to exclude columns that are not needed by placing a checkmark in the box from the skip column.  In this blog we are only going to use the FolderPath, name and the item count.  You can also exclude fields earlier in the process by specifying what fields will be exported when you do the export-csv.  The following screenshots show the Advanced properties window.

Figure 5a: Access 2010 and older

Figure 5b: Access 2013

Note:  If you think you will be doing this frequently you can use the Save As button to save your settings.  The settings will be saved inside the Access database and can then be selected during future imports by clicking on the Specs button.

7. Click OK on the Advanced dialog and then click Finish in the wizard.

8. When prompted to save the Import steps click Close.  If you think you will be repeating this process in the future feel free to explore saving the import steps.

9. Access will import the data into a table.  By default the table will have the same name as the source CSV file.  The files used in creating this blog were called 2007PF_120301 and 2010 PF_120301.  If there are any import errors they will be saved in a separate table.  Take a moment to examine what they are.  The most common is that a field got truncated.  If that field is the folderpath it will affect the comparisons later.  If there are other problems you will have to troubleshoot what is wrong with the highlighted lines (typically there should be no import errors as long as the FolderPath is set as a Memo field).

10. Go back to Step 2 to import the second file that will be used in the comparison. 

11. Now a query must be run to determine if any folderpath exceeds 255 characters.  Fields longer than 255 characters cannot be used for a join in an Access query.  If we have values that exceed 255 characters in this field we will need to exclude them from the comparison.  Additional work to split a long path across multiple fields can be done, but that is being left as an exercise for any Access savvy readers. 

12. To get started select the options highlighted in Yellow in Figure 6:

Figure 6

13. Highlight the table where we want to check the length of the folderpath field as shown in Figure 7.  Once you have selected the table click Add and then Close:

Figure 7

14. Switch to SQL view as shown in Figure 8:

Figure 8

15. Replace the default select statement with one that looks like this (please make sure you substitute your own table name for the one that I have Bolded in the example):

SELECT Len([FolderPath]) AS Expr1, [2007PF_120301].FolderPath
FROM 2007PF_120301
WHERE (((Len([FolderPath]))>254));

Note:  Be sure the semi-colon is the last character in the statement.

16. Run the query using the red “!” as shown in Figure 9: 

Figure 9

Figure 10

17. If the result is a single empty row (as shown in Figure 10) then skip down to step 19.  If the result is at least one row then go back to SQL view (as shown in Figure 8) and change the statement to look like this one (as before please make sure 2007PF_120301 is replaced with the table name actually being used in your database):

SELECT [2007PF_120301].FolderPath, [2007PF_120301].ItemCount,
[2007PF_120301].Name, [2007PF_120301].Identity INTO 2007PF_120301_trimmed
FROM 2007PF_120301
WHERE (((Len([FolderPath]))<255));

18. You will get a prompt like the one in Figure 11 when you run the query.  Select Yes:

Figure 11

19. After it is done repeat steps 11-18 for the other CSV file that was imported to be part of the comparison.  If you have done steps 11-18 for both files you will be comparing then advance to step 20.

20. Originally the FolderPath was imported as a memo field (Long Text if using Access 2013).  However we cannot join memo fields in a query.  We need to convert them to a text field with a length of 255. 

If you got a result greater than zero rows in step 16 this step and the subsequent steps will all be carried out on the table specified in the INTO clause of the SQL statement (in this blog that table is named 2007PF_120301_trimmed). 

If you were able to skip steps 17 and 18 this step and the subsequent steps will be carried out on the table you imported (2007PF_120301 in this example).

Open the table in Design view by right-clicking on it and selecting Design View as shown in Figure 12.  If you select the wrong tables for the subsequent steps you will get a lot of unwanted duplicates in your final comparison output.

Figure 12

21. Change the folderpath from Memo to Text as shown in Figure 13.  If you are using Access 2013 change it from Long Text to Short Text.

Figure13

22. With the FolderPath field highlighted look to the lower part of the Design window where the properties of the currently selected field are displayed.  Change the field size of folderpath to 255 characters as shown in Figure 14.

Figure 14

23. Save the table and close its design view.  You will be prompted as shown in Figure 15.  Don’t panic.  All the folderpaths should be shorter than the 255 characters specified in the properties of the table.  The dialog is just a standard warning from Access.  No data should be truncated (the earlier queries should have seen to that).  Say Yes and repeat steps 20-23 for the other table being used in this comparison.  If you make a mistake here remember that you will still have your original CSV files and can always fix the mistake by removing the tables and redoing the import.

Figure 15

24. We have been on a bit of a journey to make sure we prepared the tables.  Now for the comparison.  Create a new query (as shown in Figure 6) and highlight both tables that have had the FolderPath shortened to 255 characters as shown in Figure 16.  Once they are highlight click Add and then close.

Figure 16

25. Drag Folderpath from the table that is the source of your replication to Folderpath on the other database.  The result will look like Figure 17.

Figure 17

26.   In the top half of the Query Design window we have the tables with their fields listed.  In the bottom half we have the query grid.  You can make fields appear in the grid in 3 ways:

• Switch the SQL view and add them to the Select statement
• Double-click the field in the top half of the window
• Drag the field from the top half of the window to the grid
• Click in the Field line of the grid and a drop down will appear that you can use to select the fields
• Type the field name you want on the Field in the grid

For this step we need to add:

• One copy of the folderpath field from one table (doesn’t matter which one)
• The ItemCount field from each table

27.   Go to an empty column in the grid.  We need to enter the text that will tells us the difference between the two item counts.  Type the following text into the column (be sure to use the table names from your own database and not my example): 

Expr1:  Abs([2007PF_120301_trimmed].[itemcount]-[2010pf_120301_trimmed].[itemcount])

Note:  After steps 25-27 the final result should look like  Figure 18.  The equivalent SQL looks like this:

SELECT [2007PF_120301_trimmed].FolderPath, [2007PF_120301_trimmed].ItemCount, [2010PF_120301_trimmed].ItemCount, Abs([2007PF_120301_TRIMMED].[ItemCount]-[2010PF_120301_TRIMMED].[ItemCount]) AS Expr1
FROM 2007PF_120301_trimmed INNER JOIN 2010PF_120301_trimmed ON [2007PF_120301_trimmed].FolderPath = [2010PF_120301_trimmed].FolderPath;

Figure 18

28. Run the query using the red “!” shown in Figure 9.  The results will show you all the folders that exist in BOTH public folder databases, the itemscount in each database and the difference between them.  I like the difference reported as a positive number, but you might prefer to remove the absolute value function.

There is more that can be done with this.  You can use Access to run a Find Unmatched query to find all items from one table that are not in the other table (thus locating folders that have an instance in one database, but not the other).  You can experiment with different Join types in the query and you can deal with Folderpaths longer than a single text field can accommodate.  These and any other additional functionality you desire are left as an exercise for the reader to tackle.  I hope this provides you with a process that can be used to compare the item counts between two Public Folder stores (just remember the caveats at the top of the article).

Thanks To Bill Long for reviewing my caveats and Oscar Goco for reviewing my steps with Access.

Chris Pollitt

A short while ago, we posted an article on how to Use Exchange Web Services and PowerShell to Discover and Remove Direct Booking Settings. We received a lot of constructive feedback with some noting that users can experience an issue when enabling the Resource Booking Attendant on mailboxes that were cleansed of their direct booking settings via the sample script we provided. Specifically, the following error can be encountered when the organizer is scheduling a regular non-recurring meeting against the resource mailbox:

“…declined your meeting because it is recurring. You must book each meeting separately with this resource.”

We have updated the script to account for this scenario to prevent and correct this from occurring and we have also updated the article to reflect the changes as well.

In a nutshell, the issue is encountered when we have a divergence of what settings are enabled/disabled between the Schedule+ Free/Busy System (Public) Folder item representing the user’s mailbox and the user’s local mailbox free/busy item. Outlook’s Direct Booking process actually queries the Schedule+ item’s Direct Booking settings when attempting to perform Direct Booking functionality. The Schedule+ folder tree normally contains an item that contains a synced set of Direct Booking settings of that which is stored in the user’s localfreebusy mailbox item. The issue is encountered when the settings between the Schedule+ item and the local mailbox item do not match.

Normally, Outlook triggers a sync of the local mailbox item to the Schedule+ item via deliberate native MAPI code. However, in our case we are using EWS in the sample script, and that syncing trigger does not natively exist. We therefore updated the script to find the Schedule+ item and ensure its settings are congruent with the local item’s settings. The logic for this is actually a bit complicated for two main reasons:

1. No Schedule+ item exists in the organization – There are valid scenarios where the Schedule+ item may not exist, such as the mailbox was never opened with Outlook and the Direct Booking settings were enabled via another means, such as MFCMAPI and so on.
2. Co-existent versions of Exchange - EWS is rather particular on how public folder and public folder item bindings can occur. EWS by design will not allow a cross-version public folder (or item) bind operation. Period. This means a session, for example on a mailbox on Exchange 2010 would not be able to bind to a public folder or its items on Exchange 2007, there would need to be a replica of the folder on Exchange 2010 for the bind operation to be successful. Further, continuing our example, even if the there is a replica on Exchange 2010, the bind operation would still fail if the user’s mailbox database’s “default public folder database” is set to a non-2010 public folder database (i.e. an Exchange 2007 database). The EWS session would kick back an error stating: ‘There are no public folder servers available’

With these guidelines in mind, we approached the script update to maximize the congruency potential between the local mailbox item and the public folder item. We only disable the direct booking settings in the local mailbox item if one of the following criteria is met regarding the Schedule+ item:

• We can successfully bind to the user’s Schedule+ item
  • There is a replica we can touch with the EWS session, and we found the item representing the user and we can therefore safely keep congruency between the local and the Schedule+ items.
• There is no replica present that would potentially contain an item representing the user
  • There is no replica in the org (any version of exchange) that would contain an item for the user so there is no potential for getting into an incongruent state between the local and the Schedule+ items.
• There is a replica of the Schedule+ folder on the same version of Exchange that the EWS session is connected to, AND the default public folder database of the user is likewise on the same version of Exchange.
  • We could not find a Schedule+ item for the user (if we did, we would have satisfied condition 1 above), but not because there was no replica containing the item (if we did, we would have satisfied condition 2 above), and not because we could not bind to the folder via the EWS limitations we outlined above. We can therefore state that congruency between the local and the Schedule+ items are not at risk and there is no Schedule+ item representing the user.

It should be noted that we will always take action to disable the Direct Booking settings from the Schedule+ item even if the local mailbox item does not have its Direct Booking settings enabled – this keeps us true to our “congruency” logic.

In closing, please remember that the script is a sample and does not cover every possible scenario out there – we made this update because the aforementioned issue reported is central to having the script produce the desired outcome of fully disabling Direct Booking. We are thankful for and welcome your continued feedback!

Dan Smith & Seth Brandes
Exchange PFEs

Exchange Server 2013 introduces a new feature called Managed Availability, which is a built-in monitoring system with self-recovery capabilities.

If you’re not familiar with Managed Availability, it’s a good idea to read these posts:

• Lessons from the Datacenter: Managed Availability
• What Did Managed Availability Just Do To This Service?

As described in the above posts, Managed Availability performs continuous probing to detect possible problems with Exchange components or their dependencies, and it performs recovery actions to make sure the end user experience is not impacted due to a problem with any of these components.

However, there may be scenarios where the out-of-box settings may not be suitable for your environment. This blog post guides you on how to examine the default settings and modify them to suit your environment.

Managed Availability Components

Let’s start by finding out which health sets are on an Exchange server:

Get-HealthReport -Identity Exch2

This produces the output similar to the following:

Next, use Get-MonitoringItemIdentity to list out the probes, monitors, and responders related to a health set. For example, the following command lists the probes, monitors, and responders included in the FrontendTransport health set:

Get-MonitoringItemIdentity -Identity FrontendTransport -Server exch1 | ft name,itemtype –AutoSize

This produces output similar to the following:

You might notice multiple probes with same name for some components. That’s because Managed Availability creates a probe for each resource. In following example, you can see that OutlookRpcSelfTestProbe is created multiple times (one for each mailbox database present on the server).

Use Get-MonitoringItemIdentity to list the monitoring Item Identities along with the resource for which they are created:

Get-MonitoringItemIdentity -Identity Outlook.Protocol -Server exch1 | ft name,itemtype,targetresource –AutoSize

Customize Managed Availability

Managed Availability components (probes, monitors and responders) can be customized by creating an override.

There are two types of override: local override and global override. As their names imply, a local override is available only on the server where it is created, and a global override is used to deploy an override across multiple servers.

Either override can be created for a specific duration or for a specific version of servers.

Local Overrides

Local overrides are managed with the *-ServerMonitoringOverride set of cmdlets. Local overrides are stored under following registry path:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\ActiveMonitoring\Overrides\

The Microsoft Exchange Health Management service reads this registry path every 10 minutes and loads configuration changes. Alternatively, you can restart the service to make the change effective immediately.

You would usually create a local override to:

• Customize a managed availability component that is server-specific and not available globally; or
• Customize a managed availability component on a specific server.

Global Overrides

Global overrides are managed with the *-GlobalMonitoringOverride set of cmdlets. Global overrides are stored in the following container in Active Directory:

CN=Overrides,CN=Monitoring Settings,CN=FM,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=Contoso,DC=com

Get Configuration Details

The configuration details of most of probes, monitors, and responders are stored in the respective crimson channel event log for each monitoring item identity, you would examine these first before deciding to change.

In this example, we will explore properties of a probe named “OnPremisesInboundProxy”, which is part of the FrontendTransport Health Set. The following script lists detail of the OnPremisesInboundProxy probe:

(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/ProbeDefinition | % {[XML]$_.toXml()}).event.userData.eventXml | ?{$_.Name -like "OnPremisesInboundProxy"}

You can also use Event Viewer to get the details of probe definition. The configuration details most of the Probes are stores in the ProbeDefinition channel:

1. Open Event Viewer, and then expand Applications and Services Logs\Microsoft\Exchange\ActiveMonitoring\ProbeDefinition.
2. Click on Find, and then enter OnPremisesInboundProxy.
3. The General tab does not show much detail, so click on the Details tab, it has the configuration details specific to this probe. Alternatively, you can copy the event details as text and paste it into Notepad or your favorite editor to see the details.

Override Scenarios

Let’s look at couple real-life scenarios and apply our learning so far to customize managed availability to our liking, starting with local overrides.

Creating a Local Override

In this example, an administrator has customized one of the Inbound Receive connectors by removing the binding of loopback IP address. Later, they discover that the FrontEndTransport health set is unhealthy. On further digging, they determine that the OnPremisesInboundProxy probe is failing.

To figure out why the probe is failing, you can first list the configuration details of OnPremisesInboundProxy probe.

(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/ProbeDefinition | % {[XML]$_.toXml()}).event.userData.eventXml | ?{$_.Name -like "OnPremisesInboundProxy"}

Name : OnPremisesInboundProxy

ExtensionAttributes : <ExtensionAttributes><WorkContext><SmtpServer>127.0.0.1</SmtpServer><Port>25</Port><HeloDomain>InboundProxyProbe</HeloDomain><MailFrom Username="inboundproxy@contoso.com"/><MailTo Select="All" Username="HealthMailboxdd618748368a4935b278e884fb41fd8a@FM.com"/><Data AddAttributions="false">X-Exchange-Probe-Drop-Message:FrontEnd-CAT-250 Subject:Inbound proxy probe</Data><ExpectedConnectionLostPoint>None</ExpectedConnectionLostPoint></WorkContext></ExtensionAttributes>

The ExtentionAttributes property above shows that the probe is using 127.0.0.1 for connecting to port 25. As that is the loopback address, the administrator needs to change the SMTP server in ExtentionAttributes property to enable the probe to succeed.

You use following command to create a local override, and change the SMTP server to the hostname instead of loopback IP address.

Add-ServerMonitoringOverride -Server ServerName -Identity FrontEndTransport\OnPremisesInboundProxy -ItemType Probe -PropertyName ExtensionAttributes -PropertyValue '<ExtensionAttributes><WorkContext><SmtpServer>Exch1.contoso.com</SmtpServer><Port>25</Port><HeloDomain>InboundProxyProbe</HeloDomain><MailFrom Username="inboundproxy@contoso.com" /><MailTo Select="All" Username="HealthMailboxdd618748368a4935b278e884fb41fd8a@FM.com" /><Data AddAttributions="false">X-Exchange-Probe-Drop-Message:FrontEnd-CAT-250Subject:Inbound proxy probe</Data><ExpectedConnectionLostPoint>None</ExpectedConnectionLostPoint></WorkContext></ExtensionAttributes>' -Duration 45.00:00:00

The probe will be created on the specified server in following registry path:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\ActiveMonitoring\Overrides\Probe

You can use following command to verify if the probe is effective:

(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/ProbeDefinition | % {[XML]$_.toXml()}).event.userData.eventXml | ?{$_.Name -like "OnPremisesInboundProxy"}

Creating a Global Override

In this example, the organization has an EWS application that is keeping the EWS app pools busy for complex queries. The administrator discovers that the EWS App pool is recycled during long running queries, and that the EWSProxyTestProbe probe is failing.

To find out the details of EWSProxyTestProbe, run the following:

(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/ProbeDefinition | % {[XML]$_.toXml()}).event.userData.eventXml | ?{$_.Name -like "EWSProxyTestProbe"}

Next, change the timeout interval for EWSProxyTestProbe to 25 seconds on all servers running Exchange Server 2013 RTM CU2:

1. Use following command to get version information for Exchange 2013 RTM CU2 servers:
   

   Get-ExchangeServer | ft name,admindisplayversion

2. Use following command to create a new global override:
   

   Add-GlobalMonitoringOverride -Identity "EWS.Proxy\EWSProxyTestProbe" -ItemType Probe -PropertyName TimeoutSeconds -PropertyValue 25 –ApplyVersion “15.0.712.24”

Override Durations

Either of above mentioned overrides can be created for a specific Duration or for a specific Version of Exchange servers. The override created with Duration parameter will be effective only for the period mentioned. And maximum duration that can be specified is 60 days. For example, an override created with Duration 45.00:00:00 will be effective for 45 days since the time of creation.

The Version specific override will be effective as long as the Exchange server version matches the value specified. For example, an override created for Exchange 2013 CU1, with version “15.0.620.29” will be effective until the Exchange server version changes. The override will be ineffective if the Exchange server is upgraded to different version of Cumulative Update or Service Pack.

Hence, if you need an override in effect for longer period, create the override using ApplyVersion parameter.

Removing an Override

Finally, this last example shows how to remove the local override that was created for the OnPremisesInbounxProxy probe.

Remove-ServerMonitoringOverride -Server ServerName -Identity FrontEndTransport\OnPremisesInboundProxy -ItemType Probe -PropertyName ExtensionAttributes

Conclusion

Managed Availability performs gradual recovery actions to automatically recover from failure scenarios. The overrides help you customize the configuration of the Managed Availability components to suite to your environment. The steps mentioned in the document can be used to customize Monitors & Probes as required.

Special thanks to Abram Jackson, Scott Schnoll, Ben Winzenz, and Nino Bilic for reviewing this post.

Bhalchandra Atre

I continue to run into this issue over and over in the field so I wanted people to be aware of this possible problem. In a Database Availability Group (DAG), if your databases are randomly mounting or flipping from one server to another, for no apparent reason (including across datacenters) you may be suffering from your network interface card (NIC) going to sleep. And that’s not a good thing.

Power Management on the NIC

In the power management settings for the NIC on Windows Server, make sure you are not allowing the NIC to go into power save mode. Why is this important? It seems like at least once a month I’ve run into customers who have this power management setting turned on and more than one of them even had it turned on for their replication network. They were seeing some odd behavior - for example, their databases randomly flipping from one DAG node to another for no apparent reason. And yes, they were all on physical machines.

Here are the steps to look at this configuration: use Device Manager to change the power management settings for a network adapter.

To disable all Power Management settings in Device Manager, expand Network Adapters, right-click the adapter > Properties> Power Management, and then clear the Allow the computer to turn off this device to save power check box.

Figure 1: Disable power management for the network adapter from the Power Management tab

Some of your network adapters may not have the Power Management tab available. This is a good thing, as your NIC is not able to go to sleep. This means there is one less item to worry about in your setup!

CAUTION Be careful when you change this setting. If it's enabled and you decide to disable it, you must plan for this modification as it will likely interrupt network traffic. It may seem odd that by just making a seemingly non-impacting change that the NIC will reset itself, but it definitely can. Trust me; I had a customer ‘test’ this during the day by accident… oops!

PowerShell to the rescue

In addition, now that PowerShell is able to be used for just about everything, there is this page that has a PS script available to make this change. There are additional links and related forum threads to review with supplementary information near the bottom of the script download page.

This modifying script will run against all physical adapters to the machines you deploy it to, and you can also modify the script to disable wireless NIC’s. With PS, don’t forget that you can use this script to blast down these changes to all of your Exchange servers with a single step.

GPO and regedit

For those of you that are more comfortable with regedit and creating GPO’s to help control these settings, that option is also available. This page has information on both ‘one off’ fixes that you can download a .reg file and manually deploy, or using GPO Preferences, you can edit the values in a GPO and apply those changes to an Exchange Server OU (Organizational Unit).

The one step to note with the regedit process is which NIC you are working with and how many NIC’s your server has. The registry only knows of the first, second, third, etc. number of NIC’s. Now if you have identical builds between all of your servers, then this option certainly will ensure that all current and any future servers placed into an OU with the GPO applied will adhere to the proper registry settings.

Also don’t forget, you can record all of your changes on a Windows Server 2008R2 or higher OS, by using the Problem Steps Recorder (PSR) tool.

There you have it: if your DAG Databases are randomly becoming active from one server to another with no apparent reason, you may have a sleepy NIC. Please confirm that you have avoided this setting as you build out not only your DAG environment, but all Exchange related servers. Thank you.

Mike O'Neill

I’ve written a few blog posts now that get into the deep technical details of Managed Availability. I hope you’ve liked them, and I’m not about to stop! However, I’ve gotten a lot of feedback that we also need some simpler overview articles. Fortunately, we’ve just completed documentation on TechNet with an overview of Managed Availability. This was written to address how the feature may be managed day-to-day.

Even that documentation doesn’t address how you respond when Managed Availability cannot resolve a problem on its own. This is the very most common interaction with Managed Availability, but we haven’t described how specifically to do so.

When Managed Availability is unable to recover the health of a server, it logs an event. Exchange Server has a long history of logging warning, error, and critical events into various channels when things go wrong. However, there are two things about Managed Availability events that make them more generally useful than our other error events:

• They all go to the same place on a server without any clutter
• They will only be logged when the standard recovery actions fail to restore health of the component

When one of these events is logged on any server in our datacenters, a member of the product group team responsible for that health set gets an immediate phone call.

No one likes to wake up at 2 AM to investigate and fix a problem with a server. This keeps us motivated to only have Managed Availability alerts for problems that really matter, and also to eliminate the cause of the alert by fixing underlying code bugs or automating the recovery. At the same time, there is nothing worse than finding out about incidents from customer calls to support. Every time that happens we have painful meetings about how we should have detected the condition first and woken someone up. These two conflicting forces strongly motivate the entire engineering team to keep these events accurate and useful.

The GUI

Along with a phone call, the on-call engineer receives an email with some information about the failure. The contents of this email are pulled from the event’s description.

The path in Event Viewer for these events is Microsoft-Exchange-ManagedAvailability/Monitoring. Error event 4 means that a health set has failed and gives the details of the monitor that has detected the failure. Information event 1 means that all monitors of a health set have become healthy.

The Exchange 2013 Management Pack for System Center Operations Manager nicely shows only the health sets that are currently failed instead of the Event Viewer’s method of displaying all health sets that have ever failed. SCOM will also roll up health sets into four primary health groups or three views.

The Shell

This wouldn’t be EHLO without some in-depth PowerShell scripts. The event viewer is nice and SCOM is great, but not everyone has SCOM. It would be pretty sweet to get the same behavior as SCOM to show only the health sets on a server that are currently failed.

Note: these logs serve a slightly different purpose than Get-HealthReport. Get-HealthReport shows the current health state of all of a server’s monitors. On the other hand, events are only logged in this channel once all the recovery actions for that monitor have been exhausted without fixing the problem. Also know that these events detail the failure. If you’re only going to take action based on one health metric, the events in this log is a better one. Get-HealthReport is still the best tool to show you the up-to-the-minute user experience.

We have a sample script that can help you with this; it is commented in a way that you can see what we were trying to accomplish. You can get the Get-ManagedAvailabilityAlerts.ps1 script here.

Either this method or Event Viewer will work pretty well for a handful of servers. If you have tens or hundreds of servers, we really recommend investing in SCOM or another robust and scalable event-collection system.

My other posts have dug deeply into troubleshooting difficult problems, and how Managed Availability gives an overwhelmingly immense amount of information about a server’s health. We rarely need to use these troubleshooting methods when running our datacenters. However, the only thing you need to resolve Exchange problems the way we do in Office 365 is a little bit of event viewer or scheduled script.

Abram Jackson
Program Manager, Exchange Server

Exchange 2013 debuted last year with the ability to configure a Hierarchical Address Book (HAB) for on premises deployments.

As many of our customers are migrating and evaluating the cloud, a consistent user experience between online and on premises deployments is not just a requirement, but an expectation. As such, we are really excited to announce the availability of HAB to Office 365 users, worldwide!   

Now, a tenant administrator for Office 365 is able to configure a HAB using the same commands one would in an on premises deployment.

For more information on how to configure and use a HAB, please refer to HierarchicalAddressBooks: Exchange 2013 Help‎.

FAQ

We are moving some of our user mailboxes to Office 365 while keeping some in our on premises infrastructure. Are the users able to see the same HAB?

No, we are working to resolve a possible issue of the HAB being out of sync in this hybrid deployment case. A solution will be available in the near future.

Can Outlook Web App (OWA) users use the HAB?

No, browsing the HAB is available for Outlook 2010/2013 only at this time.

Paul Lo

Probes are one of the three critical parts of the Managed Availability framework (monitors and responders are the other two). As I wrote previously, monitors are the central components, and you can query monitors to find an up-to-the-minute view of your users’ experience. Probes are how monitors obtain accurate information about that experience.

There are three major categories of probes: recurrent probes, notifications, and checks.

Recurrent Probes

The most common probes are recurrent probes. Each probe runs every few minutes and checks some aspect of service health. They may transmit an e-mail to a monitoring mailbox using Exchange ActiveSync, connect to an RPC endpoint, or establish CAS-to-Mailbox server connectivity. All of these probes are defined in the Microsoft.Exchange.ActiveMonitoring\ProbeDefinition event log channel each time the Exchange Health Manager service is started. The most interesting properties for these events are:

• Name: The name of the Probe. This will begin with the SampleMask of the Probe’s Monitor.
• TypeName:The code object type of the probe that contains the probe’s logic.
• ServiceName: The name of the Health Set for this Probe.
• TargetResource: The object this Probe is validating. This is appended to the Name of the Probe when it is executed to become a Probe Result ResultName
• RecurrenceIntervalSeconds: How often this Probe executes.
• TimeoutSeconds: How long this Probe should wait before failing.

On a typical Exchange 2013 multi-role server, there are hundreds of these probes defined. Many probes are per-database, so this number will increase quickly as you add databases. In most cases, the logic in these probes is defined in code, and not directly discoverable. However, there are two probe types that are common enough to describe in detail, based on the TypeName of the probe:

• Microsoft.Exchange.Monitoring.ActiveMonitoring.ServiceStatus.Probes.GenericServiceProbe: Determines whether the service specified by TargetResource is running.
• Microsoft.Exchange.Monitoring.ActiveMonitoring.ServiceStatus.Probes.EventLogProbe: Logs an error result if the event specified by ExtensionAttributes.RedEventIds has occurred in the ExtensionAttributes.LogName. Success results are logged if the ExtensionAttributes.GreenEventIds is logged. These probes will not work if you override them to watch for a different event.

The basics of a recurrent probe are as follows: start every RecurrenceIntervalSeconds and check (or probe) some aspect of component health. If the component is healthy, the probe passes and writes an informational event to the Microsoft.Exchange.ActiveMonitoring\ProbeResult channel with a ResultType of 3. If the check fails or times out, the probe fails and writes an error event to the same channel. A ResultType of 4 means the check failed and a ResultType of 1 means that it timed out. Many probes will re-run if they timeout, up to the MaxRetryAttempts property.

The ProbeResult channel gets very busy with hundreds of probes running every few minutes and logging an event, so there can be a real impact on the performance of your Exchange server if you perform expensive queries against this event channel in a production environment.

Notifications

Notifications are probes that are not run by the health manager framework, but by some other service on the server. These services perform their own monitoring, and then feed data into the Managed Availability framework by directly writing probe results. You will not see these probes in the ProbeDefinition channel, as this channel only describes probes that are run within the Managed Availability framework.

For example, the ServerOneCopyMonitor Monitor is triggered by Probe results written by the MSExchangeDagMgmt service. This service performs its own monitoring, determines whether there is a problem, and logs a probe result. Most Notification probes have the capability to log both a red event that turns the Monitor Unhealthy and a green event that make the Monitor healthy once more.

Checks

Checks are probes that only log events when a performance counter passes above or below a defined threshold. They are really a special type of Notification probe, as there is a service monitoring the performance counters on the server and logging events to the ProbeResult channel when the configured threshold is met.

To find the counter and threshold that is considered unhealthy, you can look at Monitor Definitions with a Type property of:

· Microsoft.Office.Datacenter.ActiveMonitoring.OverallConsecutiveSampleValueAboveThresholdMonitor or

· Microsoft.Office.Datacenter.ActiveMonitoring.OverallConsecutiveSampleValueBelowThresholdMonitor

This means that the probe the Monitor watches is a Check probe.

How this works with Monitors

From the Monitor’s perspective, all three probe types are the same as they each log to the ProbeResult channel. Every Monitor has a SampleMask property in its definition. As the Monitor executes, it looks for events in the ProbeResult channel that have a ResultName that matches the Monitor’s SampleMask. These events could be from recurrent probes, notifications, or checks. If the Monitor’s thresholds are reached or exceeded, it becomes Unhealthy.

It is worth noting that a single probe failure does not necessarily indicate that something is wrong with the server. It is the design of Monitors to correctly identify when there is a real problem that needs fixing versus a transient issue that resolves itself or was anomalous. This is why many Monitors have thresholds of multiple probe failures before becoming Unhealthy. Even many of these problems can be fixed automatically by Responders, so the best place to look for problems that require manual intervention is in the Microsoft.Exchange.ManagedAvailability\Monitoring crimson channel. These events sometimes also include the most recent probe error message (if the developers of that Health Set view it as relevant when they get paged with that event’s text in Office 365).

There are more details on how Monitors work, and how they can be overridden to use different thresholds in the Managed Availability Monitors article.

Abram Jackson
Program Manager, Exchange Server

Occasionally I am asked the following question – how can I protect the messaging environment from a rogue administrator? There are essentially two concerns being asked in this question:

1. How do I protect the data from being deleted by a rogue administrator?
2. How do I protect the data from being accessed and/or altered by a rogue administrator?

Sometimes this discussion leads to a discussion about only the chosen backup architecture. The reality is that whether you implement Exchange Native Data Protection or a third-party backup solution, a backup, by itself, does not protect you from rogue administrators; it only mitigates the damage they potentially cause. Any administrator that has the privileged access to the messaging data (whether it be live data and/or backup data), can compromise the system. Therefore, some operational changes must be implemented within the organization in order to reduce the attack surface of an administrator who has gone rogue.

Important: This article is not intended to be a comprehensive set of instructions on how to restrict administrators. Instead, this article will highlight the principles and techniques that can be used.

Immutable Laws of Security

The Microsoft Security Response Center published the Ten Immutable Laws of Security. They are:

• Law #1: If a bad guy can persuade you to run his program on your computer, it's not solely your computer anymore.
• Law #2: If a bad guy can alter the operating system on your computer, it's not your computer anymore.
• Law #3: If a bad guy has unrestricted physical access to your computer, it's not your computer anymore.
• Law #4: If you allow a bad guy to run active content in your website, it's not your website any more.
• Law #5: Weak passwords trump strong security.
• Law #6: A computer is only as secure as the administrator is trustworthy.
• Law #7: Encrypted data is only as secure as its decryption key.
• Law #8: An out-of-date antimalware scanner is only marginally better than no scanner at all.
• Law #9: Absolute anonymity isn't practically achievable, online or offline.
• Law #10: Technology is not a panacea.

These guiding principles are a cornerstone in how we secure Office 365 and are equally applicable to on-premises deployments.

Active Directory

Exchange relies on Active Directory, storing configuration data within the configuration partition and storing recipient data within the domain partitions. The forest administrators, who are members of either the Enterprise Admins group or the root Domain Admins group, control all aspects of the directory and control data stored in the directory (though they sometimes delegate specific rights to others so others can perform very specific actions that are usually narrowly scoped within a forest). Likewise, domain administrators in each domain partition own their respective recipient data. Companies normally restrict forest-wide and domain-wide actions to be exclusively performed by forest administrators because of the risk that configuration changes can have broad adverse impact.

Many organizations today operate an Active Directory model with a least-privilege administrative model. If you are not operating in this fashion, this should be a top priority for your organization. Remember, any member of the Enterprise Admins or Domain Admins can alter messaging configuration settings on recipients and/or the Exchange configuration, without utilizing the Exchange Management Tools. For more information, see Best Practices for Securing Active Directory.

In addition to operating Active Directory in a least-privilege manner, it is important to implement background checks and other processes to determine the trustworthiness of your administrators, otherwise Law #6 cannot be mitigated. In addition, you may want to consider investing in an identity management solution, like Forefront Identity Manager, to manage and audit administrative permission requests and approvals.

Permissions

Exchange administrators have a wide variety of tools that they potentially can utilize to manage a messaging environment. The preferred method is via Remote PowerShellas Exchange can then authorize access and audit any actions performed. However, if an Exchange administrator is granted additional permissions in Active Directory (e.g., the ability to modify any attribute on a user), then the administrator can utilize other tools (e.g., LDP, ADSIEdit, local PowerShell, scripts, etc.) to modify recipient and/or configuration data, bypassing the authorization and auditing checks built into the system.

To effectively mitigate any possibility of modifications that could occur outside of the Remote PowerShell framework, it is imperative to follow Best Practices for Securing Active Directory, as previously mentioned. Often times, Exchange administrators are not evaluated with the same rigor as Active Directory administrators; therefore, Exchange administrators must not be granted permissions in Active Directory that allow them to circumvent Remote PowerShell (e.g., recipient modification).

If Active Directory privileges are accurately and narrowly scoped, a rogue administrator will have difficulty making unauthorized changes no matter which tool he/she tries to use.

PowerShell

Exchange leverages Remote PowerShell which is a feature within PowerShell that lets admins run commands on remote systems. Remote PowerShell enables the functionality utilized by Exchange to audit cmdlet execution and enforce authorization through RBAC.

One way to ensure that administrators can only manage the messaging environment through Remote PowerShell is to not allow the installation of the Exchange Management Tools on administrator workstations. Administrators are then forced to use the Exchange Admin Center or to connect to Exchange using Remote Shell.

In addition, PowerShell 3.0 and higher provides robust audit capabilities when module logging is enabled. PowerShell Module logging captures pipeline execution events for specified modules, and records this data in Windows PowerShellEvent log. Among the events logged, Event ID 800 (Pipeline Execution Details) provides command line commands and a script name if one is used. If a script was used, the ScriptName value will be populated with the file name of the script that was executed and an event will be logged for each line in the script, with each event including the command from that line. The data recorded in the event log can be collected, analyzed, and provide auditing data by which an organization can determine what PowerShell operations are occurring in the environment.

Mitigating Data Destruction

An Exchange administrator has the necessary privilege to access and destroy Exchange data. The best way to protect the messaging data from an administrator is to:

1. Shrink the window of opportunity for administrators to perform malicious activities. The mechanisms that can be implemented include removing local administrative access, deploying Applocker, removing access to destructive cmdlets, and deploying lagged database copies.
2. Ensure all administrative actions are logged and implement alerting and reporting based upon monitoring of logged activities.
3. Implement a least-privilege access control model whereby the elevation process grants access only to perform the intended activities. Even more effective is implementing an access control model whereby most or all administrative activities are done via a ‘control plane’ that is used by administrators to request that actions be performed against Exchange and the control plane can then implement business logic upon the request that will determine if the request will actually be executed. The business logic might include a request to a second person to review and approve the action or it might check an employee work scheduling system to see if the original requestor is on vacation, for example.

Removing Local Administrative Access

The majority of Exchange management tasks are accomplished via Remote PowerShell. As a result, the only time an Exchange administrator needs local administrative access to the Exchange server is to perform system updates (installing driver updates, installing Exchange cumulative updates, operating system updates, etc.). Therefore, local administrative access can be restricted and granted only when needed, for a specific period of time, after which access is revoked.

Without local administrative access, administrators will be unable to obtain certain information about the Exchange server health that they may need to appropriately manage the environment. Therefore, administrators should be granted access to the following:

• File shares (secured via read-only access to the appropriate security groups) can be created to enable access to Exchange diagnostic logs (which by default, are located at %Program Files%\ Exchange Server\V15\Logging and %SystemDrive%\inetpub\logs).
• Read-only access to the event logs can be granted by adding the appropriate security groups to the member server’s local Event Log Readers security group.
• Access to performance counters can be granted by adding the appropriate security groups to the member server’s Performance Monitor Users security group.
• To allow administrators to schedule performance counter logging, enable and collect tracing, the appropriate security groups can be added to the member server’s Performance Log Users security group.

In addition, local administrative access on the administrator’s workstations should also be removed. This ensures that administrators can only run approved applications and cannot bypass any security mechanisms you may put into place to audit and monitor their actions. For more information on running Windows in a least privileged manner, see Applying the Principle of Least Privilege to User Accounts on Windows.

By removing local administrative access, Laws #2 and #3 are mitigated.

AppLocker

AppLocker provides organizations with a strong defense in the prevention of malicious software and unapproved applications from affecting your server environment. AppLocker can be used to limit the software that Exchange operators can use so that only an approved list of applications (e.g., Exchange Management Tools, approved PowerShell scripts, etc.) will run on a server where AppLocker policy enforcement is in effect. For more information, see the TechNet article on AppLocker.

AppLocker allows you to mitigate Law #1.

Removing Access to Destructive Cmdlets

Both Exchange 2010 and Exchange 2013 include Role Based Access Control (RBAC), a mechanism by which administrators are authorized to perform certain administrative tasks. RBAC provides the capability for very granular control of managing the messaging environment – for example, restricting access to a particular cmdlet or to a particular property of a cmdlet.

For more information on RBAC, please see the following articles:

• Understanding Role Based Access Control
• RBAC and the Triangle of Power
• RBAC: Walkthrough of creating a role that can wipe ActiveSync Devices

While Administrator Audit Logging will record the actions taken by administrators, it does not prevent the administrators from performing destructive actions if they are authorized to do so. Using RBAC, organizations can reduce the attack surface area by removing access to destructive cmdlets. A destructive cmdlet is any cmdlet that can access, modify, or delete messaging data.

The below table identifies cmdlets and parameters that may be considered destructive (e.g., Remove-Mailbox). Each cmdlet should be evaluated to determine how it is used in your environment and whether it should be removed from day-to-day usage.

Note: The above table does not include the cmdlet list identified in the section “Removing Access to Data Access Cmdlets” below, but they should also be considered as those cmdlets provide access to messaging data.

For example, if I wanted to ensure that my administrators cannot remove database copies and manipulate the lagged database copy’s configuration, I could remove those cmdlets by using RBAC in the following manner:

1. Create two new role groups: Protected Organization Management and Protected Server Management.

   $ORoleGroup = Get-RoleGroup “Organization Management"
    
   New-RoleGroup "Protected Organization Management" -Roles $ORoleGroup.Roles
    
   $SRoleGroup = Get-RoleGroup “Server Management"
    
   New-RoleGroup "Protected Server Management" -Roles $SRoleGroup.Roles

2. Remove the Database Copies role from the protected role groups.

   Get-ManagementRoleAssignment -RoleAssignee "Protected Organization Management" -Role "Database Copies" -Delegating $false | Remove-ManagementRoleAssignment
    
   Get-ManagementRoleAssignment -RoleAssignee "Protected Server Management" -Role "Database Copies" -Delegating $false | Remove-ManagementRoleAssignment

3. Create a Protected Database Copies role.

   New-ManagementRole –Parent "Database Copies" –Name "Protected Database Copies"

4. Remove the Remove-MailboxDatabaseCopy role entry from the Protected Database Copies role.

   Remove-ManagementRoleEntry "Protected Database Copies\Remove-MailboxDatabaseCopy"

   Additionally, you can also remove access to the ReplayLagTime parameter from the Protected Database Copies role, thereby ensuring administrators cannot disable the lagged database copy.

   Set-ManagementRoleEntry "Protected Database Copies\Set-MailboxDatabaseCopy" –Parameters ReplayLagTime -RemoveParameter

5. Add the Protected Database Copies role to the protected role groups.

   New-ManagementRoleAssignment –SecurityGroup "Protected Organization Management” –Role “Protected Database Copies"
    
   New-ManagementRoleAssignment –SecurityGroup "Protected Server Management” –Role “Protected Database Copies"

6. Add users to the protected role groups.

   $OrgAdmins = Get-RoleGroupMember "Organization Management"
    
   $SrvAdmins = Get-RoleGroupMember "Server Management"
    
   $OrgAdmins | ForEach {Add-RoleGroupMember "Protected Organization Management" –Member $_.Name}
    
   $SrvAdmins | ForEach {Add-RoleGroupMember "Protected Server Management" –Member $_.Name}

7. Remove the desired users from the Organization Management and Server Management role groups.

You can repeat the appropriate steps for each destructive cmdlet that needs to be restricted. Use the following cmdlet to determine which roles contain the desired cmdlets:

Get-ManagementRoleEntry "*\<cmdlet>" | fl name,role

Lagged Database Copies

A lagged database copy is a rolling point-in-time (up to 14 days) copy of the database. Lagged database copies were first introduced in Exchange 2007 and have evolved considerably since then. Lagged database copies are part of The Preferred Architecture. While the primary reason for deploying a lagged database copy is protection against rare, catastrophic logical corruption events, lagged database copies can be used to recover mailboxes and/or items that have been deleted by a rogue administrator. By implementing the access control mechanisms previously discussed, the lagged database copy is protected from destruction by a rogue administrator.

Mitigating Data Access

There are several mechanisms you can implement to mitigate unwarranted data access within your messaging environment. These mechanisms include auditing, using Bitlocker to encrypt the disk drives, and removing access to certain cmdlets that enable administrators to gain access to user data.

Auditing

There are several different forms of auditing that can be implemented in the messaging environment. Within Exchange, you can enable Administrator Audit Logging. Administrator Audit Logging captures all operations that are performed within the Exchange Management Shell (EMS) or Exchange Admin Center (EAC).

By default, all cmdlets, except Get- or Search- cmdlets are logged in the audit logs. You can change the cmdlet logging via Set-AdminAuditLogConfig; for example, since Search-Mailbox includes the ability to delete content, you will want to add that cmdlet to the list. Audit logs are stored in an arbitration mailbox. Reports can be accessed via the Search-AdminAuditLog, New-AdminAuditLogSearch cmdlets, or via EAC.

In addition to auditing the actions taken by Exchange administrators managing the service, you will also want to audit server operation events (e.g., logon events). For more information, see the Windows Server Security Auditing Overview article and the Audit Policy Recommendations article for securing Active Directory.

Bitlocker

Another way to reduce the attack surface area is to use Bitlocker to ensure that operators with physical access to the servers cannot remove disk drives and access the data contained on them.

As mentioned previously, separation of roles is imperative, otherwise Law #7 cannot be mitigated. As Bitlocker recovery information is stored in Active Directory (specifically the msFVE-RecoveryInformation attribute), delegation of this data must not be granted to Exchange administrators.

Removing Access to Data Access Cmdlets

Using RBAC, organizations can reduce the attack surface area by removing access to cmdlets that allow access to mailbox data. The below table identifies cmdlets that grant access to data, or enable administrators to hide their tracks. Each cmdlet should be evaluated to determine how it is used in your environment and whether it should be removed from day-to-day usage.

To remove access to the above cmdlets, follow steps 1-7 from the “Removing Access to Destructive Cmdlets” section.

Requesting Elevation

Over time, administrators will require access to restricted cmdlets to perform a necessary operation (e.g., disabling mailboxes, removing unnecessary transport rules, etc.). There are many ways you could go about this. For example, you could simply build RBAC roles for each individual cmdlet (and/or property) that you restrict; when elevation is required, you add the administrator to the appropriate role group, granting them access, and then remove the administrator when the task is complete. Or you could develop a process and workflow that includes the following:

1. A means by which an administrator may submit a request for elevated access. The request needs to include specifics like the cmdlets being requested, the targeted servers/users/etc., and the length of that time elevated access is required.
2. The request can either be implicitly approved based on the requested action, or it can require human approval.
3. All actions must be logged once elevated access has been granted.
4. Elevated access must be removed once the time period has expired (either based on the request or based on a default time period for access elevations) or the task has been completed.

How is this addressed in Office 365?

Exchange Online implements a variety of mechanisms to prevent rogue administrators from accessing or destroying data.

Perry Clarke and Vivek Sharma recently discussed Lockbox, a mechanism we use to enforce no standing access, in the article From Inside the Cloud: Who has access to your data within Office 365?

In particular, Exchange Online personnel do not have persistent access to the service or servers; instead, administrators (who have undergone background checks) have to request specific access (to servers, cmdlets, etc.) and can only perform the requested tasks during a specified timeframe. Additionally, for requests for elevation to a role with access to customer data, approval must be performed by another human being and the ability to approve this type of request is restricted to a smaller set of more senior personnel.

We also use other mechanisms to protect the service and customer data. For example, Bitlocker is used to encrypt all disks that contain customer data. When the disks are end-of-life they are shredded, thereby ensuring the data cannot be accessed.

Conclusion

While this article covers the basics, there are many other mechanisms you can implement in your environment to reduce the surface attack area within your messaging environment. For more information about security mechanisms protecting Exchange Online that can be used in on-premises deployments, see the MEC session Exchange Online service security investments: you CAN and SHOULD do this at home.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience

One of the nice features introduced in Exchange 2010 was Admin Audit Logging. Concerned administrators everywhere rejoiced! This meant that a record of Exchange PowerShell activity, organization wide, was now saved and searchable.

Administrators could query the Admin Audit Log, using the Search-AdminAuditLog Cmdlet, and reveal any CmdLets invoked, the date and time they were executed and the identity of the person who issued the commands. However, the results of the search are a bit cryptic and it didn’t allow for easy bulk manipulation like parsing, reporting or archiving.

The main complaint I heard from customers went something like this: “It’s great that I can see what Cmdlets are run, and what switches were used… but I can’t see the values of those switches!” Well, as it turns out, that data has actually been there the whole time; it’s just been stored in a non-obvious manner.

Consider a scenario where you’ve been informed that many, or all, of the mail users in your organization are reporting the wrong phone number listed in the Global Address List. It seems everyone has the same phone number now, let’s say 867-5309.

Because your organization uses Office 365 Directory Synchronization (DirSync), you know the change had to occur within your on-premises organization and was then subsequently synchronized to Office 365. The Search-AdminAuditLog Cmdlet must, therefore, be run on-premises.

It’s important to remember this concept. If you were investigating a Send Connector configuration change for your Office 365 – Exchange Online tenant, a search would need to be performed against your tenant instead. But let’s get back to our Jenny Phone number issue.

You know that the change was made on the 6^th so you restrict the search to that date.

Search-AdminAuditLog -StartDate "4/6/2015 12:00:00 AM" -EndDate 4/6/2015 11:20:00 AM"

(click on screenshots that might be too small to read)

Reviewing the output, you find that Tommy executed the Set-User Cmdlet but no indication as to what parameter(s) or values were used? What exactly did Tommy run? Where are the details!?

Then, you spot a clue. The ‘CmdletParameters’ and ‘ModifiedProperties’ are enclosed with braces { }. Braces are normally indicative of a hash table. You know a hash table is simply a collection of name-value pairs. You wonder if you’re only seeing the keys or a truncated view in this output. Could more details remain hidden?

Digging a bit deeper, you decide to store the search results to an array, named $AuditLog, which will allow for easier parsing.

$AuditLog = Search-AdminAuditLog -StartDate "4/6/2015 12:00:00 AM" -EndDate "4/6/2015 11:20:00 AM"

Next, you isolate a single entry in the array. This is done by calling the array variable and adding [0] to it. This returns only the first entry in the array.

$AuditLog[0]

To determine the object type of the ‘CmdletParameter’, you use the GetType() method and sure enough, it’s an array list.

$AuditLog[0].CmdletParameters.GetType()

Finally, you return the CmdletParameters array list to reveal all the details needed to conclude your investigation.

$AuditLog[0].CmdletParameters

Considering there are hundreds or thousands of entries in the audit log, how would you generate a full list of all the objects Tommy has changed? Or better yet, report all objects that he changed where ONLY the ‘Phone’ attribute was modified?

Fortunately, you don’t have to expend too much time on this. My colleague, Matthew Byrd recognized this exact problem and he wrote a PowerShell Script that does all the aforementioned steps for you and then some!

The script can be downloaded from TechNet Gallery and you’ll find it’s well documented and commented throughout. The script includes help (get-help .\Get-SimpleAuditLogReport.ps1) and can be used within Exchange 2010, Exchange 2013 and Office 365 - Exchange Online environments. That said, I’m not going to dissect the script. Instead, I will demonstrate how to use it.

The script simply manipulates or formats the results of the Search-AdminAuditLog query into a much cleaner and detailed output. You form your Search-AdminAuditLog query, then pipe it through the Get-SimpleAuditlogReport script for formatting and parsing.

Here are some usage examples:

This first example will output the results to the PowerShell Screen.

$Search = Search-AdminAuditLog -StartDate "4/6/2015 12:00:00 AM" -EndDate "4/6/2015 11:20:00 AM"
$Search | C:\Scripts\Get-SimpleAuditLogReport.ps1 –agree

You can see that the Get-SimpleAuditLogReport.ps1 script has taken results stored in the $Search variable and attempted to rebuild the original Command run. It isn’t perfect but the goal of the script is to give you a command that you could copy and paste into an Exchange Shell Window and it should run.

Should you expect a lot of data to be returned or wish to save the results for later use, this example will save the results to a CSV file.

Search-AdminAuditLog -StartDate "4/6/2015 12:00:00 AM" -EndDate "4/6/2015 11:20:00 AM"| C:\Scripts\Get-SimpleAuditlogReport.ps1 -agree | Export-CSV -path C:\temp\auditlog.csv

This example uses one of my favorite output objects, Out-GridView, to display the results. This is a nice hybrid CSV/PowerShell output option. The results shown in the Out-GridView window is sortable and filterable. You can select, copy/paste the filtered results into a CSV file. Meanwhile the raw, unfiltered, results are saved to a CSV file for future later use or archival.

Search-AdminAuditLog -StartDate "4/6/2015 12:00:00 AM" -EndDate "4/6/2015 11:20:00 AM"| C:\Scripts\Get-SimpleAuditlogReport.ps1 -agree | Out-GridView –PassThru | Export-Csv -Path c:\temp\auditlog.csv

Here I restrict it to only commands Tommy ran and remove anything that he ran against the discovery mailbox since it is a system mailbox.

Copy/Paste the filtered results into a CSV file. The Out-GridView has no built in export or save feature. To save your filtered results, click on an entry and then ctrl-a / ctrl-c to select all and copy results to your clipboard. Finally, in Excel, paste and you’re done.

There you have it. Admin Audit Log Mastery – CHECK! Thanks to Matthew Byrd’s wonderful script you can get the most out of your audit logs. Check it out over at TechNet.

Brandon Everhardt

Coming to the next CUs for Exchange 2013 and Exchange 2016 there are some changes to how the Exchange Management Shell (EMS) connects to Exchange. In previous versions, we have seen that customers who were reliant on a load balanced solutions for third party apps and scripts may get routed to a non-Exchange 2016 server. This would lead customers to some broken administrative experiences based on the reliance of Exchange 2016 cmdlets and features. Today we will dive deeper into these changes to help the Exchange Administrator understand how these changes will affect their Exchange Organization.

In Exchange 2010 we used session affinity to provide a long-standing association between a particular client and a particular Client Access Server. With Exchange 2013 and continuing on into Exchange 2016 we have removed the requirement for session affinity. Also with the release of Exchange 2016 we added an ability to proxy between Exchange 2013 and Exchange 2016. This allowed us to have an Exchange 2013 server public facing and proxy traffic back to an Exchange 2016 server. So in order to ensure that when a connection to the Exchange Management Shell (EMS) always goes to an Exchange 2016 server we have introduced mailbox anchoring into Exchange 2013 CU11 and Exchange 2016 CU1.

In mailbox anchoring, CAS locates where the mailbox resides by querying Active Directory for the user's mailbox GUID. As soon as we obtain the GUID, HTTP Proxy refers to Active Manager to locate the database containing the user's mailbox. Once CAS knows where the user's mailbox is located, the protocol request is then proxied to the appropriate mailbox server. If that server goes down and the database is a part of a DAG, the Client Access Proxy will proxy the session to the new active mailbox server for the corresponding database. This process is utilized currently for most client protocols, such as OWA, ECP, and EWS with the exception of Remote PowerShell.

This is great for the client but what about the Exchange Management Shell (EMS)? In Exchange 2013 RTM through CU10 and Exchange 2016 (RTM), we do not use the mailbox anchoring logic to proxy the PowerShell session we just connect to the local server or proxy to another available server in the Exchange Organization. This means if I log into an Exchange 2013 Server I cannot manage any new cmdlets that became available in Exchange 2016. For that I would have to logon directly to an Exchange 2016 Server.

Starting with Exchange Server 2013 CU11 and Exchange Server 2016 CU1, the Exchange Management Shell (EMS) session will also be using mailbox anchoring. If the aforementioned environment has all servers upgraded to Exchange 2013 CU11 and Exchange 2016 CU1 the behavior of Exchange Management Shell changes. Now when a user logs on to the Exchange Server and open up EMS, the session will be proxied to the server where the user’s mailbox is located. If the user does not have a mailbox, we will utilize the arbitration mailboxes for the mailbox anchoring logic. Wherever the arbitration mailbox is mounted is where the EMS session will be proxied.

It is important to understand when you upgrade your existing Exchange Server 2010/Exchange Server 2013 organization to Exchange Server 2013/Exchange 2016, you must move the arbitration mailboxes to a database on highest version of Exchange Mailbox server. Same applies to administrators that have mailbox associated. You should move the mailbox after installing and verifying Exchange was working properly. If you don’t move the arbitration mailbox to higher version of Exchange, you may run into issues like not getting version appropriate tasks or tasks not being saved to the administrator audit log when you run the Search-AdminAuditLog cmdlet. We primarily use the Arbitration mailbox “SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}” however if that mailbox is unavailable we can use the other arbitration mailboxes to make our connection.

So let’s cover how to handle a couple of scenarios that you may run into.

Installing Exchange 2016 into an Exchange 2013 Organization

After you complete the installation of your first Exchange 2016 server and verify that everything is working properly you need to move all arbitration mailboxes to Exchange 2016. To do so you can use the Exchange admin center or Exchange Management Shell and move all of the Microsoft Exchange Migration, Federation, and System Mailboxes.

To move all the arbitration mailboxes via Exchange Management Shell we would run:

[PS] C:\PowerShell> Get-Mailbox -Arbitration | New-MoveRequest -TargetDatabase <Exchange_2016_DataBase>

To move all the arbitration mailboxes via Exchange Admin Center we would run:

1. In the EAC, go to Recipients > Migration.

2. Click New Add Icon, and then click Move to a different database.

3. On the New local mailbox move page, click Select the users that you want to move, and then click Add Icon.

4. On the Select Mailbox page, search for Microsoft Exchange add all the mailboxes that are in the list below:

5. Click OK, and then click Next.

6. On the Move configuration page, type the name of the migration batch, and then click Browse next to the Target database box.

7. On the Select Mailbox Database page, add the mailbox database indicating where you want to move the system mailbox. Verify that the version of the mailbox database that you select is Version 15.1, which indicates that the database is located on an Exchange 2016 server.

8. Click OK, and then click Next.

9. On the Start the batch page, select the options to automatically start and complete the migration request, and then click New.

How do you know which server you are connected to?

The Exchange Management Shell console will always display the name of CAS server that received the initial connection. However, the connection may be proxied to a different mailbox server in the background. You can use following trick to identify the mailbox server to which the connection is being proxied to.

Open up Exchange Management Shell, and run a simple query to determine which Exchange Server you are connected to:

As you can see from the above EMS Session even though I was on the host E1501 when I ran the cmdlet Get-ExchangeCertificate it returned the certificate for server E1503. If you want to dig deeper you can go into the logging directory for the HTTPProxy PowerShell and see what server, you proxied to. (Before you can search in this path you will need to open the path in File Explorer to take ownership)

As you see above we proxied the connection to e1503.contoso.com. Since your PowerShell connection is actually running on the server e1503, therefore all cmdlets run will be against that server. If the cmdlet you are using supports the -server switch, this can be utilized to ensure which server the cmdlet is run against. For example, if I run Get-PowerShellVirtualDirectory I can specify which server I want to communicate with.

Fail Overs

If you have an admin mailbox and the database that hosts the mailbox or DAG that the mailbox is in is down, we will then proxy to the database that has the arbitration mailbox (which is why we recommend moving this mailbox as noted before). This way cmdlets can always be run against the highest version Exchange server in the organization.

AD Site Proxying

In a multisite or geographically dispersed datacenters we will proxy the EMS session back to where the admin mailbox is mounted. In most scenarios this is fine, however there may be some tasks that would require us to have an administrator in the local site. In order to accomplish this, you will need to have either the arbitration mailboxes in the site and logon to the server with an administrator account that has no mailbox or you can create an admin account with a mailbox in the corresponding site.

Critical Failures

If you open EMS and are not able to connect to any Exchange Servers due to loss of communication, then you will need to add the PSSnapin for Exchange to your local PowerShell Session.

Rob Whaley
Beta Engineer for Exchange and Office 365