Tag Archives: Exchange 2010

Exchange 2010 SP2 – A Warning

You may remember my post a while back about issues with applying Exchange 2010 SP1 in situations where you were using Group Policy to control Powershell Execution policies. Specifically, this issue occurs because the Group Policy setting uses the WMI service to enforce the Execution Policy and as part of the Exchange install/upgrade process, the WMI service is stopped, causing the Execution Policy to revert to Restricted and the following error to pop up and the install to fail:

The following error was generated when "$error.Clear();
& $RoleBinPath\ServiceControl.ps1 EnableServices Critical
" was run: "AuthorizationManager check failed.".

Well it turns out that this still applies with Exchange 2010 SP2 in exactly the same fashion.

The (well, A) relevant KB article is here but the “workaround” is a bit half-assed to be honest and you’re much better off just disabling the associated Group Policy setting and configuring the Execution Policy locally (with set-executionpolicy) to either AllSigned,RemoteSigned or Unrestricted for the duration of the upgrade.

Why Microsoft cannot add installer logic to check for this possibility, especially given how long it’s been a potential problem, is beyond me but then I’m not an Exchange developer.

ProTip: Outlook 2003 & Public Folders

As you probably know, Outlook 2003 and older use Exchange Public Folders for their Free/Busy data and Offline Address Book. You may also know that Exchange 2007 and Exchange 2010 have deprecated Public Folders in favour of an HTTP-based distribution method for the data, which Outlook 2007 & 2010 fully support.

One could then reasonably conclude that removing public folders in an Exchange 2007 or Exchange 2010 environment might negatively affect Outlook 2003 and older clients. This is technically correct, though the reality is slightly more dramatic; it completely blocks Pre-Outlook 2007 clients from connecting to your Exchange servers and presents the users with a handy “Your administrator has blocked this version of Outlook from connecting” message.

What this all means is that if your support team have spent the last year lying to you every time you’ve asked them how they’re progressing on upgrading all the remaining Outlook 2003 installs to 2007/2010 then when you remove the last public folder store from your Exchange environment you’ll suddenly have hundreds of people whose can no longer access their emails. This, for some reason, upsets them.

The moral of this story is that you should never trust what people tell you, because they’re almost always lying bastards, and make sure you verify the information for yourself before making any changes. Yes, it’s a lot of work and no, you shouldn’t have to do it, but it’s always you that gets the flak when it all hits the fan.

You must close all dialog boxes before you can close Exchange Management Console

Update: There is a hotfix now available to address this issue, though you have to ring MS support directly to get hold of it at the moment. The plan is to roll it into an IE9 update in the near future. More details can be found here

There is a known issue with the Exchange 2007 and Exchange 2010 Management Consoles on machines with Internet Explorer 9 installed. This issue causes you to recieve the error: “You must close all dialog boxes before you can close Exchange Management Console.” when trying to close the EMC even though there aren’t any dialog boxes open.

As it stands, there are several workarounds: You can uninstall IE9, you can kill the MMC process from task manager every time and a few people have reported success with turning off “Internet Explorer Enhanced Security Configuration” for Administrators and Users (Servers only) and then adding https://localhost to the “trusted sites” in IE.

According to the Senior Program Manager for Internet Explorer Product Quality, Mark Feetham:

[...]we do now understand what’s occuring between the EMC, MMC and MSHTML. We have enough information to complete the investigation and have moved to looking at ways to improve this situation.

Rumoured ETA for a fix is Q4 2011.

Exchange 2010 Multiple Domains: Revisited

In a rare follow up to a previous post on the subject of multi-domain Exchange 2010 configurations, I have just come across the following information, which would have saved me an awful lot of time and effort if I’d known about it when I started (or, you know, it was clearly documented anywhere):

When using the Exchange 2010 Management Shell, the default recipient scope is set to domain-level. This means only local mailboxes will be listed when running something like the Get-Mailbox cmdlet. In order to change the recipient scope to forest-wide, you must run the following command: Set-ADServerSettings -ViewEntireForest:$true

Looks like you have to run it per session, however.

Easy when you know how…

On Exchange 2010 Retention Policies

As per this Technet Blog entry, Outlook doesn’t know what it’s talking about when it comes to retention policies if you’re running in Cached Exchange Mode. This means that even though the Exchange server is correctly applying your policy, Outlook may well report that your items have expired when they haven’t.

…The conclusion here is that if you’ve applied Retention Policies to any of your folders and the items are being shown by Outlook as expired, but are still there days later, take a quick look at them in OWA or an online mode Outlook profile and see if they really are expired.

Straightforward, understandable, but still annoying :)

Exchange 2010 EMC and Non-Existent Servers

It would appear that the Exchange 2010 EMC isn’t particularly bright; when you launch it, it picks a CAS to connect to from AD. This is fine.

However, should that server cease to exist, by which I mean Exchange is uninstalled and it is properly decommissioned, then the EMC will continue to try and connect to it. Even after the connection fails, it’ll keep on merrily plugging away at the non-existent server, never considering that there are probably other servers it could try.

This is very annoying and seemingly very stupid behaviour. To work around it, close the EMC, fire up your registry editor of choice, locate the following key: HKCU\Software\Microsoft\Exchangeserver\v14\AdminTools\ and delete the NodeStructureSettings value. This will reset the EMC and cause it to pick a new CAS to connect to; it may also affect other settings that you’ve changed in the console.

Another option is to close the EMC, navigate to C:\users\<username>\AppData\Roaming\Microsoft\MMC\ and delete the Exchange Management Console file. This will also reset the EMC and will definitely reset any customisations you’ve made to the console.

Why you should ever have to do this is something of a mystery to me, perhaps Microsoft just never expected anyone to decommission an Exchange server once it was built.

Monitoring Exchange 2010 DAG Status with Nagios

This one is nice and simple and allows you to monitor the health of your Exchange 2010 DAG via Nagios. For this, you will need:

Configure Nagios
Make sure you’ve got the Check_NRPE plugin in your libexec folder then add a new command definition to the commands.cfg like so:

define command{
	command_name    check_exrep
	command_line    $USER1$/check_nrpe -H $HOSTADDRESS$ -u -t 120 -p 5666 -c check_exch
	}

Then setup service definitions and hosts/hostgroups as you would normally.

Configure NSClient++
In your [NSClient++ Folder]\Scripts folder, create a new powershell script file (.ps1) called “exrep.ps1” and put the following code inside (Replacing the two sections in [] to match your environment):

Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010
 
$Status = Get-MailboxDatabaseCopyStatus -server [Servername]
 
$flag = 0
 
foreach($State in $Status){
 
	if(($state.status -match "Mounted") -or ($state.status -match "Healthy")){
		$content = $($state.name)+": "+$($state.status)
		$output += $content+" - "
	}else{
		$content = $($state.name)+": "+$($state.status)
		$output += $content+" - "
		$flag = 1
	}
 
}
 
$output = $output.trimend(" - ")
$output = $output.replace("\[Servername]","")
 
write-host $output
 
if($flag -eq 0){
	exit 0
}else{
	exit 2
}

Next, open up your NSC.ini file and uncomment the “CheckExternalScripts.dll” line. In the [External Scripts] section, create a new entry for “check_exch” that points to: cmd /c echo [NSClient++ Folder]\scripts\exrep.ps1 | powershell.exe -noprofile -nologo -command -

Finally, restart the NSClient++ service on the client machine and restart Nagios on the server. When your check next runs, if any of your storage groups are not in a Healthy or Mounted state your should get an output that looks like: [Database Name]: [Status]. Repeat the above for each server that is part of the DAG.

Exchange 2010 Bits and/or Pieces

Once more I find myself in the trenches, battling bravely with Exchange 2010 in a vain attempt to beat it into submission. As there’s nothing worse than coming across problems that, when Googled, return only a handful of results, all asking the same thing and without any clear or satisfactory answer, I am providing the following bits in the hope that it might help others in the same situation.

First up, thanks to: flamingkeys.com; if you’ve got Receive Connectors with lots of Remote IP Ranges (Usually to allow them to relay mail externally through Exchange) then the following will make it nice and easy to copy said ranges when you create a new connector. The AuthMechnaism & PermissionGroups take any combination of the below listed options, comma-separated, of course:

New-ReceiveConnector [Connector Name] -Server [Servername] -Bindings [Server IP]:25 -RemoteIPRanges (Get-ReceiveConnector "[Existing Servername]\[Existing Connector Name]").RemoteIPRanges -AuthMechanism [None,TLS,Integrated,BasicAuth,BasicAuthRequireTLS,ExchangeServer,ExternalAuthoritative] -PermissionGroups [None,AnonymousUsers,ExchangeUsers,ExchangeServers,ExchangeLegacyServers,Partners,Custom] -MaxMessageSize [Size with B/KB/MB/GB]

Next, certificates. For no good reason, Microsoft changed the behaviour of the New-ExchangeCertificate and Import-ExchangeCertificate cmdlets from their 2007 versions in order to make them much harder to use; so, to request a new cert:

New-ExchangeCertificate -GenerateRequest -KeySize 1024 -SubjectName  "c=[Country Code],s=[State],l=[City],o=[Org],ou=[Department],cn=[Common name]" -DomainName [Comma separated list of hostnames for SAN] | Set-Content -Path "[Path for CSR Txt file]"

Then, to import and enable the new certificate:

Import-ExchangeCertificate -FileData ([Byte[]]$(Get-Content -Path [Path to Certificate] -Encoding byte -ReadCount 0))
Enable-ExchangeCertificate -Thumbprint [Thumbprint of certificate] –Services [IIS,SMTP,IMAP]

To make use of the handy GAL photographs, first have a read of this from the Exchange Team Blog then, use this cmdlet to upload a photo:

Import-RecipientDataProperty -Identity [Mailbox Name] -Picture -FileData ([Byte[]]$(Get-Content -Path [Path To Image] -Encoding Byte -ReadCount 0))

And this one to remove it:

Set-Mailbox [Mailbox Name] -RemovePicture

Hopefully someone will find this useful, I’ll add more stuff as I stumble across it.

Exchange 2010 RBAC Errors

Update: This turned out to be a Nagios-related powershell script running against Exchange that was being launched by a service running as LocalSystem, which didn’t have permissions to perform various tasks within Exchange. As soon as we stopped running the script the errors went away. Still no idea why the errors were popping up on servers in the Org that weren’t referenced by the task, but that’s Exchange for you.

Right, I’m throwing this out on the tiny off-chance that anyone has come across it and knows of a solution, because so far, Microsoft support haven’t and don’t.

Frequent entries in the Application logs of all Exchange 2010 Servers as follows:

(Process w3wp.exe, PID <PID>) “RBAC authorization returns Access Denied for user <Mailbox Server Computer Account>. Reason: No role assignments associated with the specified user were found on Domain Controller <Domain Controller FQDN>”

Several things.

1) Everything in <> has obviously been changed by me to remove details of my internal infrastructure, the actual errors contain real PID, account and server values. In all cases, the computer account is that of the Mailbox server, even though the error shows up on Mailbox, CAS and UM servers.

2) This is not, I repeat, not the same issue as you’ll find all over Google with a very similar error message that features a user account rather than a computer account. That one is usually caused by people not setting up permissions for their administrators properly in the ECP or broken permissions inheritance on accounts.

3) This error has survived a complete rebuild (OS and Exchange) of the Mailbox server, a re-running of the domain/forest prep tools and a couple of weeks examination by Microsoft Support. We’re currently looking at rebuilding all the other 2010 servers to see if it survives that too.

Any suggestions will be gratefully accepted.

Exchange 2010 High Availability: Not Really That Available

As you may know, all of the Microsoft engineers involved with creating Exchange 2007 were tragically afflicted by sudden onset amnesia, which prevented them from remembering any of the new features or technology they had developed for it when they came to start work on Exchange 2010. This may seem like an unlikely occurrence, but it is the only possible scenario that could explain why Microsoft made the design decisions that they did.

In an Exchange 2007 HA scenario, such as CCR, your clients would point to the mailbox server cluster name. This cluster name would always point at the Active node of your cluster so that no matter your failover situation, your clients could connect to their respective databases. Now you still had to make sure that there were CAS and HT servers available for each AD site with a Mailbox server for mail flow and OWA to work correctly, but that was easily achievable (albeit with a bit of registry fiddling to force the sitename if the servers weren’t actually in the right site) and even if you didn’t have them, at least users could still access their mailboxes and queue messages in their outbox for delivery when everything came back up again.

Leap forward to Exchange 2010; Microsoft, in their infinite wisdom, have changed things so that now the clients connect to the CAS servers and not the mailbox servers. The CAS server they connect to is based on the RPCClientAccessServer attribute on the mailbox database that their mailbox resides on. Microsoft recommend that you use NLB to cluster CAS servers and then modify the RPCClientAccessServer attribute to point to the NLB name, otherwise if you lose a CAS server all the clients pointing to it will break. Of course, you can’t NLB across subnets, so if you’re replicating your mailboxes off site (And you’d be stupid not to if you can) then you can’t have CAS-resiliency without stretch VLANs or other nasty cludges. Microsoft’s actual recommendation for this scenario is to manually update the DNS record for the unreachable CAS array to point to the one on your other site, which is fine if it happens in-hours and you actually know about it as it happens rather than 10 minutes later (or even the next morning) as the support calls come in. Don’t worry though, they have a solution for that too; just buy, install and configure MSCOM to monitor the Mailbox server and alert you when it fails over.

There was a half-arsed fix planned for SP1 (It would have automatically updated the RPCClientAccess value to point to the CAS array on your other site), but your users would have had to restart Outlook before it would read the updated attribute, so it wouldn’t have been a huge improvement and in any case it didn’t make it into the service pack. There’s no ETA that I’m aware of.

So, to summarise, in a multi-site environment with Exchange mailbox clustering, Microsoft have managed to go from Exchange 2007 with a fully functional HA solution to Exchange 2010 with an HA solution that’s only fully functional if you’ve purchased their monitoring products, have staff on-hand 24/7 to make DNS changes or don’t mind extensive user disruption if you have a site or server failure.

Honestly, for all its new features and improvements, Exchange 2010 is easily the worst-implemented version of Exchange since 5.5; it’s like they decided that anything that would help with a smooth transition from 2007 was entirely too much effort and that anyone who deviates from the Exchange infrastructure that Microsoft envisioned should be punished for daring to do so. I’m genuinely regretting starting the 2007->2010 transition now; it’s caused immeasurably more hassle than is justified by the benefits it provides.