Wednesday, November 30, 2005

The BizMD Mike Hulser leaving CTA

It has been an "interesting" 6 months at CTA. Today, the BizMD himself, Mike Hulser announced he was leaving CTA 6 months before his cushy consulting gig was up. I figured I would immortalize this moment. If anyone were left, there would be dancing in the streets.

Tuesday, November 29, 2005

Microsoft announces that Exchange 12 will be 64-bit only

This is actually old news now, but at the Microsoft IT Forum 2005, Microsoft confirmed some rumors that have been floating around that Exchange 12 will be 64-bit only. So, yes, the decision has been made that Exchange 12 will run only on a 64-bit platform; there will NOT be a 32-bit version of Exchange 12. Microsoft is in a precarious position of delivering improved support for some of their clients and delivering a stable platform for all customers. I'm afraid that supporting two different platforms may make the product much more difficult to be supported.

Microsoft has taken some heat in the industry from pundits like Ed Brill as well as the media on this decision, however I'm not so sure this is a bad thing.
  • Most of the server-class hardware that is shipping today is already capable of running Windows 2003 64-bit operating systems. That, unfortunately, does NOT include equipment in my own lab. So before betas start shipping I will need to get my hands on some 64-bit gear if I'm going to do any testing. :-(
  • The inital performance testing on the early versions of Exchange 12 running on 64-bit hardware found a 75% performance improvement in I/O when compared with the 32-bit version. This is pretty dramatic considering that most Exchange bottlenecks now (after you toss 4GB of RAM and 4 CPUs at a server) end up being I/O.
  • Exchange 12 will probably not be available anytime before mid 2007. By that time, most organizations that have done a hardware refresh will have 64-bit hardware.
  • Most of the time, organizations that upgrade from an older version to a newer version upgrade their hardware platform at the same time they upgrade the application.
  • Hopefully the information store service will be completely re-written to overcome some of the memory limitations on that platform (right now it cannot access over 2GB of RAM).
  • The announcement now will ensure that 3rd party vendors such as antivirus and backup vendors get their 64-bit products ready for shipping sooner rather than later.

I think by the time Exchange 12 ships, this will all have blown over and the industry will be much more likely to embrace 64-bit applications.

Sunday, November 27, 2005

Public folder cleanup in Exchange 5.5

I recently had a situation where we had seperated forever some Exchange 5.5 sites. They were never going to be rejoined for directory replication. Never, ever, ever! After separating them, we found a lot of public folders that we could not take ownership of (using the Exchange 5.5 server's DS/IS Consistency Adjuster and choosing to rehome public folders from unknown sites).

Every time we tried to view the properties in the Exchange Administrator, we got a message indicating that replication had not yet completed. You get the following message:

The object canot be found in the directory.
This may be because replication has not completed.
Microsoft Exchange Directory ID no: c1010aae

It turns out that the folders we could not take control of had the "Limit Administrative Access to Home Site" property enabled. This is pretty common in most Exchange 5.5 organizations. The whole reason that feature exists in the first place is to keep someone from disconnecting their site from directory replication and taking ownership of all of the folders in the hierarchy. Trust me, it happens. :-0

These "orphaned" but unmanagable folders will automatically be cleaned up during online maintenance after they have been orphaned for 20 days after the removal of the Exchange 5.5 site in which the folder lived. Each Exchange 5.5 server will clean these up after the counter hits 20 days, so some patience is in order.

However, it is possible to speed this process up somewhat. In the following registry key, create a REG_DWORD value and set the data to 900 decimal (15 minutes):

Then wait for online maintenance to complete. If you have more than one Exchange 5.5 server with public folders, you will want to do this on each server. Do NOT run this if you suspect that you will be rejoining the site back to the organization again.

Wednesday, November 16, 2005

Slides from recent presentations

As promised, I'm posting my Powerpoint slide decks from a number of recent presentations have done.

Exchange Connections
- Clustering for Exchange 2003
- Anatomy of a Disaster Recovery
- Exchange 5.5 to Exchange 2003 Migrations
- Outlook 2003: Is RPC over HTTP right for you?

Windows IT Pro
- Planning Scheduled Downtime

Tuesday, November 08, 2005

Windows IT Pro Web Seminar: Managing and Reducing Planned Downtime

On November 16 at 2:00pm EST, I am participating in a web cast sponsored by Windows IT Pro magazine and XOsoft. The topic is Managing and Reducing Planned Downtime; my specific focus for my part will be on the reasons for planning scheduled downtime for Exchange, choosing a downtime window, and how much time you need for your downtime window. Topics include:
  • Why you need a scheduled downtime window
  • About scheduling downtime to prevent unscheduled downtime
  • Which types of activities should be included in scheduled downtime
  • What questions you should ask when choosing a downtime window
  • About the non-technical tasks involved in scheduled downtime
  • How to decide what your sensitivity is to downtime based on your availability requirements
  • About products and services that can reduce or minimize downtime while still providing the ability to do required maintenance

You can register for this web seminar on Windows IT Pro's Web Seminars page. I hope you will attend.

Dialtone restores,the Recovery Storage Group, and swapping out database files

In my Analysis of Disaster Recovery session as well as a couple of other sessions, there were a lot of good questions on using the Recovery Storage Group. Someone asked if you can do a dial-tone restore, then restore the production database to an RSG, then swap out the RSG database with the "dial-tone" database, and then finally ExMerge the data from the dial-tone database in to the production database.

This is a good idea to do, since the dial-tone database will not have a lot of the mailbox "meta-data", such as mailbox/folder rights, rules, forms, etc... This data will be in the RSG database, though. So swapping them out once the RSG database is restored from tape, is a good idea.

I had not done this, but it is possible. I found some good reference material in the Microsoft whitepaper "Using Exchange Server 2003 Recovery Storage Groups", including steps on how to swap out the RSG with the dial-tone database. Essentially, here are the steps necessary if you lose the production database and want to do a dial-tone restore:
  1. Delete the production database files and remount the store. Answer "yes" when Exchange System Manager asks if you want to creatre new database files
  2. At this point, users can go back to work, albeit with empty mailboxes.
  3. Create a Recovery Storage Group and create the mailbox store in the RSG that you wish to restore.
  4. Restore the last backup of the production database to the RSG
  5. Make a backup copy of the production database that was created from the dial-tone restore
  6. Dismount both the production database and RSG database. Copy the RSG files in to the production database location. Copy the production database files in to the RSG location.
  7. Rename the production database (now in the RSG folder) to the name of the RSG database. Rename the RSG database (now in the production database folder) to the name of the RSG database.
  8. On the database properties, click the checkbox that allows the database to be restored from backup.
  9. Mount both databases.
  10. Merge the data in the RSG database (the dialtone data) in with the production database.

While this is just an overview, it gives you an idea of what the steps are. I recommend you read the Using Exchange Server 2003 Recovery Storage Groups whitepaper and practice this before you have to do it in production.

Monday, November 07, 2005

Exchange Connections question: Mount points?

During my Exchange Connections presentation on availability and clustering, one of the attendees said that he had been told by Microsoft that he should NOT use mount points on a cluster due to performance problems. (For those of you that don't use mount points, this feature allows you to mount a disk partition in to a folder on an existing drive letter rather than creating a new disk drive letter.)

I have checked with a number of folks at Microsoft as well as some gurus from Dell Professional Services and Collective Technologies and no one has heard of this recommendation being made. Based on what I have found so far, mount points are still good. I'll dig some more.

Friday, November 04, 2005

There are seldom technical solutions to behavioral problems

In yesterday's closing session of the Exchange and Windows Connections conference, during the Q&A portion, an attendee described a problem she had with a user who was selecting hundreds or thousands of users from the GAL manually and sending worthless/non-business related messages. She asked if there was a way to stop this. We had some lively discussion on this, but one thing that was not mentioned was restricting the maximum number of recipient limits for all of the users (say to 100 recipients rather than the default of 5,000.)

This will not prevent the user from sending out obnoxious messages in batches of 100, but it will make it a little more difficult. While sitting there listening to the attendee describe the problem, though, I thought of something that Ed Crowley once said; Ed is one the most quotable Exchange gurus I know:

"There are seldom technical solutions to behavioral problems."

Thursday, November 03, 2005

Cemaphore Systems' MailShadow 2.0

One of my favorite things to do at a trade show or conference is to visit the exhibits area. Usually to look for interesting software or things I have wanted to learn about. And, of course, to troll for some swag.

At Exchange Connections in San Diego, I came across a vendor called Cemaphore Systems that has a product called MailShadow. IMHO, this definitely wins "best of show". This is a mailbox replication / sychronization system. While the software is not yet released, it is designed to create a backup mailbox for a user on a remote server, then keep that mailbox synchronized and up-to-date with the original mailbox. All changes are sychronizes (new messages, marking messages as read, deletions, calendar changes, contact changes, etc...) If there is ever a situation where the source mailbox fails, this system handles the updates the the Active Directory that are necessary to add the original mailboxes' SMTP addresses to the backup mailbox. The software can also assist in switching over the Outlook client.

I have a number of customers that will think this is very cool and worthy of consideration. If you are looking for a solution that will keep a backup mailbox for your VIP or critical mailboxes, then this is a good solution to look at.

Wednesday, November 02, 2005

Exchange Disaster Recovery Analyzer released today! (ExDRA)

Microsoft released a new tool today called the Exchange Disaster Recovery Analyzer (ExDRA). This tool looks at an Exchange server's databases, and collects configuration data and header information from databases and transaction log files. The tool analyzes all headers and creates a detailed list of instructions explaining what the problem is, and how to resolve it. The goal is to guide a user through the disaster recovery process, automating as much as possible.

The initial release of ExDRA is currently only stepping through the log replay part of disaster recovery. While the ESEUTIL tool will do many of these things, this tool makes many recovery processes much easier when you have a database that won't mount or needs transaction logs rolled forward. One thing I have already noticed, though, is that it does not detect single-bit errors in the database (-1018 errors). Those won't keep a database from mounting, but they will keep the database from being backed up successfully!

ExDRA is available at:

Sony placing rootkits on users PC's to enforce Digital Rights Management

The buzz this week at the Exchange and Windows Connections conference in San Diego has been Mark Russinovich's discovery of a rootkit on one of his own computers. The rootkit was placed their by a Sony music CD. Read Mark's discussion of how he found and analyzed this corporate malware at Sony, Rootkits and Digital Rights Management Gone Too Far. Very scary. Very slimy. Sony has gone way too far.