Wednesday, November 29, 2006

Network Monitor 3.0 released (Free!!!!)

My favorite 'swiss army knife' utility for troubleshooting problems is the Microsoft Network Monitor. (Or Ethereal). I install NetMon on every server that I deploy. Microsoft has just released Network Monitor v3.0! And it is free, no more having to own a copy of SMS or running it only from Windows servers.

Thanks to an excellent protocol analysis class from Bill Alderson from Pine Mountain Group using a Sniffer and then 2 days of intense Microsoft protocol analysis back in 1996 from Paul Adare, I know enough to know I know nothing. But, I can still track down some really tricky problems.

From the Network Monitor Blog, here is a feature list:
  • A completely new user interface
  • Real time capture and display of frames
  • Simultaneous capture on multiple network adapters
  • Multiple simultaneous capture sessions
  • Network conversations and a tree view displaying frames by conversation
  • A new script-based protocol parser language, and script-based parsers
  • Support for Vista/Windows XP/Windows Server 2003
  • Support for 32bit and 64bit platforms

Saturday, November 25, 2006

Exchange 2007 Back Pressure and SMTP 452 4.3.1 Insufficient resources SMTP error

I have been running Exchange 2007 'lab' servers since January of 2006. I installed it in to my main lab system in May, but it was not until Beta 2 was released that I got brave enough to actually move some mailboxes over to it. These are my and mailboxes, so it is still essentially a 'lab' environment. I'm not crazy enough to move a real company over to Beta software without direct Microsoft involvement and support.

Anyway, today I came across a feature that I really felt was blog worthy. All inbound mail still comes in to an E2K3 server. There is a routing group connector that connects to my E2K7 lab servers.

Yesterday, I started noticing that the E2K3's queue to E2K7 was consistently in retry and the "Additional Queue Information" box showed "The Connection Was Dropped By The Remote Host". After several reboots, I finally checked the event viewer and the protocol logs. The protocol log on the E2K3 server included this text:
2006-11-25 21:32:14 OutboundConnectionResponse - - 452+4.3.1+Insufficient+system+resources

The Application event viewer on the E2K7 server had a couple of interesting and educational events. I wanted to share these because they help to provide some insight in to how E2K7 was designed and works.
Event Type: Warning
Event Source: MSExchangeTransport
Event Category: ResourceManager
Event ID: 15002
Date: 11/25/2006
Time: 11:38:17 AM
User: N/A
Computer: HNLEX03
Description: The resource pressure is constant at High. Statistics:

Queue database and disk space ("C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue\mail.que") = 75% [High] [Normal=70% MediumHigh=72% High=74%]

Queue database logging disk space ("C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue\") = 76% [Normal] [Normal=92% MediumHigh=94% High=96%]

Version buckets = 1 [Normal] [Normal=40 MediumHigh=60 High=100]

Private bytes = 14% [Normal] [Normal=71% MediumHigh=73% High=75%]

Physical memory load = 52% [limit is 94% to start dehydrating messages.]

Inbound mail submission from other Hub Transport servers, the Internet, the Pickup directory, the Replay directory, and the Mailbox server, if it is on a Hub Transport server, has stopped. Loading of e-mail from the queuing database, if available, continues.


Event Type: Warning
Event Source: MSExchangeMailSubmission
Event Category: MSExchangeMailSubmission
Event ID: 1009
Date: 11/25/2006
Time: 11:39:10 AM
User: N/A
Computer: HNLEX03
Description: The Microsoft Exchange Mail Submission service is currently unable to contact any Hub Transport servers in the local Active Directory site. The servers may be too busy to accept new connections at this time.
Why is this happening? The Exchange 2007 Hub Transport server role has a new feature called Back Pressure. As a Hub Transport server starts to get low on particular resource, it will reject connections (rather than just stopping service). If the resource that is low (such as disk space) starts to get better, then Exchange will not exert back pressure and will start processing inbound messages again.

Notice this part of Event ID 15002?
Queue database and disk space ("C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue\mail.que") = 75% [High] [Normal=70% Medium High=72% High=74%]

The disk that hosts this server's queue database is above the threshold of 74% and thus Exchange stops accepting inbound mail. However, the services did not stop and as soon as I correct the problem Exchange will start processing mail again. Pretty cool, eh?

Thursday, November 23, 2006

Happy Thanksgiving Day USA

To everyone in the U.S., Happy Thanksgiving! Don't eat too much!

Tuesday, November 21, 2006

Removing IMVv1 prior to installing E2K3 SP2

There was a discussion today on one of the mailing lists I am on about the IMFv1 and the fact that it does not always remove properly. Prior to installing Exchange 2003 Service Pack 2, you should always make sure you logon as the account that installed the original version of the Intelligent Message Filter, go in to Control Panel -> Add/Remove Programs and remove it. If you can't get it to remove (or even find it in Add/Remove programs) re-install it from the original program and THEN remove it.

The discussion came about because Microsoft has removed the IMFv1 installation MSI from their web site. If you need it, send me an e-mail offline. See Bharat Suneja's blog for more info on the removal of the IMVv1 from Microsoft's site.

Anyway, the fruits of this discussion came in the form of two useful links. One from Evan Dodd's blog where he covers what the prereq check is actually looking for. If IMFv1 is installed, then at HKLM\Software\Microsoft\Exchange for a value called “ContentFilterVersion”. This value should be GONE if you are installing E2K3 SP2. See Evan's blog "IMFv1 getting in the way of your SP2 upgrade?" for more information.

Sunday, November 19, 2006

Directory Update rocks, too

We have gotten lots of great feedback on Directory Update. And we have taken on a number of new customers. If you have not been following Self Service Directory Update, it is small utility I designed about a year ago and my company is now selling. This utility allows a user to update or modify their own information in the Active Directory and Exchange Global Address List.

Based on some feedback from customers, we have made some useful changes over the past 3 months:

  • Integrated Windows Authentication or Forms Based Authentication support
  • Almost all fields can be either text fields or drop-down lists
  • Added new telephone number attributes, the Notes attribute, Employee ID, Employee Number, Employee Type, Web Page, and Custom Attributes 1 through 10.
  • Customizable help pages
  • Allow Log Off button to redirect to URL
The interface is simple, clean, and easy to use. Directory Update was designed to do one thing very effectively and we think it does it nicely and very affordably. Our feature set compares nicely with soluions such as Namescape's rDirectory, The DOT NET Factory's EmpowerID Suite, and Imanami's WebDir. All of these do a LOT of other things besides self-service directory management, but if all you are looking for is self-service directory management then they are way overkill.
The programmers are working on a couple of new features that have been commonly requested. These include:
  • Manager, Secretary, and Assistant fields (these use the distinguished name of other objects)
  • Set some fields to be mandatory / required
  • Firefox support!
If you have not yet evaluted Directory Update, pop on over to the site, download it, review the installation instructions, and give it a test drive. I say "read the instructions" because getting any ASP.NET application installed and configured can be tricky and the Setup program does not make all the configuration changes you need to make.

Saturday, November 18, 2006


I figured Exchange12Rocks was pretty common knowledge, but talking to a few Exchange techies this past week, I guess it is not. It is time for thoroughly useless trivia. Allow me to explain.

When you install the first Exchange 2007 server in to your organization, a new adminsitrative group and a new routing group are created. All Exchange 2007 servers are installed in this administrative group. It is created for backwards compatibility. The administrative and routing groups are named:
CN=Exchange Administrative Group (FYDIBOHF23SPDLT)
CN=Exchange Routing Group (DWBGZMFD01QNBJR)

You might be tempted to think that these are GUIDs or randomly generated character strings, but they are not. They are simple "shift" replacement ciphers (similar to a Ceaser cipher).

Take FYDIBOHF23SPDLT as an example. Shift each letter "up" one letter in the alphabet. You get EXCHANGE12ROCKS.

Now, take DWBGZMFD01QNBJR. Shift each letter "down" one letter in the alphabet. You (once again) get EXCHANGE12ROCKS.

We knew that already.

Friday, November 17, 2006

Exchange and Office Roadshow 2007 videos posted

Windows IT Pro in Finland has posted the video presentations of the Get Ready for Office and Exchange 2007 road shows. There is about 14 hours of content out here for anyone that is interested. Plus a very studly picture of me on the opening page. :-)

Wednesday, November 15, 2006

Dry Niagara Falls??!!

A few days ago, a friend sent me a picture he thought had been doctored or faked. It shows the U.S. side of Niagara Falls without any water.

I thought this was pretty amazing looking and obviously the photo is quite old. I did a little research and found that in 1969, someone decided that they should clean up the rocks at the base of the falls. A dam was built, the water was stopped, and the result you see above was the result.

Once the water was stopped, it was determined it was going to be too expensive to actually clean up all those pesky rocks.

Tuesday, November 14, 2006

Exchange Connections reflections

Thanks to everyone from the Connections conferences. The Exchange Connections conference closed on Thursday of last week (well, Friday if you count the post conference sessions). It was a great conference and refreshingly nice to have the developer conferences (SQL Connections, ASP.NET Connections, VB Connections) rolled in with us systems folks. I learned a lot of new things that I would not have learned without those folks around. It was nice seeing a lot of people I have met at previous conferences, too.

For folks that attended my sessions and want my slide decks, here are the session's PowerPoint files.

Full-Day Session: Security For Exchange: Assessment, Auditing, and Hardening
75 Minute Session: Achieving Higher Availability Without Clustering

The sessions were good. David Lemson (a program manager from Microsoft) gave a few good talks on Exchange 2007, which everyone is clamouring to find out more about. I had lunch one day with two Exchange guys from Boeing. It is always good to hear how companies like Boeing are using the technology. Plus, it pays to know people at Boeing when I get ready to buy a new 787 Dreamliner. You never know who is going to help you get a good discount.

Las Vegas was entertaining, as usual. I am continually amazed at the designs of the casinos and how they attempt to "trap" you once you are inside. I take it as a challenge to figure out how to get out. Not being a gambler, the slots and tables hold no draw for me.

I was in Vegas from Sunday through Saturday, which is a long time to be in Las Vegas! I did take in two shows while I was there. I saw the Cirque du Soleil show "O" at the Bellagio. It was beyond cool. It is worth it just to go and watch the sets! On Friday night some friends and I went to see Phantom of the Opera at the Venetian. I had seen it several years before in San Francisco, but it was still entertaining.

As usual, Vegas has a big shortage of taxi cabs. I was always surprised at the lenghts of the lines waiting for taxis. I used the Mandalay Bay tramway and the new Las Vegas monorail to get back and forth. Plus good old fashion shoe leather.

Hopefully I will see everyone again at Connections either in the spring or fall.

Sunday, November 05, 2006

A culture of upgrades - Now when is that service pack coming out?

Okay, I stole the title of this from another theory I have on "a culture of ignorance", but that is an entirely different post.

The whole IT industry seems to revolve around "latest version" craze. The past few weeks I have seen people obsessing in the newsgroups and Web forums as to the exact release dates of Visa, Office 2007, Exchange 2007, etc... They want to know how soon they "can get the bits" and start installing. I, for one, am one of the worst when it comes to "the latest version." But, I also don't want "the latest version" until it is fully baked.

Microsoft feeds on this frenzy by hyping features, functions, and benefits. Or, as my old boss Jonathan White used to say "feeds and speeds." Microsoft has encouraged this "culture of upgrades" as well they should. They are a publicly held business. Their job is to generate revenue for their stockholders. And if they build some awesome technology along the way, well that is spiffy, too.

Our IT industry press seems to further feed this "culture" as well. Every magazine I get is hyping "features, functions, and benefits" right now. If I did not know better, I would think that every magazine I read is owned by Microsoft's marketing department.

Today on an airplane, I was reading a magazine article that two columnist had written together (I won't name the magazine or the writers.) It was if Sybil herself was writing the magazine. The writers chastized Microsoft for being so "overdue" on getting a new release of their desktop OS out the door, they complained that it was late, they complained that features had been cut, they feared that there would be bugs because Microsoft was rushing it, and they praised some of the new features and improvements.

Okay guys, which is it? Should they release it quickly? Wait until it is fully tested and bug free? (That could be a long time) What is it that you want? Microsoft management pressures their product teams to get a product out the door enough as it is, let's not put any more pressure on these folks by encouraging them to release something that is not fully ready.

Saturday, November 04, 2006

I'm off to Exchange Connections in Las Vegas

I'm off to Las Vegas late tonight and headed to the Exchange Connections conference. This is the first time they have combined Windows, SharePoint, Exchange, SQL, VB, ASP.NET, Office, and Mobile Connections all in to one massive event. It sounds like there will be upwards of 5,000 recipients there!

For those of you attending the Exchange Connections track, make sure to attend the keynotes by David Lemson (Exchange Server 2007: The Next Generation Of Exchange) and Tony Redmond (All You Need To Know About Exchange 2007 And Were Afraid To Ask). I normally avoid keynotes, but I have heard a few things that are going to be in David's keynote and even I'm planning to attend. As for Tony, well Tony is always worth listening to when he speaks.

As for me, I'm presenting a high availability session on Thursday and then an all day messaging security session on Friday (Post Conference session). On Wednesday and part of the day on Thursday, I'll be in the Microsoft Community booth in the exhibits area so please stop by.

Friday, November 03, 2006

Maximum E2K7 Transport Rules

A couple of people have asked me how many Tranport Rules can be created in Exchange 2007. You can have a maximum of 1,000 rules. These are processed for EACH message that crosses through the Hub Transport server (and that would be ALL messages). They should be kept in RAM on that server. The more rules and the more messages that pass through the Hub Transport server role, the more RAM you should have and the beefier the Hub Transport servers should be.

Thursday, November 02, 2006

E2K3 Public folder management - SSL certificate server name is incorrect error

I'm still in the middle of cleaning up an E2K3 organization that has kind of been neglected for a few years. I ran in to a problem where I was trying to remove a public folder store from a front-end server. The SSL certificate on the front-end server is wrong (wrong FQDN/CN, unknown CA, and it is expired). I could not manage the public folder hierarchy using Exchange System Manager.

Depending on what I was trying to do, I got this error:

The SSL certificate server name is incorrect.
ID no: c103b404 Exchange System Manager

I also saw this error:
The token supplied to the function is invalid
ID no 80090308

Lots of newsgroup and web discussion forms pointed to this KB article indicating that the problem might be related to SSL being required on the /ExAdmin virtual directory. "You receive an SSL Certificate error message when you view public folders in Exchange System Manager" I checked that and it was NOT the case.

Finally found some instructions in a newsgroup that worked. This requires ADSIEDIT and a little bit of Exchange configuration editing.
  2. Navigate to the following object: CN=Configuration, then CN=Services, CN=Microsoft Exchange, CN=, CN=Administrative Groups, CN=First Administrative Group, CN=Servers, CN=Protocols, CN=HTTP, CN=1, CN=Exadmin
  3. Display the properties of the CN=Exadmin object
  4. Locate the msExchSecureBindings attribute, highlight it and click Edit button
  5. If it has a value of :443:, select that value in the Values list, click Remove.
  6. Click OK twice and then close ADSIEDIT
Give this a few minutes to replicate through Active Directory and try it again!