Have you ever been in the situation where you had to make a choice which Virtual Software to use for your own Infrastructure? I can imagine yes, because at this moment you can choose between a Microsoft solution and an EMC VMware solution. One of the factors that may influence your choice is support of Microsoft products within a virtual environment. If you choose Microsoft Virtual PC or Microsoft Virtual Server as your virtual solution, the Microsoft products within that virtual environment have exactly the same support as if those Microsoft products were running on physical hardware. However, if you choose one of the EMC VMware solutions (ESX server, GSX server or workstation) the support for the Microsoft products in those virtual environments is somewhat different. Microsoft explains the "hows" and "whats" in KBQ897615 (http://support.microsoft.com/?id=897615). Simply put Microsoft says: "if you encounter issues with Microsoft products within non-Microsoft virtualization software, Microsoft requires you to reproduce the issue independently from the non-Microsoft virtualization software". The support for all these products also depends on the support agreements you have and how you purchased those products. More can be read at "http://www.vmware.com/pdf/ms_support_statement.pdf".

Well, the main rason for me to write this here, is that I found an issues with Windows Server and Active Directory within a VM Workstation 5.0 virtual environment. I, at first did not imagine there could be an issue and always thought "it will never happen that issues might occur with Microsoft products within a VMware virtual environment whereas they will not occur on physical hardware." I prooved to be wrong, as an issue occured. I was also able to solve the issue. As everyone else I searched the internet for a solution, but did not find any. So for those that experience the same here is the problem description, its solution and my experiences!

The story goes like this...

HOW THE ENVIRONMENT SHOULD LOOK LIKE (within VMware):
* Forest Root Domain
  --> Forest Root Domain: CORP.NET (DFL and FFL = W2k3)
  --> 1 DC = GC = DNS = FSMOs (it happened with Windows Server 2003 SP1 with or without R2)
  --> Name of DC = ROOTDC01
  --> DC points only to itself as prim. Server
  --> DNS server hosts zones: CORP.NET (in domain app part), _MSDCS.CORP.NET (in forest app part), .(root) (in custom app part called 'ROOTDNSZONE.CORP.NET' - the only enlisted DC = ROOTDC01), 0.10.in-addr.arpa (in forest app part)
  --> All zones are AD-I and secure DDNS is enabled
  --> NO WINS used!
  --> On DNS server a delegation has been created for the zone BRANCH.CORP.NET to CHUBDC01.BRANCH.CORP.NET

* Child Domain
  --> Domain: CHILD.CORP.NET
  --> 1 DC = GC = DNS = FSMOs (it happened with Windows Server 2003 SP1 with or without R2)
  --> Name of DC = CHLDDC01
  --> DC points only to itself as prim. Server and point to ROOTDC01.CORP.NET as an alternate
  --> DNS server hosts zones: BRANCH.CORP.NET (in domain app part), _MSDCS.CORP.NET (in forest app part), 0.10.in-addr.arpa (in forest app part)
  --> All zones are AD-I and secure DDNS is enabled
  --> NO WINS used!
  --> On DNS server forwarding has been configured to ROOTDC01.CORP.NET

Before installing AD, both servers were stand alone servers and ROOTDC01 had as local password CORP and CHLDDC01 had as local password CHILD.
I installed AD on ROOTDC01 and configured everything as mentioned above. All went OK, no issues. Just for a reminder, the password for the default administrator of the forest root domain was CORP and the password for the default administrator of the stand alone server CHLDDC01 was CHILD.
Before installing AD on CHLDDC01 I did some checks:
* On ROOTDC01.CORP.NET I ran "NETDIAG /V" -> all tests passed
* On ROOTDC01.CORP.NET I ran "DCDIAG /V" -> all tests passed
* On CHLDDC01.CHILD.CORP.NET I ran "dcdiag /test:dcpromo /dnsdomain:CHILD.CORP.NET /childdomain" it said: CHLDDC01 passed test DcPromo
* Pinging between both servers is OK!
* NSLOOKUP from both servers querying different type of records works OK
* No errors in event log
* Adding CHLDDC01 to the domain CORP.NET as a member server works OK
* No firewalls used

After removing CHLDDC01 from the forest root root domain as a member server a make it a stand alone server, I tried to install AD on CHLDDC01 by configuring it as a domain controller for a new domain in an existing tree. At some point I needed to enter credentials with enough permissions to install a DC in an additional domain. So I used CORP\Administrator with the password CORP install AD on the DC. That last action failed after entering credentials and clicking OK to continue. The error presented was:

-------------------------------------
The wizard cannot gain access to the list of domains in the forest
This condition may be caused by a DNS lookup problem. For info… blababla….
The error 'The RPC server is unavailable'

-------------------------------------

In short, I was not able to make whatever DC configuration I wanted to for CHLDDC01 (additional DC for existing domain, new domain, etc.) as it kept presenting me the same error over and over again...Irritating!

After a while I started thinking about the error as it was described: "The wizard cannot gain access to the list of domains in the forest. This condition may be caused by a DNS lookup problem"... but NSLOOUPs were working OK. So it had to be something like permissions or corrupt data or whatever, but not nameresolution because that prooved to work. Imagine a Enterprise Admin not having enough permissions to do stuff in AD. Oh, well..
 
To test permissions and credentials I created a mapping (to the ADMIN$ share) from the stand alone server to the forest root DC and used username administrator and password CORP. result = OK
To test permissions and credentials I started LDP on the stand alone server and connected to the forest root DC and used username administrator and password CORP. result = OK. I was able to anything in the directory.
To test permissions and credentials and joined the stand alone server and made it a member server of the forest root domain using the username administrator and password CORP. result = OK.
 
I logged on to the stand alone server with administrator and password CHILD, which is obvious.
I started DCPROMO to install a DC for a child domain in an existing tree and as credentials I entered the credentials of the forest root domain administrator being administrator with password CORP. NO GO!

So the nameresolution tests and permission tests prooved OK, and I still was not able to promote the damn thing to a DC. WTF!

Finally I decided to do a network trace, but at first I did not believe it would give any hint as name resolution was working OK and authentication was also working OK as prooved by the mapping and LDP. OK, lets see what it gives...
I started the trace, started DCPROMO and after entering the credentials it gave me the error. Did the same with a custom enterprise admin I created in AD and it also gave me the error. In the output of the sniffer I saw CHLDDC01 connect to \\ROOTDC01\IPC$ and after that it presented the error:

-------------------------------------
SMB (Server Message Block Protocol)
     SMB Header
          Server Component: SMB
          SMB Command: Session Setup AndX (0x73)
          NT Status: STATUS_LOGON_FAILURE (0xc000006d)        <<<<<-----------------???????
-------------------------------------
 
How could there be an auth. failure when authentication prooved OK? Suddently I changed the password from the stand alone server administrator to match the password of the forest root domain administrator. Started DCPROMO again AND IT WORKED!.

Stopped DCPROMO. Started it again and tried it with the custom enterprise admin and THAT ALSO WORKED! Changed the password for stand alone server administrator back to CHILD, started DCPROMO again and that ALSO WORKED!
 
The fun part: it never gave an access denied error, it just said that DNS was not OK! Grrr...
What I'm still trying to understand is: WHAT AND WHY? That is one I think for Microsoft and VMware to solve!
 
LESSON LEARNED: if everything else fails get that network sniffer working as soon as possible!

This issue occured in VMware Workstation 5.0 and I have seen other posts on the internet that have a similar issue using GSX server and ESX server. However, the issue DID NOT occur on physical hardware (as those posts also mention).