“Virtualization is a bless” I wrote not so long ago (or lets just say that close to that – maybe different wording) and I didn’t thought that I will get back to virtualization topic so quickly. This time it was triggered by article wrote by one of Polish sys admins who had a rather non pleasant adventure (link if for those of You who can read Polish) with DCs and virtualization. After rebooting his VMWare ESX server with DC (playing also a role of PDC Emulator and time source) caused by some patches being applied to ESX he found out that time on his DC was set back to 1970-01-01 (what a pretty date … isn’t it :). This is short version of his adventure as I assume that most readers on this blog are not very skilled in Polish 😉 .
Problems with time synchronization in virtualized environments are well known and are not limited to single virtualization provider. That is a tradeoff of using it … but better know what we are dealing with.
Time as we know is playing important role in AD infrastructure. Most of us will recognize time as important factor in Kerberos authentication process. But unexpected and uncontrolled time drifts might have some other, not necessary desired consequences.
So we have our DCs virtualized and everything works fine … until one day … surprise … like in example I mentioned earlier our time drifts back to some date like 1970-01-01. Or it might drift into the future to lets say 2011 ? Do we really know what will happen or do we have a plan what to do?
It just came to my mind that this is another topic which probably should find its place in companies DR and BCP plans related to AD. But this is just a side note.
For example when time will drift ahead into a future, further than tombstone lifetime what consequences it will have:
- tombstones will get removed
- all existing backups will be invalidated.
Let’s just hope that this won’t happen when important OU will get deleted and we were just about to go through recovery process.
So is DC virtualization that BAD!!! Are we really against IT!!!?
If You will ask me I would say – YES, go for it and virtualize your DCs but do this with prober planning and knowledge. Get to know what are the pros and drawbacks, things specific to virtualized environment and possible consequences and plan for it. There is plenty information about this topic so just use it.
First off all I want to point You to recommendation (and this is recommendation not hard requirement, but it is really worth to consider) from Technet documentation:
You should retain one or two physical domain controllers per domain in your Active Directory infrastructure. An issue that is specific to virtualization software or the hardware on which it runs can interrupt services on every domain controller in the domain, or even in the forest. If possible, diversify the hardware that you use to host domain controllers so that a single hardware issue cannot interrupt your Active Directory services.
My personal opinion is that this should be considered as really vital recommendation and if it is possible one should stick to it. Of course if we have such physical machine acting as DC (getting back to our main topic – TIME) we can think about making it an authoritative time source in our domain infrastructure. Why not … if we have it anyway.
But if we want to go and put all of our DCs on virtualized hosts it is worth to read some documents and plan time synchronization in align with these information. In particular if we are talking about VMWare environments it is worth to read:
- KB 953797 “Time Synchronization issue in Windows Server 2003 systems running as VMware Guests”
- Timekeeping in VMware Virtual Machines
What I can recommend is that such important AD infrastructure element as time synchronization should not be carried out and handled by a guest OS which synchronizes its time with virtualization host (as well we should not relay on routers or other pieces of network infrastructure). If host-to-guest time synchronization is enabled for a DC OS we should disable this option (and this is also recommended in VMWare document mentioned above). In regards to VMWare documentation – they are not consequent in their recommendations as in other document (VMware Time Sync and Windows Time Service) enabling host-to-guest time synchronization is recommended to be enabled.
But if we really want to give a go to virtualization and virtualize our DCs one of things which should be taken care of in service design document is time synchronization.
If this is an option time should be synchronized with external, trusted sources and it should not rely as NTP server or device which will synchronize it with some atomic clock. I’m not real VMware expert, but even in such environment NTP server can be enabled on ESX server and used to synchronize time among virtual hosts … so we can use it also for quests. This will be much better option then just relaying on a server hardware and virtualization layer.
So to get to the point of this already too long post … keeping to current trends and using virtualization for our DCs can give us some benefit. To not ruin those benefits with unexpected outages it is better to do some planning upfront and be sure that our infrastructure is resilient for incidents like clock reset on single host. With little planning this can be avoided. Lets just try to remember that hypervisors are just another piece of software (nice one but still) and as such it might have a problems … .
P.S #1 If we are on virtualization topic – just a friendly reminder – using snapshot as backup method is EVIL 😉
P.S. #2 Jorge has put together links to few documents which touches DC and virtualization topic. Worth checking.
P.S. #3 It is a long one …. sorry 😉