Tomek's DS World : Directory services

Groups and tokens

tomek — Mon, 28 Jun 2010 20:27:00 GMT

I'm done with an intensive month of sessions, delivered for different user groups and other communities online. When you managed to attend my session about Kerberos I hope you liked it ;). Now it's time for some blogging activities.

A friend asked on his blog (PL only, sorry) a question how to quickly determine the groups a computer account belongs to. Question was asked, time for answer, or at least: one of the possible answers :). Actually I was sure that I wrote about it here before but a quick search determined that I'm wrong (I'm sure I talked about it on last TEC in Berlin). If not ... time to do this now. Starting with the basics.

Constructed attributes

First let me introduce the concept of constructed attributes in Active Directory: Active Directory (among other capabilities) can handle dynamically constructed attributes, which are calculated on the fly when a query is issued to get them. If one looks at the object using a standard LDAP client (like LDP.EXE) or other tool these attributes will not be present on the object. However, when a query is issued to the directory to return them – magic happena and the value (if exists) will be calculated and returned.

(cc) Swansea Photographer

First example, which everybody is familiar with, are back-link attributes. Back-link attributes are pair attributes with forward links, which are used to store information about references among the objects – think member –> memberOf.

If we will take a look at user object properties using the new fancy attribute editor feature from Windows Server 2008 R2 Active Directory Users & Computers (ADUC) we can't see memberOf attribute.

However if we issue a query for this attribute using ADFIND.EXE, we find:

C:\ >adfind -b CN=tom.tom,ou=Accounting,DC=w2k,DC=pl -s base memberOF

AdFind V01.42.00cpp Joe Richards (joe@joeware.net) April 2010

Using server: FIMDC01.w2k.pl:389
Directory: Windows Server 2008 R2

dn:CN=tom.tom,ou=Accounting,DC=w2k,DC=pl
>memberOf: CN=Ksiegowosc,OU=FIMGroups,DC=w2k,DC=pl

1 Objects returned

We get a response ... magic

All the magic is being done by the directory service which is calculating, on the fly, the attribute value which was requested. There is more attributes which can be constructed by AD, and they all fall into one of three categories (at least based on available documentation):

Attribute is marked as constructed in the schema using ATTR_IS_CONSTRUCTED bit in the systemFlags attribute value.
Attribute is a back link. (as showed above)
It is the rootDSE attribute..

A list of constructed attributes is available on MSDN for anyone who is interested.

tokenGroups

And here is an answer (one of possible) to the question how to determine group membership for a workstation: One way is to query for tokenGroups attribute of a computer object. Attribute description is presented below:

These two computed attributes return the set of SIDs from a transitive group membership expansion operation on a given object

So if we query AD for a security principal and we ask for the tokenGroups attribute we will get a list of SID identifiers of groups, to which this computer object belongs when it logs on. The computer object in a domain is a security principal as others, so the query can be issued to retrieve its attributes and retrieve computer attributes values.

Once again using ADFIND.EXE:

C:\ >adfind -b CN=STS,CN=Computers,DC=w2k,DC=pl -s base tokenGroups
AdFind V01.42.00cpp Joe Richards (joe@joeware.net) April 2010

Using server: FIMDC01.w2k.pl:389
Directory: Windows Server 2008 R2

dn:CN=STS,CN=Computers,DC=w2k,DC=pl
>tokenGroups: S-1-5-21-2045789631-2668715847-4178987103-1162
>tokenGroups: S-1-5-21-2045789631-2668715847-4178987103-515

As you can see, we've got a list of SIDs corresponding to the groups. How to translate these SIDs to names? Use ADFIND.EXE with SID as query parameter:

C:\ >adfind -b dc=w2k,dc=pl -s subtree -f "(&(objectSid=S-1-5-21-2045789631-2668715847-4178987103-1162))" name

AdFind V01.42.00cpp Joe Richards (joe@joeware.net) April 2010

Using server: FIMDC01.w2k.pl:389
Directory: Windows Server 2008 R2

dn:CN=ADFS Servers,OU=FIMGroups,DC=w2k,DC=pl
>name: ADFS Servers

1 Objects returned

And that's all of the trickery for today ...

AD WS diagnostic logging

tomek — Sat, 10 Apr 2010 00:09:00 GMT

I promised to get back to AD WS topic so here I am. My last post was about the process of Active Directory Web Services (AD WS) instance location from a client perspective. When a client locates the service, in most cases, it is with the purpose to do something with it – query, update ... . But what if something goes wrong and we want to troubleshoot this? Of course there is always network traffic analysis, but there is also an AD WS debug logging mechanism which can be used for it. All you need to do is turn it on. How??

(cc) ehpien

AD WS is a web service written in WCF and installed on every Windows Server 2008 R2-based DC. It is also available as the AD Management Gateway option for Windows Server 2003 and Windows Server 2008. The service has its own configuration stored in a file named Microsoft.ActiveDirectory.WebServices.exe.config, placed in the AD WS installation folder (%WINDIR%\ADWS by default).

Configuration parameters are described on these TechNet pages, however information about the diagnostic logging option is missing there. To configure this mechanism, alter the configuration file and add in an <appSettings> section with the following entries:

<add key="DebugLevel" Value="<log_level>" />

where log_level might be one of following values: None, Error, Warn or Info. Info is the highest level of debug mode, which will log full debug info and also the communication exchange between clients and the service. To configure where the debug information will be stored, add the following key to the config file:

<add key="DebugLogFile" value="<path to log file>" />

I think that options in this case are self explaining. Final configuration might look something like this:

<add key="DebugLevel" Value="Info" />
<add key="DebugLogFile" value="C:\ADWSLog\Adws_trace_log.txt" />

After making these changes in the configuration file, restart the service to make them take effect.

This change has to be introduced into each instance configuration separately. But it might be only a file copy operation – it depends on your environment.

One thing to remember – there is nothing like free debug operation – it always has some cost attached in performance. I don't know what this cost is in AD WS case but always consider it when you will decide to use it – especially in Info mode...

Where is my (AD) web service?

tomek — Tue, 30 Mar 2010 09:27:00 GMT

Windows Server 2008 R2, among other changes, brings a new interface to access directory services – the Active Directory Web Service (ADWS). It is also available for older systems – Windows 2003 and 2008 – as Active Directory Management Gateway (available as separate download).

(cc) paprikaOptic

ADWS I being used so far by a few Windows 2008 R2 components like the new AD interface AD Administrative Center and Powershell module for AD (yes, this Powershell module uses Web Service, not LDAP). This Powershell module was a cause of e-mail I got from one of my friends (and customers also).

When he tried to use Powershell module from workstation to connect to ADWS on a newly deployed Windows Server 2008 R2 box he got the following message:

Windows PowerShell
Copyright (C) 2009 Microsoft Corporation. All rights reserved.
WARNING: Error initializing default drive: 'Unable to find a default server
with Active Directory Web Services running.'.
PS C:\Windows>

Now, here comes the ultimate question: how does the client locate the ADWS instance? – in this case the client being the Powershell module.

The ultimate answer to this question is as always ... DC locator. DC Locator is a process which allows clients to locate an optimal domain controller. Optimal in the AD meaning of this word: closest to a client from a network perspective, where network is represented through sites and subnets in AD configuration.

A client can also pass some additional requests to a DC locator process, which are being used to choose a DC with specific roles, required by the client in this moment. This might be a request for a writable DC or a DC acting as a GC. The Domain controller passes such information in DS_FLAGS structure.

To allow clients to located DCs with ADWS instances an additional flag was added to the DS_FLAGS structure. Description of this new flag states as follows:

DS_WS_FLAG, The Active Directory Web Service, as specified in [MS-ADDM], is present on the server.

And this information can be used to locate a DC with ADWS instance, when a client will specify the additional DS_WEB_SERVICE_REQUIRED flag in the DC request. Same goes for DCs with ADMG installed.

This might be the end of this post, but life isn't perfect and often we will have to deal with mixed environments with W2008R2 and other, older DCs in the same network. Problem is that 2003/2008 DCs doesn't understand this new flag. To correct this, an additional hotfix has to be installed, KB969249 (2003) or KB967574 (2008).

If you will plan to deploy W2008R2 and use Powershell module or other software which uses ADWS, especially in larger environments, remember to deploy enough ADWS instances to handle client traffic and to allow DC locator to locate DCs which host such service. This is especially important in environments with large number of DCs deployed. This way you won't be surprised if your newly created powershell script will fail to locate an ADWS instance.

Enough for today ... but we will get back to ADWS soon...

Be careful what You promise … SYSVOL

tomek — Tue, 09 Mar 2010 12:43:00 GMT

... on my Polish blog a question was asked on Sunday evening if I can provide some description on the SYSVOL location process and the pitfalls which might wait there. I said ... 'Why not' ... and then you have to keep your promise. So today it will be about SYSVOL volume. Recently it is common topic for me as I gave a talk for local communities in Warsaw about GPO mechanics, which also touches this topic. If you can read Polish and you are interested, the slide deck is available on my Polish blog.

So ... regarding SYSVOL, everyone can see that it is there and it does a job ... until something bad happens. That's the short version. Its primary goal is to serve domain clients files on a DC, in particular to serve GPO templates which are the file based part of GPO. Remember: a GPO consists of two parts – the GP container (GPC) in the directory and the GP Template (GPT) in SYSVOL. Plus some extras like logon scripts etc. If there is no SYSVOL or it is not up to date because of FRS problems (sounds familiar?) there are no or outdated GPOs processed on a client side (actually if there is no SYSVOL share, a DC will not do its job).

(cc) swingnut

From a technical point of view, SYSVOL is just a DFS domain based namespace which content is being replicated with FRS in pre-Windows 2008 operating systems and with DFS-R for Windows 2008 and higher if migration was done (if not ... what are you waiting for????). In fact, SYSVOL content can be replicated in any way, as long as you know how to keep it in sync (don't tell our PSS guys I wrote this ;) ).

SYSVOL is present on every DC, it is a DFS namespace so ... how can you tell which replica is our client using at a given time??? And here is the problem we will be talking about today.

Theory …

As I wrote a few times on this blog (and Jorge wrote also about it + he will give a talk about this on the upcoming TEC 2010 – if you will be there, don't miss it – I will miss it ;) ) a client is locating DCs using DNS records and information about sites and subnets in what is called the DC location process. This way, DS client can (at least should) locate the closest (in terms of AD configuration) DC which can handle its requests. Problem is that this is not the case with SYSVOL, as SYSVOL location process does not follow the same path as the DC location process. Many AD administrators have learned this in a more painful way, when they were trying to figure out why clients are using SYSVOL replicas in some small village north of whatever country it was.

A directory service client is receiving a list of SYSVOL replicas, divided into two lists:

SYSVOL replicas in the same site
SYSVOL replicas outside of the client site.

By default, both lists are in random order and are not reflecting things like costs or location in which DCs are located, except obvious information about local DCs. This behavior does not ensure that clients will use the same DCs for logon and SYSVOL within the same site, when multiple DCs are in this site (the word random is key).

To ensure that the DC which handles logon request will be the one which will also be used for SYSVOL location some tweaks have to be performed. These tweaks (and update) are described in KB831201. After applying the tweaks, the DC which handles the request will return its own name as the first DC on the list of SYSVOL replicas returned to a client.

However the problem remains if a client, for whatever reason, is using a SYSVOL replica outside of its site. The list of replicas in the second list, which is replicas located outside of clients site, is not ordered with taking into consideration the cost of getting to this site – it is random. So it might happen that the first DC on the list is in some place far north (or south if you prefer) of the globe. With slow WAN links between them, affecting clients in terms of performance. It is also a common case I observe in customer networks, where customers are not able to access this replica anyway, because of firewall policies which are in place and are prohibiting network traffic between branches.

How to deal with this? It can be easily resolved with additional configuration for DCs, which will enable calculation of the SYSVOL replicas list with taking cost of connection between client and replica into consideration. This option is available for Windows Server 2003-based DCs by default (there is also a fix described in KB823362 for Windows 2000 Server – remember , support for 2K ends on July this year) and it is called SiteCostedRefferals. To enable this option configure this registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Dfs\Parameters
Value Name: SiteCostedReferrals
Data Type: REG_DWORD
Value: 1

However, to make this work you have to provide additional information through configuration at the directory level. This information is required for the directory service to calculate possible routes and this is information about which sites can be accessed by the client. To do this we can enable Bridge all site links (BASL) option, however this might not be the preferred way to do this. Why? Because this will also disrupt the replication topology calculation process from KCC standpoint. But if we want to enable SYSVOL cost based replica list calculation while not disturbing KCC with BASL information we can choose to enable it for given sites, which will cause KCC to ignore information about site bridging during calculations, but still seeing it (bridged) for SYSVOL replica cost calculation.

In theory you can think about it as maintaining site bridges manually as alternative to BASL however I don't know if this will work in a real world scenario (but with right people following right process ... it might).

And with the information provided above, for those who were not aware of it so far, I hope life with SYSVOL is much simpler right now.

Toolkit …

Here's some short information about tools that can be used in information gathering or troubleshooting process. The basic tool to start with is dfsutil. Dfsutil allows you to see the list of replicas from the client point of view and see which one is active at given point in time. Two switches to remember:

DFSUTIL /SPCINFO
DFSUTIL /PKTINFO

In Windows Server 2008, these switches have changed and have become:

DFSUTIL CACHE DOMAIN
DFSUTIL CACHE REFERRAL

To have access to DFSUTIL in Windows Server 2008 and later you will have to install DFS management tools using features.

And that's all for now ... at least about SYSVOL.

Spot the difference

tomek — Thu, 28 Jan 2010 12:59:00 GMT

Where is a question there is an answer (at least in most cases). This time question was “How to check schema extension introduces to a forest?” and it was asked on ActiveDir.org. There was even more than one answer … apparently some consultants are watching this list :).

So how we can capture what was changed in schema since it was established together with our forest.

(cc tobym)

One of option is using Schema Analyzer tool which comes with AD LDS (ADAM) as it is described on Ask DS Team blog. If we have AD LDS instance and LDFI file with schema we want to analyze it will allow us to get difference between target and base schema. Easy but …

it requires access to AD LDS instance and LDIF file with schema
sometimes it is a bit overhead to get LDI file with difference and we require something easier.

So next approach, also not perfect but a bit simpler and in some cases might be good enough. Just take a(dfind.exe)ny LDAP query tool and query all schema including whenCreated in output. This attribute is replicated among all DCs and we can track date of creation of object. Simple example:

adfind -schema -f "(|(objectClass=attributeSchema)(objectClass=attributeClass))" ldapDisplayName whenCreated –adcsv

now redirect output to file … open it in Excel, sort it on whencreated collumn and voile…

Of course it is not perfect. Still it requires tool like Excel and it gives You only overview when attributes where created. And what about modifications?

In cases we need such information SchemaDiff.cmd script created by Dean Wells (included in archive) comes handy. This tool is based on querying replication metadata and this will give You information about new and updated attributes. Let see how it works:

C:\Temp>SchemaDiff.cmd w2k.pl

SchemaDiff 1.1 / Dean Wells (dwells@msetechnology.com) - March 2006

STATUS - Working [review title bar for progression] ...

       - Forest/schema creation timestamp: 2009-08-23 @ 22:51:06
       - base-schema has been MODIFIED since Forest creation
       - counting classSchema and attributeSchema instances: 1438
       - querying schema ...

*MOD: CN=Schema,CN=Configuration,DC=w2k,DC=pl
       - schemaInfo........................ {modified post-instantiation}

*MOD: CN=User,CN=Schema,CN=Configuration,DC=w2k,DC=pl
       - auxiliaryClass.................... {modified post-instantiation}

+NEW: CN=AstContext,CN=Schema,CN=Configuration,DC=w2k,DC=pl
+NEW: CN=AstExtension,CN=Schema,CN=Configuration,DC=w2k,DC=pl

(…)

Done - 57 schema object(s) added, 4 schema object(s) modified
       in Forest "DC=w2k,DC=pl"

Quick, nice and easy … and no additional tools required (I don’t count repadmin.exe as an additional tool in AD environment).

In general best way to answer such question is to have implemented schema governance process in your environment. It doesn’t have to be something very complicated, sometimes simple file with some procedures is enough … or WSS site in more advanced case. Key is to stick to it and follow it. Think about it …

Where is my DC?

tomek — Fri, 15 Jan 2010 11:28:00 GMT

It is common knowledge that in AD environment client (like workstation) will always (at least it should) try to connect to most optimal domain controller. Optimal from network and AD infrastructure configuration standpoint. This process is based on DNS queries and information stored in AD configuration and in perfect case should lead to situation when client has contacted most optimal DC at given moment.

So we have all subnets defines, connected with appropriate sites and DCs placed in these sites or covered in other way. And suddenly some clients from some small location are starting to use some random DCs instead one we designated for them in our bright and shiny configuration. In such case sys admin is entering his most favorite mode … troubleshooting

(cc) trriseesthings

AD configuration has been extensively reviewed and checked, network checked … event logs are not giving us a clue … what next (besides calling cavalry of some sort :) )?

In such case we have at least one additional troubleshooting mechanism which might be extremely useful in this process, which is enabling debug logging for DC locator process. In each Windows version netlogon service comes with ability to log debug information. What has to be done is enabling this mechanisms through registry change and settings some flags … these flags are described in KB 109626 Enabling debug logging for the Net Logon service.

When this will be done netlogon service will start to log diagnostic data in %widir%\debug\netlogon.log. These information might be very useful in troubleshooting process or at least should give us idea what is going on during this process. Sample netlogon.log part (slightly modified for better reading) from my lab environment is presented below .

[SITE] Setting site name to '(null)'
[SESSION] \Device\NetBT_Tcpip_{33941FFA-DFED-4744-BF9A-972228BC6FF0}: Transport Added (192.168.1.10)
[SESSION] Winsock Addrs: 192.168.1.10 (1) List used to be empty.
[SESSION] V6 Winsock Addrs: (0)
[CRITICAL] Address list changed since last boot. (Forget DynamicSiteName.)
[SITE] Setting site name to '(null)'
[DNS] Set DnsForestName to: w2k.pl
[DOMAIN] W2K: Adding new domain
[DOMAIN] Setting our computer name to wss wss
[DOMAIN] Setting Netbios domain name to W2K
[DOMAIN] Setting DNS domain name to w2k.pl.
[DOMAIN] Setting Domain GUID to ce28b6f7-a26a-4e0f-9f39-0e63e525493e
[MISC] Eventlog: 5516 (1) "wss" "W2K"
[INIT] Replacing trusted domain list with one for newly joined W2K domain.
[SITE] Setting site name to '(null)'
[LOGON] NlSetForestTrustList: New trusted domain list:
[LOGON]     0: W2K w2k.pl (NT 5) (Forest Tree Root) (Primary Domain) (Native)
[LOGON]        Dom Guid: ce28b6f7-a26a-4e0f-9f39-0e63e525493e
[LOGON]        Dom Sid: S-1-5-21-1855823386-3643518527-1754427229
[INIT] Starting RPC server.
[SESSION] W2K: NlSessionSetup: Try Session setup
[SESSION] W2K: NlDiscoverDc: Start Synchronous Discovery
[MISC] NetpDcInitializeContext: DSGETDC_VALID_FLAGS is c00ffff1
[INIT] Join DC: \\resfs.w2k.pl, Flags: 0xe00013fd
[MISC] NetpDcInitializeContext: DSGETDC_VALID_FLAGS is c00ffff1
[MAILSLOT] NetpDcPingListIp: w2k.pl.: Sent UDP ping to 192.168.1.1
[MISC] NlPingDcNameWithContext: Sent 1/1 ldap pings to resfs.w2k.pl
[MISC] NlPingDcNameWithContext: resfs.w2k.pl responded over IP.
[MISC] W2K: NlPingDcName: W2K: w2k.pl.: Caching pinged DC info for resfs.w2k.pl
[INIT] Join DC cached successfully
[SITE] Setting site name to 'Default-First-Site-Name'
[MISC] NetpDcGetName: w2k.pl. using cached information
[PERF] NlAllocateClientSession: New Perf Instance (001E6688): "\\resfs.w2k.pl"
    ClientSession: 00237D58
[SESSION] W2K: NlDiscoverDc: Found DC \\resfs.w2k.pl
[SESSION] W2K: NlSetStatusClientSession: Set connection status to 0
[DOMAIN] Setting LSA NetbiosDomain: W2K DnsDomain: w2k.pl. DnsTree: w2k.pl. DomainGuid:ce28b6f7-a26a-4e0f-9f39-0e63e525493e
[LOGON] NlSetForestTrustList: New trusted domain list:
[LOGON]     0: W2K w2k.pl (NT 5) (Forest Tree Root) (Primary Domain) (Native)
[LOGON]        Dom Guid: ce28b6f7-a26a-4e0f-9f39-0e63e525493e
[LOGON]        Dom Sid: S-1-5-21-1855823386-3643518527-1754427229
[SESSION] W2K: NlSetStatusClientSession: Set connection status to 0
[SESSION] W2K: NlSessionSetup: Session setup Succeeded
[INIT] Started successfully

Does it look useful??? I think so … happy troubleshooting and don’t forget that Network Monitor or WireShark will tell You the truth about what’s going on on a wire. And this is ultimate troubleshooting tool.

Kerberos and non-standard port number

tomek — Sun, 20 Dec 2009 11:52:14 GMT

Kerberos in Windows Operating System is around for about 10 years and it is still causing problems and for many people it is like black magic voodoo. In most cases organizations and people in it are not aware that it is now working until it problem will occur on a surface with some application not working or reports not being displayed on MOSS web page …

… and when problem occurs some troubleshooting starts. To make this process a bit easier here is a short explanation of Kerberos, IE and and services running on non-standard port issue.

(cc) TheCX

This post is sponsored by letter A like Architect, because of our Architects inspired me to write it with his ranting about this problem.

Issue which is subject of this post is not related to Kerberos protocol itself, but to Internet Explorer and how IE handles such requests by default.

Never ending story, SPNs …

Short reminder what SPN is … when client application is trying to get access to resources and is using Kerberos authentication it requests at some point Ticket Granting Service (TGS). To specify to service to which it is requesting access in TGS request client specifies Service Principal Name (SPN). SPN then is being used by KDC to find an account which is related to this service and to prepare tickets for it. This is in short words how it works …

SPNs are just string values for servicePrincipalName attribute in form which consist of service prefix, host name and optionally port number.

For example for standard HTTP service running on www.w2k.pl host address SPN would be specified as HTTP/www.w2k.pl.

As I mentioned above there is also optional element of SPN which can be used to specify port on which service is running. In case of our HTTP service running on 8080 port SPN which will contain this port number will look like this HTTP/www.w2k.pl:8080.

Simple … it is helpful if we have services running on different ports and using different accounts – like application pools running on separate accounts associated with web sites on two different ports.

And here comes Internet Explorer …

Problem with Internet Explorer is that when it is being used as client application to request access to Kerberos enabled service on non standard port by default it will not include port number in SPN sent in TGS request. In such case network traffic capture will look somewhat like this (click to enlarge):

As we can see in this traffic IE is trying to request access to web site running on port 8080 but in TGQ request it is not exposing this information and instead of HTTP/lhr2dc01.w2k.pl:8080 it sends request with HTTP/lhr2dc01.w2k.pl as SPN value.

This behavior was first fixed for IE 6 with KB 908209. For IE6 it required fix to be installed and additional registry entry being made.

This article is not mentioning this but same behavior is present in IE7 and IE8. To fix this it doesn’t require fix to be installed but still it has to be enabled through same registry entry specified in KB mentioned above.

If this will be done same situation in network traffic looks as it is presented below (click to enlarge):

As it can be seen in this traffic analysis IE is requesting access to a web site with port specified in SPN and this allows authentication to be completed in this scenario.

When this is useful …

“Why bother???” This is required in scenarios when we have multiple services running on single host, different ports and under different security accounts. Good examples are multiple application pools on single IIS machine.

Probably anyone who will deploy MOSS sites with multiple accounts will came across this scenario and will have to deal with it.

Why not make it default …

Question is … why this is not enabled by default in IE 7 and 8? problem was fixed for IE6 but for later versions it might be included in a default configuration.

I don’t know official answer but first thing which cross my mind is - backward compatibility (you can call it IE6 curse if You want it :) ). Because IE6 worked in this way and many applications were configured to work in this way, which was allowed by IE6 problem turning it on by default in next versions would break all these applications.

IE6 was not specifying a port in SPN request and if there was suitable account with only one SPN without port being specified, and there was another service running on the same host with different port number but under the same service account it just works.

If You will enable this behavior applications running on different ports would break … registering additional SPN will fix it of course, but this would require some planning up front or quick troubleshooting (basic level of network traffic analysis required).

What I would like to see is configuration option which would enable this behavior through GPO … feedback given :).

userPassword

tomek — Sun, 22 Nov 2009 13:59:00 GMT

One of my friends PFE has asked me a question regarding userPassword attribute in directory which was related to some behavior he was observing in customer environment. We had a little chat about it and then I thought that maybe other has such questions as well so … here’s a topic for a blog.

Behavior my friend was observing was related to a fact, that after some operations performed in environment customer has noticed that on some objects affected by these operations this attribute contained user password in clear text … now I can hear screams of all security guys :) … Yes, clear text and password has some connotations .. in most cases negative once.

(cc) Somewhat Frank

Of course fact that this password was there didn’t mean that it was available for anyone willing to read it … some ACLs still apply in directory … however the fact was that IT WAS THERE.

Matched DNs:
Getting 1 entries:
>> Dn: CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl
    4> objectClass: top; person; organizationalPerson; user;
    1> cn: jan Kowalski;
    1> sn: Kowalski;
    1> userPassword: P@ssw0rd!;

Whatever You thin at this point this is not a bug and there is no point in calling MSFT 112 number (if such exists at all :) ). It is expected and it is a result of userPassword attribute behavior dualism in AD.

userPassword …

userPassword is an attribute which can act differently when it is being written or read depending on directory configuration. Depending of directory settings it can be treated as:

ordinary unicode attribute which can be written and read as any other unicode attribute in directory
shortcut to user password in directory which will allow password change operation to be performed over LDAP.

In first case, when domain is below Windows 2003 level or at this level specific value in dsHeuristics, is not set this attribute is just an unicode attribute. We can write it and read it … let’s try:

admod -b "CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl" userPassword::P@ssword!!1

AdMod V01.10.00cpp Joe Richards (joe@joeware.net) February 2007

DN Count: 1
Using server: w2003r2base.w2k.pl:389
Directory: Windows Server 2003

Modifying specified objects...
DN: CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl...

The command completed successfully

So we could modify this attribute … now let try to read it:

adfind -b "CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl" -s base userPassword

AdFind V01.40.00cpp Joe Richards (joe@joeware.net) February 2009

Using server: w2003r2base.w2k.pl:389
Directory: Windows Server 2003

dn:CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl
>userPassword: 5040 7373 776F 7264 2121 31

1 Objects returned

Success! So apparently we can write and read this attribute and if you will conduct this test on your own this will not affect user password in any way. We have just altered a text in a directory attribute.

However the game rules changes if we will set 9’th char in dsHeuristics to 1 (in fact according to documentation any character other than 0 or 2 should work) writes to this attribute will behave differently. After this modification userPassword attribute is write-only and we can’t read anymore. But it will allow us to modify user password. Let see …

First dsHeuristics modification:

admod -b "CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuratio
n,DC=w2k,DC=pl" dsHeuristics::000000001

AdMod V01.10.00cpp Joe Richards (joe@joeware.net) February 2007

DN Count: 1
Using server: w2003r2base.w2k.pl:389
Directory: Windows Server 2003

Modifying specified objects...
DN: CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=w2k,DC
=pl...

The command completed successfully

Done … now let’s try to do same modification as we did earlier:

admod -b "CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl" userPassword::P@ssword!!1

AdMod V01.10.00cpp Joe Richards (joe@joeware.net) February 2007

DN Count: 1
Using server: w2003r2base.w2k.pl:389
Directory: Windows Server 2003

Modifying specified objects...
DN: CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl...: [w2003r2base.w2k.pl] Error 0x35
(53) - Unwilling To Perform

Wow … Error, ... but why? We’ve just tried to modify user’s password over LDAP protocol and in AD this is only allowed over SSL connection which was not specified in this case. So one more try using LDAPS this time:

admod -b "CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl" userPassword::P@ssword!!1 -ssl -h w2003r2base.w2k.pl:636

AdMod V01.10.00cpp Joe Richards (joe@joeware.net) February 2007

DN Count: 1
Using server: w2003r2base.w2k.pl:636
Directory: Windows Server 2003

Modifying specified objects...
DN: CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl...

The command completed successfully

Now it has succeed, and now read test:

adfind -b "CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl" -s base userPassword

AdFind V01.40.00cpp Joe Richards (joe@joeware.net) February 2009

Using server: w2003r2base.w2k.pl:389
Directory: Windows Server 2003

dn:CN=jan Kowalski,OU=DRLab,DC=w2k,DC=pl

1 Objects returned

Nothing. This mean that we can use userPassword attribute to modify user password but of course we can’t read it afterwards … which is somehow expected..

problem …

Actual topic which started this conversation was KTPASS tool behavior which was observed in customer environment (KTPASS is a tool which allows keytab files to be created –> keytab are used with Unix boxes to allow authentication with Kerberos against AD … in short words).

So … in cases when KTPASS was used for an account, in which none modification to dsHeuristics was made password set for account with KTPASS was available for read with LDAP from appropriate directory object. Apparently KTPASS is trying to set a password using LDAP which leaves it in this attribute. Quick test shows that this is case. If we will try to generate new keytab for an account and specify a password:

ktpass -princ HOST/ubuntu.w2k.pl@W2K.PL -mapuser ubuntu$@W2K.PL -ptype KRB5_NT_SRV_HST -mapop set -pass P@ssw0rd1 -out ubuntu.keytab

(…)

Reset UBUNTU$'s password [y/n]? y
Key created.
Output keytab to ubuntu.keytab:
Keytab version: 0x502
keysize 60 HOST/ubuntu.w2k.pl@W2K.PL ptype 3 (KRB5_NT_SRV_HST) vno 2 etype 0x17
(RC4-HMAC) keylength 16 (0xae974876d974abd805a989ebead86846)

and then we will use ADFIND:

adfind -b "CN=ubuntu,OU=DRLab,DC=w2k,DC=pl" -s base userPassword

AdFind V01.40.00cpp Joe Richards (joe@joeware.net) February 2009

Using server: w2003r2base.w2k.pl:389
Directory: Windows Server 2003

dn:CN=ubuntu,OU=DRLab,DC=w2k,DC=pl
>userPassword: 5040 7373 7730 7264 31

We will see that userPassword gets populated and if you will check its value it will be password specified with KTPASS. The same will happen with any other tool which will try to use LDAP to change or reset user password in such setup.

If we will modify this behavior with setting value on dsHeuristics it will change directory behavior and userPassword will contain no trace of password data in readable form.

Solution …

I think that there is no need for special solution as we don’t have a problem. Best way is to know how it works and if we are concerned with that just use this knowledge to enforce correct behavior … either by establishing some policy around usage of tools which uses LDAP to modify password or through altering directory settings to allow password change through LDAP and thus stopping userPassword from being holding current user password just “by accident”.

Of course ACLs still applies but one might be in hard position of explaining to some AUDITOR why THE PASSWORD IS THERE. In such case … you can redirect them to my blog or better … to MSDN pages.

Snapshot recovery tool strikes back

tomek — Wed, 28 Oct 2009 15:16:38 GMT

Some time ago, when Windows 2008 was released I had some spare time (where are those days) and I wanted to master some of my .NET coding skills. What is better than find an idea to use them … and that’s how 1Identity Snapshot Recovery Tool was created.

Snapshot Recovery Tool is command line tool which might be used to un-delete existing tombstone and later to populate all or some of attributes with data from directory services snapshot data.

Snapshots is nice feature introduced in Windows 2008 which allows you to inspect Active Directory content at given point in time when snapshot was taken. My opinion is that this was half-backed attempt to introduce something like Recycle Bin which is present now days in W2008R2. But hey .. it was there so I decided to use it.

Using this tool and snapshot data one can recover all attributes including links as in memberOf attributes for a user and member for a group.

It can recover single object or multiple objects based on GUID list or LDAP query.

Few words about original place where this tool was published – 1Identiyt. 1Identity was initiative of mine to build an independent network of directory services and identity experts which would build tools and documentation … it didn’t worked this time. Mostly because lack of time from my side. Maybe one day I will get back to this idea.

In the meantime … snapshot recovery tool is back here on DirTeam.org and if You want it, and You can find use for it … feel free to use it.

Comments, bug reports and suggestions welcomed here or on t.onyszko@w2k.pl.

P.S.#1 I want to say big THANK YOU here for Jorge who has tested this tool and provided very useful feedback regarding functionality and bugs. And showed this tool few times at some (DEC\TEC) occasions.

P.S.#2 I also want to say Thank you to all of You who have tried this tool and liked it. I read some blog posts and comments about it and it was nice to read that my work has actually helped somebody.

One subnet to catch them all

tomek — Tue, 06 Oct 2009 10:08:00 GMT

This post is probably first of TEC 2009 follow-up series, at least partially as I thought about covering it just before going to TEC. However Brian Desmond has touched this topic during his session so it is good reason to follow-up on it.

This will be about usage of catch-all subnets in AD topology design. What catch-all subnet means?? Let start from definition.

(cc) f-l-e-x

What’s this about …

When client computer is trying to locate domain controller it is performing location process during which it will try to discover its site based on network subnet information which will be send to DC (Jorge has put nice description of DC location this process in three parts – I, II, III). If client will determine its site it will try then to locate DC in this site using DNS queries. Site location process fro Active Directory perspective is based on site and subnets defined in AD. If client network subnet matches subnet object defined in AD client is assigned to site to which this subnet was assigned at directory level.

But there might be situation in which client is not able to determine it’s site because subnet object corresponding to client’s network subnet was not defined in Active Directory. In such situation client will pick one of available DCs (I will cover what “available” means in this context later) it can reach and will use it for its operation. Problem is that this might be far from most optimal DC for this client to use – for example it might be DC in one of far and poor connected branch site.

So what if we will create on subnet object (or few of them) which will span across multiple sites and will cover all of our subnets used in network. If these super subnets objects will be connected to some site our client will always be able to determine its site and at the end determine corresponding DCs. In worst case client will use not optimal DC but one in a site for which catch all subnet was configured. Done. Some explanation on this topic can be found in article in TechNET Magazine.

So what’s the catch …

Looks promising, but – can we do this in other way? Yes we can and I wrote about it earlier in my post How to cover un-covered – the case of missing subnet. In short words we can use DNS registration to control which site will be chosen by client in case it will not be able to determine exact site it belongs to. This can be achieved through proper registration of site and domain specific domain SRV records. If client will not be able to locate its own site it will pick one of DCs which registered domain specific records.

And that’s it … is it better approach than catch-all subnet? Is this better approach than catch-all subnet?? Probably it is just a personal preference but I like to use DNS records over such subnets. It is more elegant solution for me and I think that it is easier to manage and troubleshoot in case of some problems. The choice is Yours …

When catch-all subnet can benefit …

However I can see scenarios in which catch-all subnet can have some benefit. Let’s take a look at topology which is not exactly hub-n-spoke but is something which sometimes is called snow flake. In such topology we have central site (hub) and two or more tires of satellite sites.

If we would want to gather traffic from all clients from 3’rd tier at the 2’nd tier level and even if they can’t find their site not re-direct them to one of DCs in a hub we can’t do this with DNS records. In such case we can use catch-all subnet for each region \ sub configured at 2’nd tier sites level to control behavior of clients and keep all clients attached to correct site at 2’nd tier of our topology as on this picture.

Of course DNS records registration should also be correctly planned and configured for such design.

And that’s basically it – this just came out on Brian’s session on TEC and maybe it would not catch my ear\eye if I would not read this article on TechNET just before TEC.

So what do you think about using catch-all subnets? Are You using them? Any other ideas or comments? Comments are open … so is contact form :).

(Web)Press review

tomek — Thu, 17 Sep 2009 11:11:06 GMT

During preparation to TEC sessions and during TEC I noted some topics to blog about in a future so I hope that I will find time to blog about them soon. I noted also some URLs to tools which are out there so today’s post is some kind of web press release.

Patch management. If you have ever wondered how to deploy updates maybe You will get interested in script which was posted by Brian Desmond on his blog. Pretty interesting if you will ask me. Worth to check.

Group nesting. If you are managing AD environment and You have nothing against using W2008R2 Powershell you might take a look at script posted on AD Powershell team blogs site. It allows You to select group and analyze how it is nested in other groups and even present it in (sic!) tree form. Nice example how to utilize R2 Powershell capabilities.

Speaking about R2 Powershell. As some of You may know these cmdlets are not utilizing LDAP but brand new AD Web Service which is also being shipped with R2. For down level DCs (look how quickly W2008 has become down level :) ) there is web download which delivers this service for Windows 2003 and 2008 DCs and ADAM \ AD LDS. It is called Active Directory Management Gateway and will allow You to manage these DCs with Powershell.

At the end something from other area – file server. New tool has hit Downloads web site – it is File Server Capacity Tool which comes in 32-bit i 64-bit flavor. I think name of this tool is self explaining.

So that’s all from web review for today …

Using multiple UPN suffixes for users in single directory

tomek — Mon, 24 Aug 2009 13:17:00 GMT

ActiveDir.org is always a source of all sorts of directory related discussion. In most cases interesting once. I have to admit that I would like to have more time to catch up with ActiveDir.org and to be more active there (note to self) but with Wojtek @ home (he’s growing) it is getting even harder then before.

BTW – if you want to look for something AD related you can use custom search engine which was put together by Rick, one of ActiveDir.org members.

Today I was lurking through posts and I found discussion about using multiple UPN suffixes within a domain, and by multiple member who asked this question was meaning few thousands. This configuration was intended to allow some users (partners) to log on with their e-mail addresses to hosted directory.

Few useful information were thrown in the thread. Quick summary:

GUI limits number of suffixes possible to be entered at forest level to 850 (Andrew Levicki), more can be added with scripts
more means ~1300 in Windows 2003 and later UPN suffixes which can be stored in upnSuffixes attribute on CN=Partitions,<configuration partition> and with script you can enter whatever you like for specific user (joe). It is UI which enforces forest wide suffixes on user object. And You have to be careful if it is configuration with forest trust [1]. But for that number of users and suffixes probably GUI won’t be preferred tool.
We have explicit and implicit types of UPNs (Rick S.). See also KB 929272.
If you want to use GUI anyway you can easily extend context menu with some script which will allow you to set desired UPN suffix for a user (Jorge).

Last comment from Jorge about extending UI in this way my eye as there is a bit more comfortable option if you want to have option to set different UPN suffix for users in hosted (or similar environment) which is often omitted. This is setting upnSuffixes attribute on OU level.

If users which will share common UPN suffix can be grouped in single OU structure (for example users from single partner company) one can set upnSuffixes attribute at OU level for desired value

This value will be later presented in GUI when new user will be created, among with other UPN suffixes configured for a forest.

Voile … and its done. Problem is that for another OU in this structure the same value (or different) will have to be set, as this information is not inherited from parent OU.

But I will agree with joe – with that number of users probably GUI won’t be preferred tool. But anyway … it is good to know and maybe somebody will benefit from this knowledge.

[1] - Using multiple UPN suffixes you have to remember that as long as these suffixes are only being used within single forest they are not so important. However with multiple forests UPN suffixes are being used to route authentication requests so you don’t want to get it broken so plan for it before you will deploy it in production.

Ex2007SP2 – step towards virtual directory … sort of

tomek — Sun, 02 Aug 2009 11:53:00 GMT

There are number of things in which my current employer managed to succeed. Among greatest success I think one can count way in which Microsoft managed to scare people with Active Directory schema extension. Probably it has started somewhere down the road with Windows 2000 shipped, some communication, talks … but the fact is … people are scared to extend the schema in most part and some former communication from Microsoft has its part in this process.

There are several different implications of a fact that organizations are not willing to extend or modify its directories schema. One part is that sometimes it keeps consultant like myself busy ;). Other side of this equation is that some changes instead of being made in schema definition are hard coded in OS (example – SIDHistory preserved on a tombstone introduced with SP1).

After some discussion on my Polish blog we have extracted another problem with this attitude towards schema modification. Because customers are not very eager to extend the schema it slows down adoption of software which is doing this so developers are abandoning directory as a place where they can store data. Of course directory is not a dumpster and You should not keep all information in it but sometimes it does makes sense indeed.

But getting back on track with title of this post. I had some thoughts about it when I read description of new feature which is being introduced in Exchange 2007 SP2 (and consequently in later Exchange editions). It is called Dynamic Active Directory Schema Update and Validation and described on Exchange Team blog in this way:

The dynamic AD schema update and validation feature allows for future schema updates to be dynamic deployed as well as proactively preventing conflicts whenever a new property is added to the AD schema. Once this capability is deployed it will enable easier management of future schema updates and will prevent support issues when adding properties that don't exist in the AD schema

(cc) origamidon

Rather short description but today I read some communication which put some more light on it and I thought I will share my understanding of this feature here. So it looks like this new Exchange feature will do as follows:

it will enable Exchange with knowledge that particular attribute in Active Directory is “dynamic” which from logical point of view will be closer to “optional”
If Exchange will try to read such attribute from directory where schema was not extended with it, driver used to read data from AD will just provide information as this attribute would exist with no value set for given object (<not set>)
If Exchange will try to write to such attribute it will fail with some specific exception being returned.

What does it mean (or at least can) in practice. Potentially it will allow Exchange team to develop new features, released in hot fixes or rollups which might require schema extension which will also be provided. But customer will have a choice of deploying these extensions or not. Without it being deployed the feature will be inactive or will behave for given user as it would have some “defaults” set. But it will not prevent organization from deploying such update if schema extension would be what would hold it. Potential win-win situation from Exchange team and customer. Customer will be able to enable this feature later after extending the schema and starting to use it.

If this will be used in this way … will see, maybe Exchange team has other purpose for this feature in minds.

From my perspective I think that this will increase a need for organization to take care about schema and its governance more carefully (what was deployed, what not etc).

But what this all has to do with virtual directory from this post title. Well not much, but this feature is very basic and simplified implementation of something which most of such virtual directories provides – dynamic schema extension. Of course virtual directories are not limited only to validation of schema upon reads and writes but this is something similar. I doubt that Exchange is about implementing such full blown solution but will see what will happen (at the end they have implemented RBAC for AD in Ex 2010 in some limited scope).

At the end it is not that bad. I would like to see such features to be developed and placed at directory service level or at least in Web Service for AD. This will be more coherent way of developing and deploying features around directory, but I’m not in charge to change this :). Will see in which direction such changes will go … one of my friends made a joke lately that soon we will have three OSs from Microsoft: Windows OS, Exchange OS and MOSS OS :).

PS. I’m wondering how many of you have my dear readers have in your organizations some procedures \ regulations around schema governance for AD? Comments are open.

Microsoft IT (AD) Health Scanner

tomek — Tue, 07 Jul 2009 12:16:00 GMT

Take a lot of work, add 7 week old son and serious flue (which strikes 2 days after You got back from travel which incorporates transfer on some heavy used airports – first thought – pork ;) ) and you will get some silence on a blog. I hope that while my son will get older silence periods will get shorter and shorter :). To day just quick download announcement.

In my MS Downloads feed I came across a new tool called Microsoft IT Health Scanner released by Essential Business Server team. Later I found also blog entry on this tool on ESB team blog. I called it AD not IT in blog entry title as right now this is tool which is focused on running AD and AD related infrastructure tests for most common problems, as ESB team says based on most common issues solved by Microsoft support. Maybe in future it will run more tests as this tool has update mechanism included.

IT Health scanner runs around 100 tests including (quoting ESB blog):

(…)

Configuration of sites and subnets in Active Directory

Replication of Active Directory, the file system, and SYSVOL shared folders

Name resolution by the Domain Name System (DNS)

Configuration of the network adapters of all domain controllers, DNS servers, and e-mail servers running Microsoft Exchange Server

Health of the domain controllers

Configuration of the Network Time Protocol (NTP) for all domain controllers

(…)

Definitely looks interesting and my quick scan through result list (I will explain later why this was really quick scan) shows that these are really most common causes of problems in AD environments. This tool is intended to let you find these problems and provide links to article which will allow these problems to be resolved. As tool description states it is targeted for small to medium networks (which of course is different in definition across the world):

(…) recommended up to 20 servers and up to 500 client computers (…)

(I think that mostly because it runs with WMI and this might be a bit expensive for larger networks) but it still might be very useful for many environments.

Why my first touch with this tool was very quick … well as it states in its description it doesn’t support W2008R2 yet and all my virtual labs at this moment runs in full W2008R2 mode. Because of that my first approach to test it ended in discovery phase. But definitely I will build different lab and I will try to use it to analyze healthy AD and also directory with some issues.

Anyway … worth to remember … worth to have it in a toolbox.

PS. As it is a case in many automated scanning tools report should be read with understanding and caution ;) .

Where to put SSL certificate for LDAP …

tomek — Wed, 17 Jun 2009 12:06:00 GMT

Protecting LDAP traffic with SSL is a good idea, especially if in network environment some applications are (ab)using LDAP as authentication protocol.

Some explanation of abusing word – LDAP never was designed as authentication protocol (like Kerberos is). Its name states it clear “DIRECTORY ACCESS PROTOCOL”. However because it is simple to use and effective it is often used as such. Because it wasn’t designed to be an authentication protocol it lacks features which would protect credentials etc. which might expose authentication data for different threats which are common for every important information sent in a clear text over a network. To say it in simple word – when you are doing simple LDAP bind over non secure connection you are just exposing your credentials to others. For proof see see screenshot from network trace below:

So I think this screenshot itself should be enough to start to think about securing LDAP traffic with SSL (convincing developers to use different protocol or to enforce SSL connection in an app might be on the other hand tough task).

Getting back to the topic – good news is that Active Directory infrastructure makes it pretty easy to enable SSL on LDAP protocol. Just give DC proper certificate and it will start to accept LDAP over SSL connections. “Proper” means that it has to meet some criteria, one of them is that its purpose statement should contain Server Authentication (1.3.6.1.5.5.7.3.1) OID on a list.

Deploying certificates in environment with Enterprise CA based on Windows Server integrated with AD is also easy – you can take advantage of auto enrollment feature and DCs will request and install certificates on their own. It is also possible to use third party certificates. Procedure for requesting and installing such certificates on DCs in described in KB321051. In both cases (auto enrollment and KB manual procedure) certificate is being installed in Personal store of local system.

This store has this disadvantage that it might contain many different certs installed by other services and applications if required. If it will contain many certificates meeting requirements for DC to use this cert to protect LDAP traffic it will just pick one. We don’t have real control which one.

Since Windows 2008 there is a way to have more control on this behavior and select certificate which will be used to protect LDAP traffic. If certificate will be put in NTDSA\Personal store instead of default Local system store it will be picked up by directory service in first place and used for LDAP traffic protection.

If there will not be any of certificates in NTDSA\Personal store DC will fall back to old behavior and search for certificate in Local system store

Looks easy … one problem I have with this solution is that I can’t find a way to use auto enrollment mechanism to enroll certificate for DC and put it directly in NTDSA store. If there is anyone who have idea how to do this … comments are open :).