Hi all, Ned here again with an actual back to back mail sack. This week we discuss:

• Running out of USNs and Versions
• DFSR RDC LAN WAN FWIW AOK
• NPS and dotted NetBIOS domain names
• USMT and the case of the failing sourcepriority
• Revisiting NIC teaming
• Weird DFSR files
• MaxConcurrentAPI in depth (elsewhere)
• KB2663685 goodness
• Other stuff

Question

I was reading an article that showed how to update the computer description every time a user logs on. A commenter mentioned that people should be careful as the environment could run out of USNs if this was implemented. Is that true?

Answer

This was a really interesting question. The current USN is a 64-bit counter maintained by each Active Directory domain controller as the highestCommittedUsn attribute on the rootDSE object.  Being an unsigned 64-bit integer, that means 2⁶⁴-1, which is 18,446,744,073,709,551,615 (i.e. 18 quintillion). Under normal use that is never going to run out. Even more, when AD reaches that top number, it would restart at 1 all over again!

Let's say I want to run out of USNs though, so I create a script that makes 100 object write updates per second on at DC. It would take me 54 days to hit the first 1 billionth USN. At that rate, this means I am adding ~6.5 billion USN changes a year. Which means at that rate, it would take just under 3 billion years to run out on that DC. Which is probably longer than your hardware warranty.

My further thought was around Version metadata, which we don't document anywhere I can find. That is an unsigned 32-bit counter for each attribute on an object and again, so huge it is simply not feasible that it would run out in anything approaching normal circumstances. If you were to update a computer’s description every time a user logged on and they only had one computer, at 2³²-1 that means they have to logon 4,294,967,295 times to run out. Let’s say they logon in the morning and always logoff for bathroom, coffee, meetings and lunch breaks rather than locking their machines – call it 10 logons a day and 250 working days a year. That is still 1.7 million years before they run out and you need to disjoin, rename, and rejoin their computer so they can start again.

That said - the commenter was a bit off about the facts, but he had the right notion: not re-writing attributes with unchanged data is definitely a good idea. Less spurious work is always the right answer for DC performance and replication. Figure out a less invasive way to do this, or even better, use a product like System Center Config Manager; it has built in functionality to determine the “primary user” of computers, involving auditing and some other heuristics. This is part of its “Asset Intelligence” reporting (maybe called something else in SCCM 2012).

Interesting side effect of this conversation: I was testing all this out with NTDSUTIL auth restores and setting the version artificially high on an object with VERINC. Repadmin /showmeta gets upset once your version crosses the 2³¹ line. J See for yourself (in a lab only, please). If you ever find yourself in that predicament, use LDP's metadata displayer, it keeps right on trucking.

Maybe a li'l ol' casting issue here

Ahh, that's better. Get out the hex converter.

Question

I find replication to be faster with RDC disabled on my LAN connected servers (hmmm, just like your blog said J), so I have it disabled on the connections between my hub servers and the other servers on the same LAN. I have other servers connected over a WAN, so I kept RDC enabled on those connections.

By having some connections with RDC enabled and others disabled, am I making my hub server do ‘twice’ the work? Would it be better if I enabled it on all connections, even the LAN ones?

Answer

You aren’t making your servers do things twice, per se; more like doing the same things, then one does a little more.

Consider a change made on the hub: it still stage the same file once, compresses it in staging once, creates RDC signatures for it once, and sends the overall calculated SHA-1 file hash to each server once. The only difference will be that one spoke server then receives the whole file and the other spoke does the RDC version vector and signature chunk dance to receive part of the file.

The non-RDC LAN-based communication will still be more efficient and fast within its context, and the WAN will still get less utilization and faster performance for large files with small changes.

Question

I'm trying to get Network Policy Server (RADIUS) to work in my environment to enable WPA-2 authentication from a slick new wireless device. I keep getting the error "There is no domain controller available for domain CONTOSO.COM" in the event log when I try to authenticate, which is our legacy dotted NetBIOS domain name. On a hunch, I created a subdomain without a dot in the NetBIOS name and was able to authenticate right away with any user from that subdomain. Do you have any tricks or advice on how to deal with NPS in a dotted domain running in native Windows 2008 R2 mode other than renaming it (yuck).

Answer

I don't even know how to spell NPS (it's supported by our Networking team) but I found this internal article from them. You are not going to like the answer:

Previous versions of IAS/NPS could not perform SPN lookups across domains because it treated the SPN as a string and not an FQDN. Windows Server 2008 R2 corrected that behavior, but now NPS is treating a dotted NetBIOS name as a FQDN and NPS performs a DNS lookup on the CONTOSO.COM name. This fails because DNS does not host a CONTOSO.COM zone.

That leaves you with three main solutions:

• Rename your domain using rendom.exe
• Migrate your domain using ADMT
• Use a Windows Server 2008 NPS

There might be some other workaround - this would be an extremely corner case scenario and I doubt we've explored it deeply.

The third solution is an ok short-term workaround, but Win2008 isn’t going to be supported forever and you might need some R2 features in the meantime. The first two are gnarly, but I gotta tell ya: no one is rigorously testing dotted NetBIOS names anymore, as they were only possible from NT 4.0 domain upgrades and were rare as an honest politician. They are ticking time bombs. A variety of other applications and products fail when trying to use dotted NetBIOS domain names and they might not have a workaround. A domain rename is probably in your future, and it's for the best.

Question

We are using USMT 4.0 to migrate data with the merge script sourcepriority option to always overwrite data on the destination with data from the source. No matter what though, the destination always wins and the source copy of the file is renamed with the losing (1) tag.

Answer

This turned out to be quite an adventure.

We turned on migdiag logging using SET MIG_ENABLE_DIAG=migdiag.xml in order to see what was happening here; that's a great logging option for figuring out why your rules aren’t processing correctly. When it got to the file in question during loadstate, we saw this weirdness:

<Pattern Type="File" Path="C:\Users\someuser\AppData\Local\Microsoft\Windows Sidebar [Settings.ini]" Operation="DynamicMerge,&lt;unknown&gt;"/>

Normally, it should have looked like:

<Pattern Type="File" Path="C:\Users\someuser\AppData\Roaming\Microsoft\Access\* [*]" Operation="DynamicMerge,CMXEMerge,CMXEMergeScript,MigXmlHelper,SourcePriority"/>

More interestingly, none of us could reproduce the issue here using the customer's exact same XML file. Finally, I had him reinstall USMT from a freshly downloaded copy of the WAIK, and it all started working perfectly. I've done this a few times in the past with good results for these kinds of weirdo issues; since USMT cannot be installed on Windows XP, it just gets copied around as folders. Sometimes people start mixing in various versions and DLLS, from Beta, RC, and hotfixes, and you end up with something that looks like USMT - but ain't.

Question

Is teaming network adapters on Domain Controllers supported by Microsoft? I found KB http://support.microsoft.com/kb/278431.

Answer

Sharp-eyed readers know I was already asked this before. However, I have a new answer: yes, if you use Windows Server "8" Beta.

Whoa, we joined the 1990s! Seriously though, NIC teaming is the bane of our Networking team's existence, so hopefully by creating and implementing our own driver system we stop the pain customers have using third party solutions of variable quality. At least we'll be able to see what's wrong now if it doesn’t work.

For a lot more info, grab the whitepaper.

Question

What are the DFSR files $db_dirty$, $db_normal$, and $db_lost$ mentioned in the KB article Virus scanning recommendations for Enterprise computers that are running currently supported versions of Windows ? I only see $db_normal$ on my servers (presumably that's a good thing).

Answer

$Db_dirty$ exists after a dirty database shutdown and acts as a marker of that fact. $Db_normal$ exists when there are no database issues and is renamed to $db_lost$ if the database goes missing, also acting as a state marker for DFSR between service restarts.

Question

Where is the best place to learn more about MaxConcurrentAPI?

Answer

Right here, and only quite recently:

http://social.technet.microsoft.com/wiki/contents/articles/9759.configuring-maxconcurrentapi-for-ntlm-pass-through-authentication.aspx

Not a question (new DFSR functionality in KB 2663685)

If you missed it, we released a new hotfix for DFSR last month that adds some long-sought functionality for file server administrators: the ability to prevent DFSR from non-authoritatively synchronizing replicated folders on a volume where the database suffered a dirty shutdown:

Changes that are not replicated to a downstream server are lost on the upstream server after an automatic recovery process occurs in a DFS Replication environment in Windows Server 2008 R2 - http://support.microsoft.com/kb/2663685

DFSR now provides the capability to override automatic replication recovery of dirty shutdown-flagged databases. By default, the following registry DWORD value exists:

HKLM\System\CurrentControlSet\Services\DFSR\Parameters\

StopReplicationOnAutoRecovery = 1

If set to 1, auto recovery is blocked and requires administrative intervention. Set it to 0 to return to the old behavior.

DFSR writes warning 2213 event to the DFSR event log:

MessageId=2213

Severity=Warning

Message=

The DFS Replication service stopped replication on volume %2.

This occurs when a DFSR JET database is not shut down cleanly and Auto Recovery is disabled. To resolve this issue, back up the files in the affected replicated folders, and then use the ResumeReplication WMI method to resume replication.

Additional Information:

Volume: %2

GUID: %1

Recovery Steps

1. Back up the files in all replicated folders on the volume. Failure to do so may result in data loss due to unexpected conflict resolution during the recovery of the replicated folders.

2. To resume the replication for this volume, use the WMI method ResumeReplication of the VolumeConfig class.

For example, from an elevated command prompt, type the following command:

wmic /namespace:\\root\microsoftdfs path dfsrVolumeConfig where volumeGuid="%1" call ResumeReplication

For more information, see http://support.microsoft.com/kb/2663685.

You must then make a decision about resuming replication. You must weigh your decision against the environment:

• Are there originating files or modifications on this server? You can use the DFSRDIAG BACKLOG command with this server as the sending member and each of its partners as the receiving member to determine if this server had any pending outbound replication.
• Do you need an out of band backup? You can check you latest backup logs and compare to file contents to see if you should first backup the RFs.
• Are the replicated folders read-only? If so, there is little reason to examine the server further and you can resume replication. It is impossible for the RO RFs to have originated changes in that case.

You then have several options:

• Resume replication. By executing the WMI method listed in the event, the database rebuild commences for all Replicated Folders on that volume. If the database cannot be rebuilt gracefully, DFSR deletes the database and performs initial non-authoritative sync. All data local in those replicated folders is fenced to lose conflict resolutions. Any files that do not match the SHA1 hash of upstream servers move to the circular ConflictAndDeleted folder and, potentially, lost forever.

Example:

Wmic /namespace:\\root\microsoftdfs path dfsrVolumeConfig where volumeGuid="<some GUID>" call ResumeReplication

• Reconfigure replication on RFs to be authoritative. If the data is more up to date on the non-replicating RFs or the RFs are designed to originate data (such as Branch servers replicating back to a central hub for backups), you must manually reconfigure replication to force them to win.

Other Stuff

Holy crap!

Pandora.com is a great way to find new music; I highly recommend it. It can get a little esoteric, though. Real radio will never find you a string duo that plays Guns and Roses songs, for example.

AskDS reader Joseph Moody sent this along to us:

"Because I got tired of forwarding the Accelerating Your IT Career post to techs in our department, we just had it printed poster size and hung it on an open wall. Now, I just point to it when someone asks how to get better."

My wife wanted to be a Marine Biologist (like George Costanza!) when she was growing up and we got on a killer whale conversation last week when I was watching the amazing Discovery Frozen Planet series. She later sent me this tidbit:

"First, the young whale spit regurgitated fish onto the surface of the water, then sank below the water and waited.

If a hungry gull landed on the water, the whale would surge up to the surface, sometimes catching a free meal of his own. Noonan watched as the same whale set the same trap again and again. Within a few months, the whale's younger half brother adopted the practice.

Eventually the behavior spread and now five Marineland whales supplement their diet with fresh fowl, the scientist said."

It's Deep Blue Sea for Realzies!!!

Finally

Have you ever wanted to know what AskDS contributor Rob Greene looks like when his manager 'shops him to a Shrek picture? Now you can:

Have a nice weekend folks,

- Ned “” Pyle

Hi all, Ned here again. We get emailed here all the time about issues involving delays in user logons. Often enough that, a few years back, Bob wrote a multi-part article on the subject.

Taking it to the next level, some of my esteemed colleagues have created a multi-part TechNet Wiki series on understanding, analyzing, and troubleshooting slow logons and slow boots. These include:

• Root causes for slow boots and logons (sbsl) - http://social.technet.microsoft.com/wiki/contents/articles/10130.root-causes-for-slow-boots-and-logons-sbsl.aspx
• Tools for Troubleshooting slow boots and slow logons (sbsl) - http://social.technet.microsoft.com/wiki/contents/articles/10128.tools-for-troubleshooting-slow-boots-and-slow-logons-sbsl.aspx
• Troubleshooting slow operating system boot times and slow user logons (sbsl) - http://social.technet.microsoft.com/wiki/contents/articles/10123.troubleshooting-slow-operating-system-boot-times-and-slow-user-logons-sbsl.aspx

Before you shrug this off, consider the following example, where we assume for our hypothetical company:

• Employees work 250 days per year (50 weeks * 5 days per week)
• Employee labor costs $2 per minute
• Each employees boots and logs on to a single desktop computer only once per day
• There are 25 and 30 seconds of removable delay from the boot and logon operations

That means an annual cost of:

Benjamin Franklin would not be pleased

Even if you take just the understated US Bureau of Labor private sector compensation cost numbers (roughly $0.50 average employee total compensation cost per minute), you are still hemorrhaging cash. And those numbers just cover direct compensation and benefit costs, not all the other overhead  that goes into an employee, as well as the fact that they are not producing anything during that time - you are paying them to do nothing. Need I mention that the computer-using employees are probably costing you nearly twice that number?

Get to reading, people – this is a big deal.

- Ned “a penny saved is a penny earned” Pyle

Hi folks, Ned here again. It’s been a little while since the last sack, but I have a good excuse: I just finished writing a poop ton of Windows Server 2012 depth training that our support folks around the world will use to make your lives easier (someday). If I ever open MS Word again it will be too soon, and I’ll probably say the same thing about PowerPoint by June.

Anyhoo, let’s get to it. This week we talk about:

• Invoke-command and the ActiveDirectory Windows PowerShell module
• The mysterious case of USMT hardlink running out of disk space
• Pre-seeding DFSR with robocopy /B
• Blank client names in auditing
• USMT and Windows 8 modern app breakage  
• OCSP scripting
• WDS + DFSN = SUX
• USMT hates files ending with dots or spaces
• Other stuff

Question

Is it possible to use any ActiveDirectory module cmdlets through invoke-command against a remote non-DC where the module is installed? It always blows up for me as it tries to “locally” (remotely) use the non-existent ADWS with error “Unable to contact the server. This may be because the server does not exist, it is currently down, or it does not have the active directory web services running”

Answer

Yes, but you have to ignore that terribly misleading error and put your thinking cap on: the problem is your credentials. When you invoke-command, you make the remote server run the local PowerShell on your behalf. In this case that remote command has to go off-box to yet another remote server – a DC running ADWS. This means a multi-hop credential scenario. Provide –credential (get-credential) to your called cmdlets inside the curly braces and it’ll work fine.

Question

We are using a USMT /hardlink migration to preserve disk space and increase performance. However, performance is crazy slow and we’re actually running out of disk space on some machines that have very large files like PSTs. My scanstate log shows:

Error [0x000000] Write error 112 for C:\users\ned\Desktop [somebig.pst]. Windows error 112 description: There is not enough space on the disk.[gle=0x00000070]

Error [0x080000] Error 2147942512 while gathering object C:\users\ned\Desktop\somebig.pst. Shell application requested abort![gle=0x00000070]

Answer

These files are encrypted and you are using /efs:copyraw instead of /efs:hardlink. Encrypted files are copied into the store whole instead of hardlink'ing, unless you specify /efs:hardlink. If you had not included /efs, this file would have failed with, "File X is encrypted. Use the /efs option to specify a different way to handle this file".

Yes, I realize that we should probably just require that option. But think of all the billable hours we just gave you!

Question

I was using your DFSR pre-seeding post and am finding that robocopy /B is slows down my migration compared to not using it. Is that required for preseeding?

Answer

The /B mode, while inherently slower, ensures that files are copied using a backup API regardless of permissions. It is the safest way, so I took the prudent route when I wrote the sample command. It’s definitely expected to be slower – in my semi-scientific repro’s the difference was ~1.75 times slower on average.

However, /B not required if you are 100% sure you have at least READ permissions to all files.  As a middle ground approach, you might consider using /ZB to be somewhat faster but then hopefully still copy if there are issues; the downside here is a lot of failures due to permissions might end up making things even slower than just going /B; you will have to test it. You can also add /MT to offset the /B slowness.

Question

Recently I came across an user account that keeps locking out (yes, I've read several of your blogs where you say account lockout policies are bad "Turning on account lockouts is a way to guarantee someone with no credentials can deny service to your entire domain"). We get the Event ID of 4740 saying the account has been locked out, but the calling computer name is blank:

Log Name:     Security

Event ID:     4740

Level:         Information

Description:

A user account was locked out.

Subject:

Security ID: SYSTEM

Account Name: someaccount

Account Domain: somedomain

Logon ID: 0x3e7

Account That Was Locked Out:

Security ID: somesid

Account Name: someguy

Additional Information:

Caller Computer Name:

The 0xC000006A indicates a bad password attempt. This happens every 5 minutes and eventually results in the account being locked out. We can see that the bad password attempts are coming via COMP1 (which is a proxy server) however we can't work out what is sending the requests to COMP1 as the computer is blank again (there should be a computer name).

Are we missing something here? Is there something else we could be doing to track this down? Is the calling computer name being blank indicative of some other problem or just perhaps means the calling device is a non-Microsoft device?

Answer

(I am going to channel my inner Eric here):

A blank computer name is not unexpected, unfortunately. The audit system relies on the sending computers to provide that information as part of the actual authentication attempt. Kerberos does not have a reliable way to provide the remote computer info in many cases. Name resolution info about a sending computer is also easily spoofed. This is especially true with transitive NTLM logons, where we are relying on one computer to provide info for another computer. NTLM provides names but they are also easily spoofed so even when you see a computer name in auditing, you are mainly asking an honest person to tell you the truth.

Since it happens very frequently and predictably, I’d configure a network capture on the sending server to run in a circular fashion, then wait for the lock out and stop the event. You’d see all of the traffic and now know exactly who sent it. If the lockout was longer running and less predictable, I’d recommend using a network capture configured to trace in a circular fashion until that 4740 event writes. Then you can see what the sending IP address is and hunt down that machine. Different techniques here:

• http://nm3eventcap.codeplex.com/
• http://blogs.technet.com/b/netmon/archive/2007/02/22/eventmon-stopping-a-capture-based-on-an-eventlog-event.aspx

[And the customer later noted that since it’s a proxy server, it has lots of logs – and they told him the offender]

Question

I am testing USMT 5.0 and finding that if I migrate certain Windows 7 computers to Windows 8 Consumer Preview, Modern Apps won’t start. Some have errors, some just start then go away.

Answer

Argh. The problem here is Windows 7’s built-in manifest that implements microsoft-windows-com-base , which then copies this registry key:

HKEY_LOCAL_MACHINE\Software\Microsoft\OLE

If the DCOM permissions are modified in that key, they migrate over and interfere with the ones needed by Modern Apps to run. This is a known issue and already fixed so that we don’t copy those values onto Windows 8 anymore. It was never a good idea in the first place, as any applications needing special permissions will just set their own anyways when installed.

And it’s burned us in the past too…

Question

Are there any available PowerShell, WMI, or command-line options for configuring an OCSP responder? I know that I can install the feature with the Add-WindowsFeature, but I'd like to script configuring the responder and creating the array.

Answer

[Courtesy of the Jonathan “oh no, feet!” Stephens – Ned]

There are currently no command line tools or dedicated PowerShell cmdlets available to perform management tasks on the Online Responder. You can, however, use the COM interfaces IOCSPAdmin and IOSCPCAConfiguration to manage the revocation providers on the Online Responder.

1. Create an IOSCPAdmin object.
2. The IOSCPAdmin::OCSPCAConfigurationCollection property will return an IOCSPCAConfigurationCollection object.
3. Use IOCSPCAConfigurationCollection::CreateCAConfiguration to create a new revocation provider.
4. Make sure you call IOCSPAdmin::SetConfiguration when finished so the online responder gets updated with the new revocation configuration.

Because these are COM interfaces, you can call them from VBScript or PowerShell, so you have great flexibility in how you write your script.

Question

I want to use Windows Desktop Search with DFS Namespaces but according to this TechNet Forum thread it’s not possible to add remote indexes on namespaces. What say you?

Answer

There is no DFSN+WDS remote index integration in any OS, including Windows 8 Consumer Preview. At its heart, this comes down to being a massive architectural change in WDS that just hasn’t gotten traction. You can still point to the targets as remote indexes, naturally.

Question

Certain files – as pointed out here by AlexSemi – that end with invalid characters like a dot or a space break USMT migration. One way to create these files is to use the echo command into a device path like so:

These files can’t be opened by anything in Windows, it seems.

When you try to migrate, you end up with a fatal “windows error 2” “the system cannot find the file specified” error unless you skip the files using /C:

What gives?

Answer

Quit making invalid files! :-)

USMT didn’t invent CreateFile() so its options here are rather limited… USMT 5.0 handles this case correctly through error control - it skips these files when hardlink’ing because Windows returns that they “don’t exist”. Here is my scanstate log using USMT 5.0 beta, where I used /hardlink and did NOT provide /C:

In the case of non-hardlink, scanstate copies them without their invalid names and they become non-dotted/non-spaced valid files (even in USMT 4.0). To make it copy these invalid files with the actual invalid name would require a complete re-architecting of USMT or the Win32 file APIs. And why – so that everyone could continue to not open them?

Other Stuff

In case you missed it, Windows 8 Enterprise Edition details. With all the new licensing and activation goodness, Enterprise versions are finally within reach of any size customer. Yes, that means you!

Very solid Mother’s Day TV mash up (a little sweary, but you can’t fight a something that combines The Wire, 30 Rock, and The Cosbys)

Zombie mall experience. I have to fly to Reading in June to teach… this might be on the agenda

Well, it’s about time - Congress doesn't "like" it when employers ask for Facebook login details

Your mother is not this awesome:

That, my friend, is a Skyrim birthday cake

SportsCenter wins again (thanks Mark!)

Don’t miss the latest Between Two Ferns (veeerrrry sweary, but Zach Galifianakis at his best; I just wish they’d add the Tina Fey episode)

But what happens if you eat it before you read the survival tips, Land Rover?!

Until next time,

- Ned “demon spawn” Pyle

Hey all, Ned here with a quick “xerox” post: the Dynamic Access Control developers have released a good intro on their octo-feature through the Windows Server Blog:

Introduction to Windows Server 2012 Dynamic Access Control

It’s written by Nir Ben-Zvi, a Program Manager on the Windows Server development team. If you’re unfamiliar with DAC, this is a great first read. Here’s a quote:

These focus areas were then translated to a set of Windows capabilities that enable data compliance in partner and Windows-based solutions.

• Add the ability to configure Central Access and Audit Policies in Active Directory. These policies are based on conditional expressions that take into account the following so that organizations can translate business requirements to efficient policy enforcement and considerably reduce the number of security groups needed for access control:
  • Who the user is
  • What device they are using, and
  • What data is being accessed
• Integrate claims into Windows authentication (Kerberos) so that users and devices can be described not only by the security groups they belong to, but also by claims such as: “User is from the Finance department” and “User’s security clearance is High”
• Enhance the File Classification Infrastructure to allow business owners and users to identify (tag) their data so that IT administrators are able to target policies based on this tagging. This ability works in parallel with the ability of the File Classification Infrastructure to automatically classify files based on content or any other characteristics
• Integrate Rights Management Services to automatically protect (encrypt) sensitive information on servers so that even when the information leaves the server, it is still protected.

Click to the read the rest.

If you are looking for more depth and “how it works”, check out our very own Mike Stephens’ downloadable whitepaper:

Understand and Troubleshoot Dynamic Access Control in Windows Server "8" Beta

Until next time,

Ned “10 cent copies” Pyle

Hey all, Ned here. The Building Windows 8 blog recently posted a new article from Chris Clark that you might have passed over due to the title, which sounds like another article on boot performance:

Designing for PCs that boot faster than ever before

Don’t skip it! A more descriptive title would have been “The F8 and F2 boot menus are gone on Windows 8 and you need to read this post to do your job, IT Pro.”

Windows 8 is designed to run on hardware that boots too fast for a human to react through a keyboard, requiring new methods to get to a boot menu. Note down what the article describes so that when you need to troubleshoot a slow logon or slow boot, you can get into Safe Mode and other diagnostic states (PC Settings, shift+restart, shutdown.exe /o /r, msconfig.exe). All of these apply to Windows 8 Developer Preview and Windows Server “8” Beta, which you can get your hot little hands on right now.

It is also important to note – and not mentioned in the article – that on Windows Server 2012 only, you can still use F8. The new boot menu system eventually gets you back to the familiar menu with your favorite DSRM option too, so don’t feel like we’re making you relearn everything:

Also not mentioned but preemptively answered now: while shutdown /o was updated to support the new boot menus, the restart-computer Windows PowerShell cmdlet was not.

- Ned “Doc Martens” Pyle

Hi all, Ned here. Our friend Nir has another new DAC-related post up, this time on the File Cab blog: 

Getting started with Central Access Policies - Reducing security group complexity and achieving data access compliance using Dynamic Access Control

If you need a reason to go read this, consider the following quote:

"So, we have 2,000 groups, 2,000 ACLs and many groups that are affected by a person changing a role not to mention the complexity of adding another level (say Branch) or the implications if we want to change the folder structure.

With Dynamic Access Control, you can cut the number of groups down from 2,000 to 71 (50 for country, 20 for department and 1 for sensitive data access). This is made possible by the ability to use expressions in Windows ACL. For example: You would use MemberOf (Spain_Security_Group) AND MemberOf (Finance_Security_Group) AND MemberOf(Sensitive_Security_Group) to limit access to Spain’s finance department sensitive information."

Get on over there and give it a read.

I swear we are going to post some original content here at some point. Just crushed under the load.

- Ned "sock puppet" Pyle

Ned here. Go there:

• Download Windows 8 Release Preview ISO images
• Download Windows Server 2012 Release Candidate Datacenter (ISO and VHD images)
• Windows Assessment and Deployment Kit (ADK) for Windows 8 Release Preview (includes USMT 5.0)
• Remote Server Administration Tools for Windows 8 Release Preview

While you wait for the downloads, check out Delivering the Windows 8 Release Preview on the B8 blog to learn a few more things about Windows 8 from a non-enterprise standpoint. More Windows 8 here. More Windows Server 2012 here.

IT Pro content will trickle out; keep your eyes peeled on the Windows Server blog and elsewhere. And know this: we have everything documented to the nth degree and you will learn everything you need, at least after RTM, even if it harelips the governor.

See you then.

Ned “Slim Pickens” Pyle

Hey all, Ned here. Mike and I start teaching Windows Server 2012 and Windows 8 DS internals this month in the US and UK and won’t be back until July. Until then, Jonathan is – I can’t believe I’m saying this – in charge of AskDS. He’ll field your questions and publish… stuff. We’ll make sure he takes his medication before replying.

If you’re in Reading, England June 10-22, first round is on me.

I didn’t say what the first round was though.

Ned “crikey” Pyle

[This is another guest post from our pal Mark in Oz. Even if you don’t care about DFSR, I highly recommend this post; it teaches some very clever log analysis techniques, useful in a variety of troubleshooting scenarios – The Neditor]

Hi there! It’s Mark Renoden – Premier Field Engineer in Sydney, Australia – here again. Today I’m going to talk about an issue where a customer's DFSR environment lost file updates and they’d see regular alerts in SCOM about replication backlog. While I was on site working with them, I came up with a few creative ideas about how to use the DFSR debug logs that led us to the root cause.

The problem at hand was that a large DFS replication backlog would accumulate from time to time. Finding the root cause meant understanding the trend in changes to files in the replica. To do this, we needed to use the debug logs as our data source: to manipulate them so that they would tell the story.

With the aid of some custom scripts and tools, and a lab environment, I’m going to simulate their experience and talk through the investigation.

Test Lab Setup

The test lab I’ve used for this post is pretty simple: I’ve got two file servers, DPS1 and DPS2 configured with a single replication group called RG1 that replicates C:\ReplicatedFolder between the servers. There are 100,000 files in C:\ReplicatedFolder.

Prepare for Battle

In Ned’s previous post Understanding DFSR Debug Logging (Part 1: Logging Levels, Log Format, GUID’s), the various options for debug log verbosity are discussed. For this scenario, the only one I’ll change is the number of debug log files. 1000 is generally a good number to choose for troubleshooting purposes:

Harvest the Data

After reproducing the symptoms, we want to harvest the logs from the server that has pending replication. When on site at the customer, I just copied DFSR*.log.gz from the %windir%\debug folder, but the best possible practice would be to stop DFSR, copy the logs and then start the service again. This would prevent log rollover while you harvest the logs.

After you copy the logs for investigation, they need to be un-g-zipped. Use your favourite gzip-aware decompression utility for that.

Understand the Log Format

Before we can mine the debug logs for interesting information, we need to look at what we’re dealing with. Opening up one of the logs files, I want to look for a change and to understand the log format –

20120522 12:39:57.764 2840 USNC  2450 UsnConsumer::UpdateIdRecord LDB Updating ID Record:
+       fid                             0x2000000014429
+       usn                             0x693e0ef8
+       uidVisible                      1
+       filtered                        0
+       journalWrapped                  0
+       slowRecoverCheck                0
+       pendingTombstone                0
+       internalUpdate                  0
+       dirtyShutdownMismatch           0
+       meetInstallUpdate               0
+       meetReanimated                  0
+       recUpdateTime                   20120521 01:04:21.513 GMT
+       present                         1
+       nameConflict                    0
+       attributes                      0x20
+       ghostedHeader                   0
+       data                            0
+       gvsn                            {5442ADD7-04C7-486B-B665-2CB036997A67}-v937024
+       uid                             {5442ADD7-04C7-486B-B665-2CB036997A67}-v615973
+       parent                          {8A6CF487-2D5A-456C-A235-09F312D631C8}-v1
+       fence                           Default (3)
+       clockDecrementedInDirtyShutdown 0
+       clock                           20120522 02:39:57.764 GMT (0x1cd37c42d5a9268)
+       createTime                      20120516 00:41:05.011 GMT
+       csId                            {8A6CF487-2D5A-456C-A235-09F312D631C8}
+       hash                            00000000-00000000-00000000-00000000
+       similarity                      00000000-00000000-00000000-00000000
+       name                            file0000021380.txt
+      
20120522 12:39:59.326 2840 USNC  2453 UsnConsumer::UpdateIdRecord ID record updated from USN_RECORD:
+       USN_RECORD:
+       RecordLength:        96
+       MajorVersion:        2
+       MinorVersion:        0
+       FileRefNumber:       0x2000000014429
+       ParentFileRefNumber: 0x4000000004678
+       USN:                 0x693e0ef8
+       TimeStamp:           20120522 12:39:57.764 AUS Eastern Standard Time
+       Reason:              Close Security Change
+       SourceInfo:          0x0
+       SecurityId:          0x0
+       FileAttributes:      0x20
+       FileNameLength:      36
+       FileNameOffset:      60
+       FileName:            file0000021380.txt

What I can see here is a local database update followed by the USN record update that triggered it. If I can gather together all of the date stamps for USN record updates, perhaps I can profile the change behaviour on the file server…

The command above finds every line in every log file that contains USN_RECORD: and then excludes lines that contain a + (thereby eliminating occurrences of + USN_RECORD: as seen in the log excerpt above). Finally, it directs that output into USN.csv.

Let’s open our CSV file in Excel and see what we can do with it.

Graph the Data

I'd ideally like to graph the data that I've got, to make it easy to spot trends. The data I have right now isn't super-easy to work with, so I'm going to sanitize it a bit, and then make a Pivot Table and chart from the sanitized data.

Here is a single column of USN_RECORD: timestamps:

I’d like to figure out the rate of change on the file system for files in the replicated folder so I’ll use text to columns. I’m using a fixed width conversion and I’m going to split out my timestamp to the minute (so I can see how many changes per minute I have) and I’ll split USN_RECORD: off the end of the line so that I have something to count:

Now I’ve got columns like this:

I delete the columns I don’t need (A and C). My result is a column of timestamps down to the minute and a column of identical values (column B) which I can count to understand the rate of change:

To do this, I insert a pivot table. I simply select columns A and B and then choose PivotTable from the Insert menu in Excel.

Now I configure my PivotTable Field List as follows:

After configuring the PivotTable, it looks like this

All that’s left for me to do is to click on one of the row labels and to select a chart from the Insert menu. The resulting chart tells us quite a lot:

Here I can see that there are constant changes at roughly 230 per minute, and that for a two-hour window, the changes increase to about 1500 per minute.

Conclusions so far

For the entire duration of the logs, a roughly consistent level of change was occurring. However, for a two-hour window, lots of change was occurring. There are two possibilities here: either the cause of change has become more aggressive during this time or this chart represents two different activities.

We need more investigation …

Back to the Debug Logs

I start by skimming a log that contains the timestamps from the two-hour window where we see many changes, and look at the USN record updates. Skimming through, I can see two different types of change:

20120523 10:54:41.249 2840 USNC  2453 UsnConsumer::UpdateIdRecord ID record updated from USN_RECORD:
+       USN_RECORD:
+       RecordLength:        96
+       MajorVersion:        2
+       MinorVersion:        0
+       FileRefNumber:       0x20000000175FA
+       ParentFileRefNumber: 0x4000000004678
+       USN:                 0xbf0ad430
+       TimeStamp:           20120523 10:54:39.827 AUS Eastern Standard Time
+       Reason:              Close Security Change
+       SourceInfo:          0x0
+       SecurityId:          0x0
+       FileAttributes:      0x20
+       FileNameLength:      36
+       FileNameOffset:      60
+       FileName:            file0000031231.txt

And:

20120523 10:54:41.249 2840 USNC  2085 UsnConsumer::UpdateUsnOnly USN-only update from USN_RECORD:
+    USN_RECORD:
+    RecordLength:        96
+    MajorVersion:        2
+    MinorVersion:        0
+    FileRefNumber:       0x2000000019AD4
+    ParentFileRefNumber: 0x4000000004678
+    USN:                 0xbf0ad4f0
+    TimeStamp:           20120523 10:54:39.843 AUS Eastern Standard Time
+    Reason:              Basic Info Change Close
+    SourceInfo:          0x0
+    SecurityId:          0x0
+    FileAttributes:      0x20
+    FileNameLength:      36
+    FileNameOffset:      60
+    FileName:            file0000038828.txt

Skimming a log that covers a timeframe with a low rate of change, I can only seem to find:

20120522 23:28:54.953 2840 USNC  2453 UsnConsumer::UpdateIdRecord ID record updated from USN_RECORD:
+    USN_RECORD:
+    RecordLength:        96
+    MajorVersion:        2
+    MinorVersion:        0
+    FileRefNumber:       0x2000000022440
+    ParentFileRefNumber: 0x4000000004678
+    USN:                 0x7e7f5188
+    TimeStamp:           20120522 23:28:52.984 AUS Eastern Standard Time
+    Reason:              Close Security Change
+    SourceInfo:          0x0
+    SecurityId:          0x0
+    FileAttributes:      0x20
+    FileNameLength:      36
+    FileNameOffset:      60
+    FileName:            file0000072204.txt

Now I have a theory – Basic Info Change Close events only occur during the two-hour window where there are many changes and there’s an underlying and ongoing security change the rest of the time. I can prove this if I extract the timestamps for Basic Info Change Close changes and similarly, extract timestamps for Close Security Change changes.

Looking back at the log entries, I can see I have a time stamp followed by a series of lines that start with a +. I need to parse the log with something (I chose PowerShell) that takes note of the timestamp line and when a Basic Info Change Close or Close Security Change follows soon after, return the timestamp.

Here’s my PS script:

$files = Get-ChildItem *.log

$processingBlock = $False
$usnBlock = $False

foreach ($file in $files)
{
    $content = Get-Content $file
    foreach ($line in $content)
    {
        if (!($line.ToString().Contains("+")))
        {
            $outLine = $line
            $processingBlock = $True
        }
        if ($processingBlock)
        {
            if ($line.ToString().Contains("+    USN_RECORD:"))
            {
                $usnBlock = $True
            }
        }
        if ($usnBlock)
        {
            if ($line.ToString().Contains("+    Reason:              Basic Info Change Close"))
            {
                $outLine.ToString()
                $processingBlock = $False
                $usnBlock = $False
            }
        }
    }
}

And:

$files = Get-ChildItem *.log

$processingBlock = $False
$usnBlock = $False

foreach ($file in $files)
{
    $content = Get-Content $file
    foreach ($line in $content)
    {
        if (!($line.ToString().Contains("+")))
        {
            $outLine = $line
            $processingBlock = $True
        }
        if ($processingBlock)
        {
            if ($line.ToString().Contains("+    USN_RECORD:"))
            {
                $usnBlock = $True
            }
        }
        if ($usnBlock)
        {
            if ($line.ToString().Contains("+    Reason:              Close Security Change"))
            {
                $outLine.ToString()
                $processingBlock = $False
                $usnBlock = $False
            }
        }
    }
}

I run each of these (they take a while) against the debug log files and then chart the results in exactly the same way as I’ve done above.

First, Basic Info Change Close (look at the time range covered and number plotted):

And Close Security Change, below:

This confirms the theory – Basic Info Change Close takes place in the two hours where there’s a high rate of change and Close Security Change is ongoing.

Root Cause Discovery

If this is an ongoing pattern where the high rate of change occurs during the same two hours each day, I can capture both activities using Process Monitor.

Once I have a trace, it’s time to filter it and see what’s happening:

Here I’ve reset the filter and added Operation is SetBasicInformationFile then Include. I chose SetBasicInformationFile because it looks like a good fit for the USN record updates labelled Basic Info Change Close. After clicking OK, my filtered trace has the answer…

As it turns out, the backup window matches nicely with the storm of Basic Info Change Close updates.

Clearly, this is my own little application replicating behaviour but in the case of my customer, it was actually their backup application causing this change. They were able to talk to their vendor and configure their backup solution so that it wouldn’t manipulate file attributes during backups.

Now all we need to do is identify the source of Close Security Change updates. Once again, I reset the filter and look for an operation that sounds like a good match. SetSecurityFile looks good.

What I found this time is that no entries show up in Process Monitor

What explains this? Either I chose the wrong operation or the filter is broken in some other way. I can’t see any other sensible operation values to filter with so I’ll consider other options. Looking at the filter, I realize that perhaps System is responsible for the change and right now, Procmon filters that activity out. I remove the exclusion of System activity from my filter and see what happens:

Aha! Now I’ve got something:

Now I need to understand what System is doing with these files. I right click the path for one of these entries and select “Include C:\ReplicatedFolder\file…”:

I also need to remove the filter for SetSecurityFile:

In summary, I’m interested in everything that happened to file0000033459.txt:

If I look at operations on the file that took place prior to SetSecurityFile, I can see a CreateFile operation. This is where System obtained a handle to the file. Looking at this entry, adding the Details column to Process Monitor and examining the fine print I find:

System is making this change in the context of the account CONTOSO\ACLingApp that just happens to be the service account of an application used to change permissions on resources in the environment.

Conclusion

The process I've described today is a good example of the need to Understand the System from my earlier post. The Event Logs - and even the debug logs - won’t always tell you the answer straight away. Know what you’re trying to achieve, know how to use the tools in your arsenal, and know how they can be made to produce the outcome you need.

Knowing what I know now, I might have found the root cause by starting with Process Monitor but there’s a chance I’d have missed Close Security Change updates (considering that System is excluded from Process Monitor by default). I may have also missed the Basic Info Change Close updates if the tracing interval wasn’t aligned with the backup window. By mining the debug logs, I was able to establish there were two separate behaviours and the appropriate times to gather Process Monitor logs.

- Mark “Spyglass” Renoden

Hi folks, Jonathan here. Dave and I wanted to share some important information with you.

By now you’ve all been made aware of the Microsoft Security Advisory that was published this past Sunday.  If you are a Terminal Services or Remote Desktop Services administrator then we have some information of which you should be aware.  These are just some extra administrative steps you’ll need to follow the next time you have to obtain license key packs, transfer license key packs, or any other task that requires your Windows Server license information to be processed by the Microsoft Product Activation Clearinghouse.  Since there’s a high probability that you’ll have to do that at some point in the future we’re doing our part to help spread the word.  Our colleagues over at the Remote Desktop Services (Terminal Services) Team blog have posted all the pertinent information. Take a look.

Follow-up to Microsoft Security Advisory 2718704: Why and How to Reactivate License Servers in Terminal Services and Remote Desktop Services

If you have any questions, feel free to post them over in the Remote Desktop Services forum.

Jonathan Stephens

Hi folks, Jonathan here again. Adam Conkle has posted some new goodies related to Office 365 and AD FS/DirSync that may interest you. Take a look, and be sure to rate the content if you find it useful.

----

Client Access Policy Builder - http://gallery.technet.microsoft.com/scriptcenter/Client-Access-Policy-30be8ae2

Short description:

Hotfix Rollup Update 2 for AD FS 2.0 had a new feature for Office 365 customers named Client Access Policy. Some organizations may want to create policies that limit access to Office 365 services, depending on where the client resides. For example, you might want to:

• Block all extranet client access to Office 365
• Block all extranet client access to Office 365, except for devices accessing Exchange Online for Exchange Active Sync

AD FS 2.0 provides a way for organizations to configure these types of policies. Office 365 customers using identity federation who require these policies can now use client access policy rules to restrict access based on the location of the computer or device that is making the request.

Today, customers must manually implement Client Access Policy rules within AD FS 2.0, and it can be confusing. The goal of Client Access Policy Builder is to automate the creation of these rules for the most common use case scenarios.

Microsoft Office 365 Federation Metadata Update Automation Installation Tool - http://gallery.technet.microsoft.com/scriptcenter/Office-365-Federation-27410bdc

Short description:

This tool can be used to automate the update of the Microsoft Office 365 federation metadata regularly to ensure that changes in the case of the token signing certificate configured in Active Directory Federation Services 2.0 are replicated to the identity platform automatically.

Microsoft Office 365: DirSync - Count Total Synchronized Objects - http://gallery.technet.microsoft.com/office/Office-365-DirSync-Count-617d9268

Short description:

‘DirSync - Count Total Synchronized Objects' shows total counts of users, groups, contacts, and grand total objects by extracting the FIM SourceAD Connector Space data to an XML file named SourceAD.xml.

When an object makes its way past the Office 365 DirSync filters, they become 'synchronized holograms', and this tool parses the SourceAD.xml looking for synchronized holograms and counts the objects by object type.

The Office 365 Deployment Readiness Tool (DRT) assumes that you have not yet deployed Office 365 DirSync, and it is giving you total object counts, without having the ability to see if those objects will actually make it past the DirSync filters. 'DirSync - Count Total Synchronized Objects' allows you, now that you are in production with DirSync, to see more accurate numbers of objects that have made it past the filters.

---

Be sure to keep those questions coming. Yes, Ned is going to be busy for the next several weeks, but the entire Directory Services team is available to answer questions and point you to the best available resources.

Jonathan “What’s a lackey?” Stephens

[Editors Note: This is a guest post from Joe Isenhour, a Senior Program Manager in the Microsoft Engineering Excellence Center.]

Notes from the Field

Customers can come up with some fairly complex requirements for access control. It can be a challenge to accommodate these requirements in an Office 365 world. The ADFS claims rule system in ADFS 2.0 UR1 provides some powerful options to implement these controls—and some limitations.

The requirements:

1. No one shall access email via Outlook when off the corporate network
2. Members of a specific security group may not use ActiveSync
3. Members of a specific security group may not access OWA off the corporate network
4. All OWA users must log in via a forms based login

It is important to note that the rule processing system always processes all rules. It is NOT a first match system. Because of this, the first rule is most always an “allow everything” rule followed by additional rules that block some access. Schematically, it could look like this:

1. Allow everyone
2. Block members of the group “Bigwigs” from access to OWA
3. Allow the CEO access to OWA

When the CEO attempts to log in to OWA: Rule #1 allows him, Rule #2 blocks him, and then Rule #3 allows him. It is the state of the last matching rule that determine his final outcome. If the CIO were to attempt to log in, he would only match rules 1 and 2 and therefore would be blocked.

Requirement #1: No one shall access email via Outlook when off the corporate network

This is a fairly common requirement in a lot of enterprise Exchange environments. Security folks get a bit freaked out by the thought that a user could set up Outlook on a home machine. Meeting this challenge is pretty straightforward with ADFS 2.0 claims rules.

First, let’s review a bit how ADFS claims work in an Office 365 deployment. There are two flavors of ADFS claims requests: Active and Passive. When a claims request is created and submitted by the service (O365) it is an “active” request. This seems kind of counter-intuitive since you (the client) don’t have to do anything for these requests. The active/passive nature of the request refers to the participation of the _service_, not the client. Examples of O365 services that use active claims requests are Outlook 2007/2010 (RPC + HTTPS, EWS), Outlook 2011 for Mac (EWS), ActiveSync, Autodiscover and Lync Online.

A passive claims request is when the service sends you off to get the claim yourself. The main examples of this in O365 are OWA and Sharepoint Online. When you try to log in to OWA in O365 (and you are using federated identity) your browser is redirected to your ADFS endpoint. There you provide your credentials (either via a web form, basic authentication pop-up or integrated auth.), get your token and then return to OWA with your token in hand.

Back to the issue at hand: Blocking Outlook when client is not connected to the corporate network. To translate that into ADFS speak—We need to block active ADFS claims for the RPC+HTTPS and EWS services if the IP address doesn’t match a known set of corporate addresses.

Logically, the rule will look like this:

If {
     {ClientApplication is RPC
     Or
     ClientApplication is EWS}
     AND
     ClaimType is Active
     AND
     ClientIPAddress is not in <corporate ip address list>}
THEN DENY THE CLAIM

Now let’s translate that to ADFS Claim Rule language:

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy"]) &&
exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/services/trust/2005/usernamemixed"]) &&
exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-client-application", Value == “Microsoft.Exchange.RPC|Microsoft.Exchange.WebServices"]) &&
NOT exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-forwarded-client-ip", Value =~ “<public NAT addresses>"])

=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

Makes perfect sense, right? Oh…it doesn’t? Allow me to break it down:

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy"])

The ‘Type’ x-ms-proxy exists. This simply means that the claim came through an ADFS Proxy server (or other compatible proxy). Note: we are not checking the ‘value’ for this type, just that the type exists.

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/services/trust/2005/usernamemixed"])

The type x-ms-endpoint-absolute-path exists and has a value of usernamemixed. This is the name of the endpoint for _Active_ ADFS Claims. Summary: This is an active ADFS claim.

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-client-application", Value == "”Microsoft.Exchange.RPC|Microsoft.Exchange.WebServices"])

ClientApplication is RPC or WebServices. We can use this ‘or’ (using the ‘|’ character) syntax to check the value field.

NOT exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-forwarded-client-ip", Value =~ “\b192\.168\.4\.([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-5][0-9])\b|\b10\.3\.4\.5\b"])

The value for the type x-ms-forwarded-client-ip has a value that DOES NOT MATCH the regular expression “<public NAT addresses>”. That brings up two important questions:

1. Where does that “x-ms-forwarded-client-ip” come from and what values should I expect to see there?
2. What does the format of the regular expression look like?

According to TechNet:

“This AD FS claim represents a “best attempt” at ascertaining the IP address of the user (for example, the Outlook client) making the request. This claim can contain multiple IP addresses, including the address of every proxy that forwarded the request. This claim is populated from an HTTP header that is currently only set by Exchange Online, which populates the header when passing the authentication request to AD FS.”

(http://technet.microsoft.com/en-us/library/hh526961(v=ws.10).aspx)

So, the value is going to be the border IP address that Exchange Online (EXO) sees for the client. That would be either the border firewall doing NAT/PAT or the border Proxy server. Exchange Online add this IP to the ADFS claim request. Perfect for our Outlook scenario here: Outlook attempt to connect to EXO, EXO builds up a claims request that includes the client IP and heads out to the ADFS endpoint to submit the request.

The second question is a bit easier (or perhaps a bit harder—regular expressions can get complicated) due to the fact that the regular expression format follows the general rules for regular expressions. The Internet is full of regular expression examples to filter IP addresses. For example, let’s say that your network has one block of addresses in use in a NAT pool: 192.168.4.0-192.168.4.255. You also have one satellite office with a single public IP address: 10.3.4.5. An expression you may use could be:

“\b192\.168\.4\.([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-5][0-9])\b|\b10\.3\.4\.5\b”

To break that down:

\b192\.168\.4\.([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-5][0-9])\b applies to the 192.168.4.0-255 network.

\b192\.168\.4\. matches 192.168.4.

[1-9] matches address ending in 1-9

[1-9][0-9] matches 10-99

1[0-9][0-9] matches 100-199

2[0-5][0-9] matches 200-259 (yeah…I know a few more than needed)

The ‘|’ represent “or”

\b10\.3\.4\.5\b applies to the 10.3.4.5 address.

These can get tricky. I recommend you use a regular expression verification tool and test.

Finally, if ALL of these conditions are true:

=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

We deny the claim.

If any one of the elements of the rule evaluate to false, the entire rule is skipped. So, if the client IS coming from one of the addresses that match the regular expression, they do not match this rule.

Requirement #2: Members of a specific security group may not use ActiveSync

The previous example illustrated how to allow or block users based upon where they are. This is a simple example of how to block users based upon who they are.

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy"]) &&
exists([Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/groupsid", Value =~ "Group SID value of allowed AD group"]) &&
exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-client-application, Value == “Microsoft.Exchange.ActiveSync”])

=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

The new element here from the previous example is the “groupsid” type. Yeah, you need to dive into AD and hunt down the SID of the group in question. As is hinted by the “=~” operator, you could create a regular expression that would match more than one group. You could also use the “==” operator and the “|” to do a multiple “or” match.

That one was easy—sets us up well for the next. Which gets a bit…complicated.

Requirement #3: Members of a specific group may only use OWA on the corporate network

We built a rule above that did something very similar for Outlook, couldn’t we just add on or slightly alter that rule? Nope, we can’t. OWA login uses a passive ADFS claim, so the behavior is different. With Outlook, or other active claims, all requests come from O365 and land on the external ADFS proxy. To determine if a user is internal or external, we have to examine the “x-ms-forwarded-client-ip” value. With a passive claim request, like OWA, the client’s browser will be connecting directly to the ADFS endpoint (sts.contoso.com for example). So, we can control who gets in based upon where they are asking. If they ask the external proxies (and they are in the specified group) we say “no”. If they ask the internal ADFS servers we say “yes”.

The rule to accomplish this would look something like this:

exists([Type == http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy])
&& exists([Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/groupsid", Value =~ "S-1-5-21-299502267-1364589140-1177238915-114465"])
&& exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/ls/"])

=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

Line 1: Request went through a proxy

Line 2: User is a member of the specified group

Line 3: This is a passive claim to the /adfs/ls/ endpoint

Line 4: Deny

If they hit the internal ADFS servers directly, line 1 would be false and they would be allowed in.

Requirement #4: Oh…your solution to #3 doesn’t work because we want everyone to use forms-based authentication

Well…shoot. Easy enough to fix, right? Send everyone to the ADFS proxy and add in a line to the above rule that specifies which client IP addresses are allowed. Something like:

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy"])
&& exists([Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/groupsid", Value =~ "S-1-5-21-299502267-1364589140-1177238915-114465"])
&& exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/ls/"])
&& NOT exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-forwarded-client-ip", Value =~ “\b192\.168\.4\.([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-5][0-9])\b|\b10\.3\.4\.5\b"])

=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

I hope the blaring red font illustrates the point that this will not work. Why not? Scroll up a bit to the section on blocking Outlook based on IP address. Notice what fills in that x-ms-forwarded-client-ip? EXO. In this example we are dealing with _passive_ ADFS claims. EXO is not creating this claim—the user is hitting the ADFS login page directly. If you turn on this rule, everyone in the specified group will be blocked no matter where they are coming from. The x-ms-forwarded-client-ip type does not exist at all, so that line will evaluate to true. In order to make it false (and thereby stopping the deny rule from firing on someone) the element would need to exist AND the value need to match the regular expression.

If we can’t use the client’s source IP as a filter, how can we solve this problem?

The answer lies in an ADFS claim element that we have been checking all along, but not checking it fully. Let’s look at the debug log (for more on turning on debugging for ADFS see: http://technet.microsoft.com/en-us/library/adfs2-troubleshooting-configuring-computers(v=WS.10).aspx ).

Event ID 151 AD FS 2.0 Tracing:

<Claims>
ClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname Value CONTOSO\USER1 ValueType http://www.w3.org/2001/XMLSchema#string Issuer AD AUTHORITY OriginalIssuer AD AUTHORITY
ClaimType http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name Value CONTOSO\USER1 ValueType http://www.w3.org/2001/XMLSchema#string Issuer AD AUTHORITY OriginalIssuer AD AUTHORITY
ClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid Value S-1-5-21-3640651473-4051545122-2937135913-1136 ValueType http://www.w3.org/2001/XMLSchema#string Issuer AD AUTHORITY OriginalIssuer AD AUTHORITY
ClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/groupsid Value S-1-5-15 ValueType http://www.w3.org/2001/XMLSchema#string Issuer AD AUTHORITY OriginalIssuer AD AUTHORITY Value S-1-5-11 Value S-1-5-2 Value S-1-5-32-545 Value S-1-1-0 Value S-1-5-21-3640651473-4051545122-2937135913-513
ClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid Value S-1-5-21-3640651473-4051545122-2937135913-513 ValueType http://www.w3.org/2001/XMLSchema#string Issuer AD AUTHORITY OriginalIssuer AD AUTHORITY
ClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/authenticationmethod Value
http://schemas.microsoft.com/ws/2008/06/identity/authenticationmethod/password ValueType http://www.w3.org/2001/XMLSchema#string Issuer LOCAL AUTHORITY OriginalIssuer LOCAL AUTHORITY
ClaimType http://schemas.microsoft.com/ws/2008/06/identity/claims/authenticationinstant Value 2012-04-19T17:32:41.459Z ValueType http://www.w3.org/2001/XMLSchema#dateTime Issuer AD AUTHORITY OriginalIssuer AD AUTHORITY
ClaimType http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy Value adfs01p ValueType http://www.w3.org/2001/XMLSchema#string Issuer CLIENT CONTEXT OriginalIssuer CLIENT CONTEXT
</Claims>

We have been checking for the existence of the x-ms-proxy element, but we haven’t looked into its value. The value identifies the name of the proxy server that the request passed through. What if we could tell if a user was internal or external based upon which proxy server they came through?

With this change, internal OWA users will land on internal ADFS Proxy servers and external OWA users will land on external ADFS Proxy servers. That will allow us to add a rule like this:

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy", Value =~ "\badfsp[0-9][0-9]\b"])
&& exists([Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/groupsid", Value =~ "S-1-5-21-299502267-1364589140-1177238915-114465"])
&& exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/ls/"])
=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

Line 1: User is coming through an ADFS Proxy and that proxy has a name that matches ADFSP##

Line 2: User is in the specified group

Line 3: User is hitting the passive endpoint

Line 4: Deny the claim

If a user in the specified group presents a claim to ADFS from outside the network, all elements of this rule will be true and the claim will be denied. If the same user is inside the network and is using one of the internal proxies, line 1 will be false (the proxy name will not match ADFSP##) and the claim will be allowed.

For illustration purposes, we can express the same thing in a slightly different way:

NOT exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy", Value =~ "\badfspi[0-9][0-9]\b"])
&& exists([Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/groupsid", Value =~ "S-1-5-21-299502267-1364589140-1177238915-114465"])
&& exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/ls/"])
=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "true");

Line 1: The user is NOT coming through an ADFS Proxy that matches ADFSPI##

…

While you may not need to meet access control requirements this complex, I hope that these notes provide some enlightenment into the ADFS claim rule language.

Joe “Piscopo” Isenhour

Hey all, Ned here again. After a few months of training room huffing, airline food loathing, and PowerPoint shilling, I’m back in Charlotte. I’ve got a backlog of legacy product posts to share from colleagues, and with Windows 8 and Windows Server 2012 nigh, more new goo coming your way from me and Mike. And if I can’t come up with a halfway-decent Mail Sack after nearly two months, I’ll eat my own pants – expect that Friday. The blog post, I mean; not trouser consumption.

See you soon,

Ned “In case you’re wondering, the Dutch are like the Yankees of Korfbal, and it still rains in England” Pyle

Hey all, Ned here again with one of my rare public service announcement posts:

In August 2012, Microsoft will issue a critical security update for Windows XP, Windows Server 2003, Windows Server 2003 R2, Windows Vista, Windows Server 2008, Windows 7, and Windows Server 2008 R2. The update will block the use of RSA cryptographic keys that are less than 1024 bits.

To understand what all this means, why we are doing this, and how to determine your current certificate usage, check out:

• RSA keys under 1024 bits are blocked
• Blocking RSA Keys less than 1024 bits (part 2)
• (and a third post from Kurt after this update releases, for troubleshooting and whatnot)

Ned “Rob Greene made me post this, so I will make him answer your comments” Pyle

Hi, Gary here, and I have been seeing a few more questions regarding the recycle bin on redirected folders. With the advent of Windows Vista there was a change in redirected folders and the support for the Recycle bin. Now each redirected folder has a Recycle Bin associated with it. Windows XP only implemented it for the My Documents folder. Unfortunately, the Administrative Templates only give control of the bin as a whole and not for each redirected folder. The global settings can be controlled by the following policies that work with at least Windows XP:

User Configuration\Administrative Templates\Windows Components\Windows Explorer

Display confirmation dialog when deleting files

Do not move deleted files to the recycle bin

Maximum allowed Recycle bin size

Since there are no policies for the individual bins, we have to modify the registry settings for the user in other ways. That would be through the use of Group Policy Preferences Registry settings or script that would import or set the registry settings. No matter which way you would do it, the registry would have the following settings for each known folder recycle bin:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\BitBucket\KnownFolder\{KnownfolderGUID}

MaxCapacity (REG_DWORD) in MB with a minimum value of 1

NukeOnDelete (REG_DWORD) 0 (move to recycle bin) or 1 (delete)

Note: Known Folder GUIDs can be found at here.

Example registry path for the redirected My Documents/Documents folder would be:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\BitBucket\KnownFolder\{FDD39AD0-238F-46AF-ADB4-6C85480369C7}

What is the easy way to get the registry settings and push them out?

The easy way to get the registry settings would be by redirecting the desired folders and configuring the Recycle bin for each folder as desired through the GUI with a user that is already redirected. That is by right-clicking on the Recycle Bin icon and bringing up Properties to configure the individual recycle bin settings of a user that has the folders already redirected. The following picture is an example where just the Documents folder was redirected and configured to delete the files immediately. As other folders are redirected there will be other entries that show up there for each of the additional locations.

Once you have the registry settings, you have the following options to push out the desired settings to the client machines:

Export the KnownFolder key and then import with a logon script or other method to import the values

1. Open Regedit.exe by clicking the Start button and typing it in the Search section.
2. Navigate to the following registry location:
   HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\BitBucket\KnownFolder
3. Right-click on the KnownFolders key and choose export to save it to a file such as recyclebin.reg
4. Use the following example command line to import the recyclbin.reg file in a logon script or GPO script:

regedit /s recyclebin.reg

Create a Group Policy Registry preference setting to push out the NukeOnDelete and MaxCapacity settings for each folder to control. The advantage of this is that if you desire you can modify the settings and get pushed out and use the built in filtering to control specific settings application or value.

1. With GPMC installed on the machine edit or create a new policy in the domain
2. Navigate within the editor to the following location
   User Configuration\Preferences\Windows Settings\Registry
3. Right-click and choose New->Registry Item
4. Click on the “…” button next to the Key Path edit box
5. Navigate to the KnownFolder key path described in this blog and the GUID of the redirected folder.
6. Select the MaxCapacity value and click OK
7. On the Common tab, check the box for “Run in logged-on user’s security context (user policy option)
8. Click OK
9. Repeat steps 3-8 for the NukeOnDelete value
10. Repeat steps 3-9 for each additional GUID you want to control

What happens to files already in the recycle bin when the setting is applied to the registry?

The NukeOnDelete value just tells Explorer that no additional files will be put in the recycle bin and will be deleted from the file system. Files and folders that are already in that bin will remain until it is emptied and the MaxCapacity essentially gets ignored at that time. It might be best to configure the MaxCapacity to 1 MB and NukeOnDelete to 0 to start out with. That way the maximum size the recycle bin would be is 1 MB and the next time the user deletes a file it will be reduced to that size as well. Then the NukeOnDelete could be changed to 1 later on and the recycle bin won’t be taking up any additional space.

Can we just delete the $RECYCLE.BIN folder from the network location?

That would be one way to “empty” the specific recycle bin on the user’s behalf. However, no matter what the settings are, the next time a file is deleted it will get recreated so don’t expect it to remain deleted for very long. Also, from then on the recycle bin will honor the setting anyway.

Can the deleted files move the local recycle bin instead of staying on the network location?

No. The Recycle Bin folder view is a merged view of all the locations into one view. When a file or folder is deleted, it is just moved and renamed on the same drive. No copying is involved.

While we do not have a way to control the individual recycle bin folders through policy in box we can do it through the registry to get that out to multiple machines now for management.

Thanks,

Gary “Waste Management Consultant” Mudgett

[This is a guest post from Joe Isenhour, a Senior Program Manager in Windows Server. You may remember him from his previous ADFS claims rule post. If you are not yet up to speed on the DAC security suite in Windows Server 2012, I recommend our own Mike Stephens’ treatise Understand and Troubleshoot Dynamic Access Control in Windows Server "8" Beta - the Neditor]

Hello, my name is Joe Isenhour and I work in the Windows Server team. I’m continuing our blog series on Dynamic Access Control (DAC) with a post that addresses some of the scenarios related to deploying DAC to multiple Forests.

You can find a demo video that shows the examples from this post here:

To give you a little bit of background, DAC objects are stored in the Configuration container in Active Directory. This means that the scope for these objects is the Forest level. All of the Domains in a given Forest will use the same set of DAC objects.

 CN=Claims Configuration,CN=Services,CN=Configuration,<forest-dn>

If your organization has multiple Active Directory Forests then you may encounter some of the scenarios covered in this post:

1. Manage DAC objects between new and existing Active Directory Forests.
2. Standardize DAC objects between existing Forests.

We have built a couple of tools that will help you with the scenarios above. The tools are included in the Data Classification Toolkit. You can obtain the beta version of the tools here.

The following conditions are required in order to use the tools that are described in this post:

• You need an Active Directory Forest with the Windows Server 2012 schema
• The tools must run on a Windows Server 2012 machine with Windows PowerShell and the Active Directory Administration tools

Export Dynamic Access Control Objects from a Forest

Let’s say that your company has invested some time authoring Central Access Policies and Claims related data for your corporate Active Directory Forest. Now your CTO office is mandating that some or all of your existing DAC objects need to be included in all of your company’s other Active Directory Forests. This can be challenging for several reasons. One challenge is that the task of creating the objects in the new Forest is labor intensive and prone to error. Another challenge is keeping the unique object ID’s consistent between the Forests. Claim Types and Resource Properties have ID’s that will need to be identical across Forests in order to enable more advanced scenarios like Claim Transformations.

The Export-ClaimsConfiguration.ps1 tool will allow you to export specific DAC objects from a given Forest. The tool will allow you to export:

• Claim Types
• Resource Properties
• Central Access Rules
• Central Access Policies

By default, the tool will export any dependent DAC objects. For example, if you choose to export only Central Access Policies, the tool will also export any Central Access Rules that are defined for the policies and any Claim Types and Resource Properties that are used in the policies.

The following example demonstrates how to export all Central Access Policies from a Forest (including dependent objects). In this example the export will be stored in file called DACExport.xml.

1. Open Windows PowerShell and navigate to the Data Classification Toolkit Tools directory

 C:\Program Files (x86)\Microsoft\Data Classification Toolkit\Tools

2. Execute the following command:

 .\Export-ClaimsConfiguration.ps1 -File "C:\Temp\DACExport.xml" -IncludeCentralAccessPolicies

When the export is finished, you should have a new file called DACExport.xml. Here is an example of the information contained in the file:

As you can see, there are many other data types defined in this file besides just Central Access Polices. There are Central Access Rules, Resource Properties, and a section called domainIdentities which we will discuss in a moment. This is because the Central Access Policy that was exported has dependencies on other objects. In order for the Central Access Policy to exist in a new Forest, it will need to have all of these dependent objects.

Take a look at the section titled domainIdentities. This section is used to store a mapping between the Security Identifiers that were found in the Central Access Rules and their friendly SamAccountName. Here is an example of how this section might look:

As you can see, the export tool created an entry for every Security Identity that it discovered during the export and stored the id along with the SamAccountName.

Security Identities are unique to a Domain and therefore cannot be exported/imported into other Domains or Forests. This section is used during the import process to map accounts or groups in the target Forest with the accounts found in the export file.

For a detailed list of the options available for exporting DAC objects, execute the following command:

 .\Export-ClaimsConfiguration.ps1 –Help

Import Dynamic Access Control Objects into a Forest

Once you have created your export, you can begin importing the data into other Active Directory Forests. You have a few options when importing:

WhatIf: The import tool implements the standard Windows PowerShell WhatIf condition. If you import using this switch, none of the data will be written to the directory. Instead, you will see the output of what would have been written to the directory had you not chosen the WhatIf switch.

Overwrite: The import process will not write any data to the directory if it finds a conflict with one or more objects. A conflict occurs when a DAC object already exists in the target Forest and has one or more properties that differ from the object in the export file. You may choose the Overwrite option to overwrite the object properties in the target Forest with the properties found in the export file. Certain properties; like IsSingleValued, can’t be overwritten. In this case, the tool will alert you that the property will not be overwritten, but will continue overwriting the other properties.

The following example demonstrates how to import Central Access Policies from an export file. In this example the export will be stored in file called DACExport.xml.

1. Open Windows PowerShell and navigate to the Data Classification Toolkit Tools directory

 C:\Program Files (x86)\Microsoft\Data Classification Toolkit\Tools

2. Execute the following command to import the data in WhatIf mode:

 .\Import-ClaimsConfiguration.ps1 -File "C:\Temp\DACExport.xml" -Server [domaincontroller] –WhatIf

If the import file contains Central Access Rules then you will likely get prompted to match the identities in the source file with identities in the target Domain/Forest. The tool accomplishes this by matching the SamAccountName in the source file with accounts in the target Forest. You may choose to bypass this prompt using the DontPromptForSIDConfirmation switch. This switch will indicate that you want to accept any matches that are found by the tool.

In some cases, there may be multiple matching accounts in the target Forest. If that happens, you will be presented with a list of accounts that match the source account and you will need to select the proper account.

If there are no conflicts in the source file and the target Forest, then the script should complete with output similar to the image below.

If a conflict was found, then you will instead see a report indicating that there is a conflict. Remember, you can choose to overwrite conflicts using the –Overwrite switch.

If the output from the command looks correct then you can choose to run the command again without the WhatIf switch and write the DAC objects to the target Forest.

For a detailed list of the options available for exporting DAC objects, execute the following command:

 .\Export-ClaimsConfiguration.ps1 –Help

I hope that you’ve found these examples helpful. Watch for more great posts and examples of Dynamic Access Control in Windows Server 2012.

Joe Isenhour

Hi all, Ned here again. After a few months of talking about Windows Server 2012 to other ‘softies from around the globe, I’m back with the sack. It was great fun – and not over yet, it turns out – but I am finally home for a bit. The only way you don’t know that the next version of Windows is nearly done is if you live in a hobbit hole, so I’ll leave all those breathless announcements to the rest of the internet.

This week we talk:

• DFSR and reeeeaaaaallllyy big pre-existing recovery
• DfsUtil reparse point hunting
• USMT 4.0 group membership changes
• Server Manager remoting menu
• USMT 4.0 and offline files
• GMSA backgrounder
• DFSR disabledirectoryverification
• Other Stuff

Let’s get to it.

Question

I accidentally chose the wrong source replicated folder when setting up DFSR and now I have a few terabytes of data in the preexisting folder. I found your RestoreDfsr script to pull out the intentionally-mangled data, but it’s taking a long time to put everything back. Is there an alternative?

Answer

The script just wraps xcopy and makes copies rather than moving files, so it is not scalable when you get into the multi-TB realm (heck, you might even run out of disk space). If it’s reaaaallly slow (when compared to another server just copying some similar files) I’d worry that your disk drivers or firmware or SAN or NAS anti-virus or third party backup agents or whatever are contributing to the performance issues.

However!

All of the files and folders in pre-existing deeper than the root are not mangled and don’t require any special scripts to copy out. Only the files and folders at the root of the preexisting RF are mangled and require the preexistingmanifest.xml for the heavy lifting. Therefore, a quicker way to fix this would be to just figure out the original folder names at the root by examining the pre-existing manifest file with your eyeballs. Rename them to their original name and then use Windows Explorer MOVE (not copy) to just move them back into the original folder. That would leave only the mangled files in the root of the pre-existing folder, which you could then use the script to restore – presumably with less data to restore and where the slower xcopy performance no longer matters.

Question

When I run dfsutil diag viewdfsdirs c: verbose on this Win2008 R2 server, I see errors like this:

Unable to open file by ID
Unable to open file by ID
This volume (C:\) contains 5 DFS Directories.
Done processing this command.
C:\DFSRoots\root1\link1
C:\DFSRoots\root2\link2
C:\DFSRoots\root3\link3

What is the ID in the error? How can I tell the other two folders that it’s missing?

Answer

Dfsutil.exe uses the reparse point index to find DFS links on a volume.

Due to some error, dfsutil.exe failed to open some of them. We definitely need a better error that tells you return code and failed path. Sorry.

First, look in c:\dfsroots. The two link folders not returned by your list below are probably in there. If they are not in c:\dfsroots at all, use:

DIR c:\ /S /AL

That returns all reparse points on the volume. Any besides the default ones (in user profiles, programdata, and sysvol) are probably your bad guys. You’d want to make sure they still show up correctly in fsutil, that you have no errors with chkdsk, that they have not been intercepted by some wacky third party, etc.

Question

I am using USMT 4.0 to migrate users that are members of the Administrators group and using a config.xml to make those users only be members of the Users group on the destination computer.  I am running these USMT scripts as the users themselves, so they are already administrators on both the source and destination computer when scanstate and loadstate run.

I am finding that the users are still members of administrators after loadstate. Am I doing something wrong or does this XML not work?

<Configuration>

<ProfileControl>

    <localGroups>

    <mappings>

         <changeGroup from="administrators" to="Users" appliesTo="MigratedUsers">

            <include>

               <pattern>*</pattern>

            </include>

         </changeGroup>

    </mappings>

    </localGroups>

</ProfileControl>

</Configuration>

Answer

Long answer, deep breath:

1. USMT 4.0 requires that the user running loadstate.exe is a member of the built-in Administrators group and holds privileges SeBackupPrivilege, SeRestorePrivilege, SeTakeOwnershipPrivilege, SeDebugPrivilege, SeSecurityPrivilege.

2. It is not a best practice that you log on as the end user being migrated or that end users run their own migrations:

• From a security perspective, it’s bad if USMT migration users have to know the end user’s domain password.
• From a USMT perspective, it’s bad because the end user’s more complex profile and settings are more likely to be in-use and fight the migration, unlike a simple migration user that exists only to run USMT.
• If the end user is running it himself, it’s bad because they have no way to understand if USMT is working correctly.
• Therefore, you should always use separate migration user accounts.

It’s easy to misinterpret the results of using this XML, though. It is not retroactive - if the group memberships already exist on the destination before running loadstate, USMT does not alter them. USMT is designed to copy/skip/manipulate source groups but not destroy destination existing groups.

Since your design requires destination administrator group memberships before running loadstate, this XML cannot work as you desire. If you switch to using separate migration accounts, MDT, or SCCM, then it will work correctly.

Question

I am using new Server Manager in Windows Server 2012 Release Candidate to manage the remote machines in my environment. If I right-click a remote server I see that list of management tools for the given roles I installed. When I run some of the GUI tools like LDP or Computer Management, they target the remote servers automatically. However, the command-line tools just show the help. Is this intentional or should it run in the context of the remote server?

Answer

All of the command-line tools are run in this fashion, even when they support remote servers (like repadmin or dcdiag) and even when targeting the local server. We can’t get into a design that deals out a million command-line arguments – imagine trying to provide the menus to support all the various remote scenarios with NETDOM, for example. :-D

Oy vey

Since providing a remote server alone isn’t enough to make most tools work – Dcdiag alone has several dozen other arguments – we just went with “let’s get the admin a  prompt and some help, and let them have at it; they’re IT pros and smart”.

If you haven’t used Server Manager yet, get to it. It’s a great tool that I find myself missing in my Win200L environments.

The “L” is for legacy. YeeaaaahhinyourfaceolderproductsthatIstillsupport!!!

Question

Does USMT 4.0 migrate the Offline Files cache from Windows XP to Windows 7? My testing indicates no, but I find articles implying it should work.

Answer

Unfortunately not. Through an oversight, the migration manifest and down-level plugin DLL were never included. The rules of USMT 4.0 are:

• USMT 4.0 does not migrate CSC settings and the "dirty" (unsynchronized to server) file cache from Windows XP source computers
• USMT 4.0 does migrate CSC settings and the "dirty" (unsynchronized to server) file cache from Windows Vista and Windows 7 source computers

In order to migrate the CSC dirty cache, USMT needs plugin DLLs provided by Offline Files. The Offline Files changes from XP to Windows 7 were huge, but even Win7 to Win7 and Vista to Win7 need the plugins for the path conversions.

To workaround this issue, just ensure that users manually synchronize so that all offline files are up to date on the file server. Then migrate.

If you are migrating from Vista (why?! it’s great!!!) or Windows 7 and you want to get the entire cache of dirty and synchronized files, you can use the DWORD value to force the cscmig.dll plugin to grab everything:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\CSC\Parameters
MigrationParameters = 1 

This is rather silly in a LAN environment, as it will take a long time and increase the size of your compressed store for little reason; these files are going to sync back to the user anyway after the migration. Maybe useful for remote users though; your call.

Question

I'm using Windows Server 2012 Release Candidate and I'm trying to create a new Managed Service Account. I'm running the following from an elevated PS Active Directory module:

New-ADServiceAccount -name Goo

The command fails with error:

NewADServiceAccount : Key does not exist

Answer

There are new steps required for managed service accounts in Win2012. We created an object class called a group Managed Service Account (gMSA). GMSA supersedes the previous standalone Managed Service Account (sMSA) functionality introduced in Windows Server 2008 R2. It adds support for multiple computers to share the same service account with the same password automatically generated by a domain controller. This makes server farms using MSAs possible for the first time. SQL 2012, Win2012 IIS app pools, scheduled tasks, custom services, etc. all understand it. It’s very slick.

The new Microsoft Key Distribution Service on Win2012 DCs provides the mechanism to obtain the latest specific key for an Active Directory account. The KDS also continuously creates epoch keys for use with any accounts created in that epoch period. For a gMSA, the domain controller computes the password on the keys provided by the KDS in addition to other attributes of the gMSA. Administrator-authorized Windows Server 2012 and Windows 8 member hosts obtain the password values by contacting a Win2012 DC through Netlogon and cache that password for use by applications. The administrator never knows the password and it’s secure as it can be – just like a computer account password, it’s the maximum 240-bytes of randomly generated goo, changed every 30 days by default. The new KDS is used for other features besides gMSA as well.

And this finally brings me to your issue – you have to first create the root key, using the Add-KdsRootKey cmdlet. This root key is then used as part of all the subsequent gMSA work.

If you want to see some preliminary step-by-step documentation, check out Getting Started with Group Managed Service Accounts. I’ll be coming back to this new feature in detail after we RTM Windows Server 2012.

Question

What does the what /disabledirectoryverification option do in DFSRadmin.exe membership new?

Answer

If the folder you specified for the RF with /localpath does not exist with membership new, dfsradmin.exe will create it for you by default with the correct permissions. If it does exist already, it will modify the existing permissions to let the DFSR service use that folder. If the folder already exists with the correct permissions, it does nothing. Using this argument prevents all of these convenient actions.

You would use this argument mainly if you were a crazy person.

Other Stuff

I didn’t get to go to Comic-Con this year, but thanks to the InterWaste, you can at least see some of the footage sans the special trailers. The best ones I’ve found are… well, pretty obvious:

• The Walking Dead panel (part 1,part 2, part 3) + Kirkman’s panel. Because, you know, there are supposed to be comics at comic-con
• Firefly 10th Anniversary panel
• Community panel
• Expendables 2 panel (with Arnold and Terry Crews! Pah-pah-pah-pah-pah-pah-power!)
• Robert Downey Jr. making a bunch of kids’ day 
• But the best of all time was still from last year… Game of Thrones Season 1 panel. Jason Momoa steals the show.

There are often some heavy-duty spoilers in these panels – no whining if you find out that Superman is from Krypton and Batman’s parents were killed by the Joker. They also get a little sweary sometimes.

Naturally, if comic-con is your thing, you need your fill of cosplay. The two best galleries are Comicvine.com and – rather oddly – tested.com.

This is how David Fisher sees himself when he looks in the mirror.

Via tumblr

It’s summer, which means good reading. IO9 has their annual list of the best Sci-Fi and Fantasy to check out. Redshirts was pretty good and I am starting Ready Player One shortly as no one can shut up about it (Ernest Cline will give you a Delorean if you are an ultrageek). Charles Stross is an automatic if you’re in IT or a British person; in that vein I recently enjoyed The Rook and am revisiting William Gibson this month.

And finally:

Hey, look it’s Ned in a restaurant, grab a picture!

And look, he’s with Satan!

Until next time,

- Ned “I always feel like someone’s watching me” Pyle

Hey all, Ned here with a quickie: Robert Paige just published an interesting read on Windows Server 2012 Dynamic Access Control over at the Windows Server blog:

http://blogs.technet.com/b/wincat/archive/2012/07/20/diving-deeper-into-windows-server-2012-dynamic-access-control.aspx

It highlights the work done with Independent Software Vendor partners and the products they are creating to integrate with the DAC suite. Definitely worth a read.

As always, I recommend you check out Mike Stephens’ depth whitepaper Understand and Troubleshoot Dynamic Access Control in Windows Server "8" Beta if you are looking to get neck deep.

- Ned “the regurgitator” Pyle

Hello all, Dave here again. Over a year ago, Warren created an ASKDS blog post covering common DFS Replication (DFSR) mistakes and oversights. Someone asked me recently where the common DFS Namespaces (DFSN) mistakes article was located. Well, here it is now...

Below you will find details about the more common configuration mistakes or oversights with the DFSN service. I hope this post helps you avoid them. Where possible, I provided links for additional details.

Improperly decommissioning a namespace server or reinstalling the operating system of a namespace server

If someone removes a namespace server from the network, turns it off permanently, or reinstalls the operating system, namespace access by clients and administrators alike may experience delays. If the server is the only namespace server hosting the namespace, namespace access will fail. In addition, this blocks you from creating a new namespace with that original namespace's name, as its configuration information still resides within the Active Directory. It is also possible to leave orphaned configuration information in the registry of a namespace server that may prevent it from hosting a future namespace.

None of these are irreversible situations, but it takes some manual work to remove references to an improperly decommissioned namespace server as detailed here (and check out all those errors you could cause!):

977511        About the DFS Namespaces service and its configuration data on a computer that is running Windows Server 2003 or Windows Server 2008
http://support.microsoft.com/kb/977511/EN-US

Before decommissioning a namespace server, remove the server from all namespaces using dfsmgmt or dfsutil. If a namespace server goes permanently offline, clean up the namespace configuration in Active Directory.

Disabling the DFS Namespace service of domain controllers

It may seem tempting, but domain controllers should never have the DFS Namespace service disabled. Clients require access to the DFS service of domain controllers to enumerate trusted domains, obtain lists of available domain controllers, and to process domain namespace referral requests. The DFSN service typically requires very little CPU and memory to service requests, even from numerous clients. Also, realize that Group Policy processing requires issuing DFSN referrals to clients that attempt to access the SysVol and Netlogon shares. Sysvol and Netlogon are specialized namespaces that Active Directory automatically manages.

Always keep the DFS Namespaces service running on domain controllers.

Managing a namespace using the original Windows Server 2003 (non-r2) management MMC snap-in

In Windows Server 2003, there exists an original version of the tool to manage DFS Namespaces called dfsgui.msc

("Distributed File System"). This version lacks many features provided by the 2003 R2 (and later) versions of the DFSN management tool dfsmgmt.msc ("DFS Management"). Although you may use dfsgui.msc to manage a namespace running on these later operating systems, the Windows product team never tested interoperability. The newer MMC also is much more efficient and has some usability improvements. Lastly, the older tool does not allow the configuration of delegation, referral ordering, client fail back, polling configuration, Access-based enumeration (2008 and later), DFSR, etc.

Always use the newer "DFS Management" console in lieu of the "Distributed File System" console.

Configuring the "Exclude targets outside of the client's site" setting on a namespace though some Active Directory sites have no targets

The major benefit of DFSN is fault tolerance. However, there are times when namespace administrators would prefer a client's file access to fail if the local site's namespace or file server (DFS folder target) is unavailable. A common mistake is to enable this feature and not realize a number of conditions may prevent clients from reaching any targets.

For instance, to issue proper referrals, the DFSN service must determine the site of all namespace and folder targets. Then, it must identify the site of the client. With the exclusion setting configured, if the client's site cannot be determined, if the client is erroneously associated with a site without any targets, or if the site association of namespace servers and/or folder targets cannot be properly determined, access to the namespace's resources may fail. Below, I attempt to identify the site of the system named “workstation” using DFSUtil and the operation failed:

At the bottom of KB article 975440 are methods for troubleshooting this scenario. You may find the other details in the article useful as well.

Before configuring the "exclude targets outside of the client's site" setting, ensure each site containing clients that require access to a namespace path has a target within their site and verify your Active Directory site configuration is accurate.

Configuring multiple DFSN folder targets on a replicated folder hosting user profiles data

Ned's details found here have become the standard for why Microsoft does not recommend or support using Distributed File System Replication (DFSR) and DFSN (multiple folder targets) for user profile data. For example: if a user were to be referred to a file server that hasn't yet replicated in recently changed data (or perhaps you don't have replication enabled on the shared folder), then the user will not have access to their expected data. The user will logon and receive an old version of their user profile. At best, this generates helpdesk calls--at worst, a user may unknowingly utilize outdated files and lose data.

Do not use DFSN paths with multiple folder targets to store user profiles or volatile user data. Whenever implementing DFSN, you must be vigilant to ensure users and clients are never affected by referrals to alternate file servers where expected data may not reside.

Active Directory site configuration doesn't accurately reflect the physical network topology

As mentioned previously, site associations evaluated by the DFS namespace service are crucial for clients’ direction to optimal folder targets. It is important to validate that each site required to have a target has one (or more) targets, that site configurations defined in the Active Directory are accurate, and that you monitor the environment for changes that can affect site associations.

DFSDiag.exe is one method to validate site associations of all servers involved in the referral process and you should run it from a client if you suspect it isn't working properly. I recommend using the "/testreferral" argument as it diagnoses just about everything related to the namespace's health along with the site associations of clients and targets.

Routinely use DFSDiag.exe to check for failures before they impact users or prior to going "live" with a new namespace.

Incorrectly disabling IPv6 on servers involved in the DFSN referral process

As detailed in this blog post, IPv6 can cause the DFSN referral process to fail or be inaccurate (even on networks that do not support IPv6 functionality). If you are using IPv4 addresses that are not considered a member of the private address space (look here for details), then it is likely your servers are registering IPv6 address in DNS. If you haven't configured IPv6 subnet definitions in Active Directory, these DNS records may be used by DFSN when processing referrals. They cause site associations to fail and incorrect referral ordering.

Once easy method to see if this scenario affects you is to simply open up the DNS Manager MMC snap-in and scan all the host (A) records for your domain's forward lookup zone. If you see any that report an IPv6 address and they are namespace servers or folder targets, you must take corrective action. The aforementioned blog post details some options.

Validate your site configuration for IPv6 compatibility.

Not utilizing multiple namespace servers for domain-based namespaces or a clustered namespace server for standalone namespaces

Avoid single-point-of-failure wherever possible. If hosting important data within a DFS Namespace that must be available at all times, there is no reason to have only a single namespace server hosting it. As mentioned previously, DFSN requires very little CPU and memory resources under typical conditions. If the single namespace server goes down, no one can access the data.

If server resources are scarce and prevent the configuration of additional namespace servers to host a namespace, consider adding domain controllers as namespace servers. On Windows Server 2008 and 2008 R2 domain controllers, install the "DFS Namespaces" role service to get the DFSN management console and command-line tools. Otherwise, the DFSN service is already installed on domain controllers via the DCPromo operation, but none of the tools are available on it.

Always utilize multiple namespace servers. If a standalone namespace is necessary, use a clustered namespace.

Using the less capable namespace type

Windows Server 2008 introduced a new type for DFS Namespaces, aptly named "Windows Server 2008 mode". This offers increased scalability and functionality over a "Windows 2000 Server Mode" namespace. As long as your environment supports the minimum requirements for 2008 mode namespaces, there is no reason not to use them.

To use the Windows Server 2008 mode, the domain and namespace must meet the following minimum requirements:

• The forest uses the Windows Server 2003 or higher forest functional level
• The domain uses the Windows Server 2008 domain functional level
• All namespace servers are running Windows Server 2008 (or later)

Not choosing this mode is a common mistake by administrators. However, it is possible to migrate from a Windows 2000 Server Mode namespace to a Windows Server 2008 mode namespace using the information found here. However, you should perform the migration during a period where your DFS clients can tolerate an outage--migration requires recreating the namespace.

Standalone namespaces used to be the only option if your namespace required more than 5,000 DFS folders (links). If you fall into this scenario, consider replacing it with a Windows Server 2008 mode namespace.

Choose the correct namespace mode when creating a namespace.

Using mount points

Ned covers this very succinctly in a mail sack response found here. There is no reason to utilize mount points within a DFS Namespace's configuration for the namespace's shared folder. Note: we fully support utilizing mount points under a file share that is the target of a DFS folder.

Do not use mount points within the namespace.

Enabling DFSR or FRS on the namespace share

Generally, we do not recommend enabling DFSR or File Replication Service (FRS) replication on a DFS namespace's share. Doing so causes a number of issues, as the replication service attempts to replicate modifications to the reparse folders maintained by the DFSN service within the share. As DFSN removes or adds these reparse folders, FRS/DFSR may replicate the changes before or after DFSN has an opportunity to update the reparse folders based on recent configuration changes.

Ken has an excellent blog here that goes into this further. He also covers additional problems if you try to mix DFSN with DFSR read-only replicas.

Replicate data within separate shared folders and refer clients to those shares using DFS folders within a namespace.

Placing files directly in the namespace share

Avoid placing files or folders within the namespace share. Place any data requiring access via the namespace within a share targeted by a DFS folder. This prevents any potential conflicts with reparse folders. It also ensures that in the event a user connects to an alternate namespace server, they always have a consistent view of the data.

Do not place files or folders within the namespace share.

Not implementing DfsDnsConfig on namespace servers

“DfsDnsConfig?” you may ask. This setting alters the types of names, NetBIOS or DNS FQDN, used for namespace referrals. As network environments grow larger and more complex, it makes sense to move away from the flat NetBIOS WINS namespace to a DNS namespace. Only utilize NetBIOS names if you have network clients that specifically require it and must access DFS namespaces. By using FQDN names, you reduce complexity as clients no longer are dependent on appending correct DNS suffixes to the targets’ names. .

Configure namespace servers to utilize the FQDN names for namespace referrals.

Using Access Based Enumeration on “Windows 2000 Server Mode” namespaces hosted by 2008 and/or 2008 R2 namespace servers

If you have enabled Access Based Enumeration (ABE) on a “Window server 2000 Mode” namespace using the instructions found here, you may find that the Windows Server 2008 or 2008 R2 namespace servers are not correctly maintaining permissions their reparse folders. Your custom permissions may be lost if you restart the server or its DFSN service. If any Windows Server 2003 namespace servers also host that namespace, they do not experience this symptom…

An update is available for both Windows Server 2008 and 2008 R2 to correct this behavior. It prevents the DFS Namespaces service from replacing your custom reparse folder permissions you implement via the instructions found in KB article 907458. Again, those instructions are only appropriate if you have a specific requirement to utilize “Windows 2000 Server Mode” namespaces with 2008 and/or 2008r2 namespace servers.

If supported by your environment (see section “Using the wrong namespace version” above), it is better to leverage the native support for ABE in a “Windows Server 2008 mode” namespace. You no longer require use of icacls.exe (or similar NTFS security editing tool) on each namespace server. The DFSN service automatically enforces the reparse folder permissions defined in the DFS Namespace configuration. Because of the mode requirement, Windows Server 2003 servers cannot be namespace servers in the namespace.

This is also a good opportunity to mention the "latest hotfixes" articles for DFS Namespace servers:

968429        List of currently available hotfixes for Distributed File System (DFS) technologies in Windows Server 2008 and in Windows Server 2008 R2

958802        List of currently available hotfixes for Distributed File System (DFS) technologies in Windows Server 2003 and in Windows Server 2003 R2

Ensure you update the DFSN services, especially if you wish to use ABE with 2008 and/or 2008 R2 namespace servers.

Configuring domain-based namespaces in an unhealthy Active Directory environment

All DFS Namespace deployments require a healthy Active Directory environment to operate successfully. That is why the "How DFS Works" TechNet article begins by describing what constitutes an "optimal environment" for DFS. If DNS, Active Directory replication, site configuration, PDC FSMO owner domain controller, DFS service state, client health, network communications, and security settings are not operating properly, DFSN operations will suffer. Too often, administrators deploy DFSN in unhealthy environments and clients are unable to access resources properly.

Below are some strategies for verifying many components of your environment:

Use DFSDiag.exe to test all major dependencies of the DFSN service.

If you don't yet have any namespaces in the environment, you may still run the following command to test the domain controllers for DFSN service state and consistency of their site associations:

Dfsdiag /testdcs > dfsdiag_testdcs.txt

If a domain-based namespace already exists, the following commands may be utilized to exercise all of Dfsdiag's tests and output results to the specified file:

dfsdiag.exe /testreferral /dfspath:\\contoso.com\namespace1 /full > dfsdiag_testreferral.txt

dfsdiag.exe /testdfsintegrity /dfsroot:\\contoso.com\namespace1 /recurse /full > dfsdiag_testdfsintegrity.txt

Note: Many of DFSDiag's diagnostics are relative to the system running the utility (e.g. site associations, network connectivity, etc.). Thus, you may find running it from various systems throughout your environment gives you more accurate health assessment . In addition, you may acquire dfsdiag.exe from the Remote Server Administration Tools included with Server editions of 2008 or 2008 R2, or from the download for Vista or Windows 7. Install the "Distributed File System Tools" role administration tool.

Evaluate domain controller health, site configurations, FSMO ownerships, and connectivity:

Use Dcdiag.exe to check if domain controllers are functional. Review this for comprehensive details about dcdiag.

Suggested commands:

Dcdiag /v /f:Dcdiag_verbose_output.txt

Dcdiag /v /test:dns /f:DCDiag_DNS_output.txt

Dcdiag /v /test:topology /f:DCDiag_Topology_output.txt

Active Directory replication

If DCDiag finds any replication failures and you need additional details about them, Ned wrote an excellent article a while back that covers how to use the Repadmin.exe utility to validate the replication health of domain controllers.

Suggested commands:

Repadmin /replsummary * > repadmin_replsummary.txt

Repadmin /showrepl * > repadmin_showrepl.txt

Always validate the health of the environment prior to utilizing a namespace.

My hope is that you find this article helpful (I guess sometimes the glass half-empty folks need an article too!)

-Dave “Doppelwarren” Fisher

Hi guys, Joji Oshima here again. When troubleshooting Kerberos authentication issues, a network capture is one of the best pieces of data to collect. When you review the capture, you may see various Kerberos errors but you may not know what they mean or if they are real problems. In this post, I’m going to go over many of the common Kerberos errors seen in these traces, explain what they mean, and what to do about it when you see it.

I designed this post for IT professionals who have experience reviewing network captures. If you are unfamiliar with Kerberos Authentication, I recommend reading Kerberos for the Busy Admin by Rob Greene.

What is the best way to get the network capture?

If you are looking for Kerberos related problems, it is important to see the ticketing process over the wire. Follow the steps below to see the requests and possible returned failures. I typically prefer Network Monitor to Wireshark for captures as it gathers the process name, but you can use either one.

1. To reduce the possibility of caching data, do one of the following:

• Close/Reopen client application
• Logoff/Logon client workstation
• Reboot client workstation

2. Start the network capture

3. Clear DNS cache using:

ipconfig /flushdns

4. Clear NetBIOS cache using:

nbtstat –RR

5. Clear user Kerberos tickets using:

klist purge

6. Clear system / computer Kerberos tickets using (Vista or higher only):

Klist –li 0x3e7 purge

7. Reproduce the authentication failure with the application in question

8. Stop the network capture

Now that you have the capture, you can filter the traffic using the string ‘Kerberosv5’ if you are using Network Monitor. If you are using Wireshark, you can filter using the string ‘Kerberos’.

To be more thorough, load the Authentication Traffic filter that shows packets containing Kerberos tickets as well. You can do this by clicking the Load Filter button, choose Standard Filters, and then click Authentication Traffic.

You need to click the Apply button before the filter is actually loaded.

If there is a lot of traffic, remove the lines for NLMP to reduce some of the noise. Remember to click the Apply button again to make the changes effective.

Important: Depending on the application, the topology, and the domain structure, it may be beneficial to take simultaneous network captures from various points including the client, middle-tier server(s), and back-end server(s).

KDC_ERR_S_PRINCIPAL_UNKNOWN

When a domain controller returns KDC_ERR_S_PRINCIPAL_UNKNOWN, it means the client sent a ticket request for a specific Service Principal Name (SPN) and was unable to locate a single Active Directory object via an LDAP query with that service principal name defined on it.

There are two major causes of this error. The first is the SPN is not registered to any principal. In that case, you should identify which principal will be decrypting the ticket, and register the SPN to that account. The other major cause for this is the SPN was registered to more than one principal in the same Active Directory domain. In this scenario, the domain controller does not know which principal to use, so it returns the same error. Determine which principal is appropriate, and remove the SPN from the other(s). You can read more about this error here.

Extra details to keep in mind:

The HOST SPN (host/server.domain.com) works for multiple services like HTTP & RPCSS. If you would like to see the default Host to SPN mappings use LDP or ADSI Edit and navigate to: cn=Directory Services,CN=Windows NT,CN=Services,CN=Configuration,DC=[Your Domain Component]. Then look at the sPNMappings attribute.

KDC_ERR_C_PRINCIPAL_UNKNOWN

Similar to KDC_ERR_S_PRINCIPAL_UNKNOWN, KDC_ERR_C_PRINCIPAL_UNKNOWN means the domain controller does not know which client principal it should use to encrypt the ticket. The difference here is that instead of a missing or duplicate SPN, there is a missing or duplicate User Principal Name (UPN).

To resolve this, determine if the requestor has the correct UPN. If so, then determine if there is a principal with a matching UPN. If there is a match, look for a duplicate UPN.
You may be scratching your head on the duplicate UPN part because if you try to add/modify a principal that has a duplicate UPN in Active Directory Users & Computers (ADUC), it will block you from doing this. Active Directory does not actually enforce the uniqueness of User Principal Names, but it leaves that up to the application. ADUC checks for duplicates, but other utilities like adsiedit.msc and ktpass.exe do not.

KDC_ERR_ETYPE_NOTSUPP

Here, the client has requested a ticket from the domain controller with a specific algorithm of which the domain controller does not have a hash. In the request, the client will list all the algorithms it supports. The domain controller will pick the highest one that it supports and returns the ticket encrypted with that algorithm. If there are no matches, the domain controller returns KDC_ERR_ETYPE_NOTSUPP.

One common cause of this is older devices that are requesting DES encrypted tickets. By default, DES encryption is disabled in Windows 7 and Windows Server 2008 R2. Ideally, you should update those devices or Kerberos clients to support the newer encryption algorithms. If they cannot be upgraded or replaced, then you can enable DES through group policy. For a good way to find these devices, I recommend reading Hunting down DES in order to securely deploy Kerberos by Ned Pyle.

Another common cause of this is when a device requests an AES encrypted tickets before you raise the functional level of the domain to 2008 or higher. This does not typically occur on Windows clients as they request the legacy algorithms in addition to AES. This scenario is more likely to occur on Unix/Linux systems where an administrator specifies a single algorithm in the krb5.conf file. In this case, raise the functional level of the domain or configure the client to utilize another algorithm, like RC4-HMAC.

If the client is requesting an algorithm that the domain controller should support, but is still returning the error, try resetting the password on the account and wait for replication to converge. Resetting the password regenerates the hashes stored in the directory.

Note: Domain controllers do not store the password of the user. Instead, they store various hashes of the password using various algorithms.

KDC_ERR_PREAUTH_REQUIRED

If you see this error in the trace, it does not indicate there is a problem at all. The client requested a ticket but did not include the pre-authentication data with it. You will typically see the same request sent again with the data and the domain controller issuing the ticket. Windows uses this technique to determine the supported encryption types.

KDC_ERR_PREAUTH_FAILED

KDC_ERR_PREAUTH_FAILED indicates the pre-authentication data sent with the ticket is not valid. It usually means the user does not exist or the password supplied is invalid.

KRB_AP_ERR_SKEW

To avoid packet replay attacks, Kerberos tickets include an authenticator with the ticket. This authenticator is based on a timestamp so an attacker cannot reuse them. Getting KRB_AP_ERR_SKEW typically means there is a time synchronization issue in your domain, and the time difference is greater than the default 5 minutes. If this is a common problem, start looking for time drifts across the infrastructure.

KRB_AP_ERR_REPEAT

This is another mechanism created to reject replay attacks. The server caches information from recently received tickets. If the server name, client name, time, and microsecond fields from the Authenticator match recently seen entries in the cache, it will return KRB_AP_ERR_REPEAT. You can read more about this in RFC-1510. One potential cause for this is a misconfigured network device in between the client and server that could send the same packet(s) repeatedly.

KRB_AP_ERR_MODIFIED

If a service returns KRB_AP_ERR_MODIFIED, it indicates that the service was unable to decrypt the ticket that it was given. The name of the error suggests that an attacker may have modified the ticket in order to gain access to a system. While this is possible, the most common reason is when the Service Principal Name (SPN) is registered to the wrong account. If Service A gets a ticket encrypted with Service B’s password, Service A cannot decrypt it using its password. There is no way for the service to know why it cannot decrypt the ticket, so it returns this error. To resolve this issue, determine which account is actually running the service and move the SPN to that account. If the service is running as Local System, Local Service, or Network Service, set the SPN on the computer account.

Another possible cause is a duplicate SPN in two different domains in the forest. If it appears the SPN is registered to the correct account, search the entire forest for a duplicate SPN. For example: Say there is a service in Domain A that uses the SPN http/service.contoso.com and the same SPN exists in Domain B. If a user from Domain B tries to access the service in Domain A, it will fail with this error. The reason for this is the client in Domain B will first try to contact a domain controller in Domain B for that SPN. That domain controller will return a ticket for the account in Domain B.

An interesting issue we see revolves around IIS7 and Kernel Mode Authentication. Typically, you should register the SPN to the account that is running the application pool. Kernel Mode Authentication speeds up authentication requests and performs the decryption in the context of the computer account. In the case of load balanced web servers, you cannot have multiple nodes using the computer different contexts to decrypt the ticket. Either disable Kernel Mode Authentication or use the useAppPoolCredentials in the applicationhost.config file of the web server. For more information, review:

• configure IIS to use the application pool’s identity
• Troubleshoot Kerberos In WCF

KDC_ERR_BADOPTION

If the domain controller returns KDC_ERR_BADOPTION, it means that one of the KrbFlags set in the KdcOptions is not allowed. You can see a sample of the options in the figure below.

The most common scenario is a request for a delegated ticket (unconstrained or constrained delegation). You will typically see this on the middle-tier server trying to access a back-end server. There are several reasons for rejection:

1. The service account is not trusted for delegation

2. The service account is not trusted for delegation to the SPN requested

3. The user’s account is marked as sensitive

4. The request was for a constrained delegation ticket to itself (constrained delegation is designed to allow a middle tier service to request a ticket to a back end service on behalf on another user, not on behalf of itself).

KDC_ERR_WRONG_REALM

This error may occur when a client requests a TGT from a domain controller for a domain to which the client does not belong. This error refers the client to the correct domain and does not indicate a problem. You can read more about it here.

KDC_ERR_TGT_REVOKED

Getting a KDC_ERR_TGT_REVOKED error means that the TGT presented to the domain controller in order to get a service ticket is not valid. These errors are common when the client is in a site with a Read Only Domain Controller (RODC) and is attempting to access a resource in another site. Seeing this error does not necessarily mean there is a problem. Read more about the ticketing process with RODCs here.

Conclusion

Troubleshooting authentication issues is not always cut and dry. I hope you now understand the meanings behind common Kerberos errors what you can do about them. If you want to dive a bit deeper into this, I recommend reading the RFCs associated with Kerberos.

Kerberos RFC
http://www.ietf.org/rfc/rfc1510.txt
http://www.ietf.org/rfc/rfc4120.txt

I’ve also linked to a more comprehensive list of Kerberos errors you may encounter.

Kerberos and LDAP Error Messages
http://technet.microsoft.com/en-us/library/bb463166.aspx

Until next time,

Joji “three-headed puppy” Oshima