Client: "We need an email sent to so-and-so back in June 2012. We can't find it. Can you?"
On the client's Exchange 2010 SP1 server, I assigned myself the "Discovery Management" role under https://server/ecp/?p=AdminRoleGroups. When I created a search at https://server/ecp/?exsvurl=1&p=MailboxSearches, it threw an error shown in the screenshot.
This was solved by following these instructions on a TechNet blog to fix an issue peculiar to Exchange 2010 SP1, and then rebuild the index.
Also learned about some built-in troubleshooting scripts. This Technet article talks about the full-text index catalog. This Technet article is all about "Understanding Multi-Mailbox Search". This article says that the built-in Exchange search is great for finding content, but not so good for legal searches because the data is easily contaminated.
10/29/13
Hands-on: IPAM
The first thing to note about IPAM is that you can't put it on a domain controller. It can oversee multiple domains within a single forest. IPAM is a feature, not a role. After installing it on IPAM1.seattle.contoso.local, I installed the client on a domain controller and tried to manage it, but got the delightful error shown in this screenshot. This was because I was signed in with an enterprise admin account from a parent domain - whoami /groups to the rescue. Signing in with a domain admin account eliminated the error. Otherwise, I think that adding my account to the WinRMRemoteWMIUsers group on the IPAM server would've worked too.
Moving back to the IPAM server itself, I set the provisioning method to GPO (instead of manual) and noted that you have to run Invoke-IPAMGPOProvisioning in each domain that you want to manage.
After provisioning, you tell it what domains you want to manage (you still have to use Powershell to create the GPOs in each domain). I don't know what effect this step actually has. Next, you get to "Step 4" which is to manually kick off the ServerDiscovery task in Task Scheduler. Interestingly, you can't view the details of what this task actually does.
Next up, ran the Invoke-IPAMGPOProvisioning commandlet. This creates three GPOs in the specified domain, letting the IPAM server read event logs, get through the Windows firewall, and create scheduled tasks on the servers that you choose to manage through the IPAM console.
When you go back to the IPAM console and tell it that a discovered server should be managed, that server is added to the newly created GPOs that are attached at the domain level.
Next, you right-click a discovered server and select "Retrieve all server data". This immediately fires up several tasks in the Task Scheduler:
- Audit: collects events from domain controllers, NPS, and DHCP servers.
- Address Expiry: tracks IP address expiry state and logs notifications.
- ServiceMonitoring: collects DNS zone status events from DNS servers.
- AddressUtilization: collects IP address space usage data from DHCP servers.
- ServerConfiguration: collects configuration info from DHCP and DNS servers.
I found the explanation of these groups in a Microsoft Word document, but it's also listed in the "Description" field of the task.
There's also the ServerAvailability task which "collects status information from DHCP and DNS servers" and the ServerDiscovery task which "automatically discovers domain controllers, DHCP servers and DNS servers in the domains you select".
This is all that I'm going to do with IPAM for now.
Moving back to the IPAM server itself, I set the provisioning method to GPO (instead of manual) and noted that you have to run Invoke-IPAMGPOProvisioning in each domain that you want to manage.
After provisioning, you tell it what domains you want to manage (you still have to use Powershell to create the GPOs in each domain). I don't know what effect this step actually has. Next, you get to "Step 4" which is to manually kick off the ServerDiscovery task in Task Scheduler. Interestingly, you can't view the details of what this task actually does.
Next up, ran the Invoke-IPAMGPOProvisioning commandlet. This creates three GPOs in the specified domain, letting the IPAM server read event logs, get through the Windows firewall, and create scheduled tasks on the servers that you choose to manage through the IPAM console.
When you go back to the IPAM console and tell it that a discovered server should be managed, that server is added to the newly created GPOs that are attached at the domain level.
Next, you right-click a discovered server and select "Retrieve all server data". This immediately fires up several tasks in the Task Scheduler:
- Audit: collects events from domain controllers, NPS, and DHCP servers.
- Address Expiry: tracks IP address expiry state and logs notifications.
- ServiceMonitoring: collects DNS zone status events from DNS servers.
- AddressUtilization: collects IP address space usage data from DHCP servers.
- ServerConfiguration: collects configuration info from DHCP and DNS servers.
I found the explanation of these groups in a Microsoft Word document, but it's also listed in the "Description" field of the task.
There's also the ServerAvailability task which "collects status information from DHCP and DNS servers" and the ServerDiscovery task which "automatically discovers domain controllers, DHCP servers and DNS servers in the domains you select".
This is all that I'm going to do with IPAM for now.
10/26/13
Hands-on: require a user certificate in IIS for authentication
Went through a tutorial on certificates in IIS. Installed two certificate role services – the CA and Web Enrollment. I’m not entirely clear on the difference between an Enterprise CA and a Stand-Alone CA, so am installing an Enterprise CA in my lab. Enabled SSL on the default website. Enrolled a user certificate via the web site. Found that out-of-the-box, I was missing some authentication methods, so added ‘em via the Roles & Services wizard. Successfully enabled the requirement to authenticate to Active Directory via basic authentication over SSL *and* posess a user certificate.
10/19/13
Hands-on: BranchCache
10/14:
Two improvements to BranchCache in Server 2012 and Windows 8: 1) workstations can be configured via group policy to act in distributed mode, but reconfigure themselves if a hosted cache server is later implemented in their subnet; 2) you don't need a SSL certificate on your hosted cache server anymore because data encryption is improved.
I wanted to simulate a WAN environment because workstations decide whether or not to use BranchCache by checking network latency. By default, it kicks in if latency is above 80 milliseconds. Hyper-V 3.0 can simulate low bandwidth environments, but not low latency. So, tried the NetworkSimulator, but it wouldn't install on Server 2012. Next, tried the popular live-Linux ISO WANem, which is based on Knoppix. The 3.0 beta gave wildly unpredictable latency, ranging from 2ms to over 3,000ms, with or without any configuration in it. So, switched to the 2.3 version...which worked out perfectly. This blog post encouraged me. I can now set *exactly* how much network latency I want to have between two virtual machines!
10/17:
Spent 4.5 hours studying tonight, mostly on "Why can I ping both interfaces of my WAN simulator from either side, but not through it?".
It was because WANem (and everything else in 192.168.0.x) was forwarding packets to my D-Link router at 192.168.0.1...which consulted its internal routing table and thought the return path for 192.168.5.x must be on the Internet, ignorant that 192.168.0.10 on the LAN should handle it. This I deduced, but confirmed it using Hyper-V 3.0's new port mirroring feature. The solution was to add a routing statement to each server - route add 192.168.5.0 mask 255.255.255.0 192.168.0.10.
10/19:
Fell into a deep time sink when a newly minted domain controller - pdx-dc1.portland.contoso.local - couldn't replicate Active Directory because its replication partner lacked a manual routing statement (needed for this lab setup). After that was fixed, the slow pace of replication imposed by my simulated 1.54Mb link with 90ms of latency meant that incomplete AD replication made WinRM unable to communicate using the server's FQDN, causing the DHCP installation wizard to fail. Nice, huh?
Some troubleshooting take-aways from the above paragraph:
ncpa.cpl opens Network Connections
Here’s a list of the machines involved in today’s study:
When I first opened up the GPMC on pdx-dc1, was puzzled to find that it only presented the top-level domain. To fix, right-click “Domains” and select “Show Domains”. While researching, I learned that applying GPOs across domains is generally frowned upon. You can’t apply a GPO to a generic Active Directory container, such as the default “Computers” folder.
On sea-dc1 (file server), installed BranchCache for Network Files (under the File Server Role), then enabled the GPO “Allow hash publication for all shared folders” so BC doesn’t have to be manually enabled for each share. Configured a GPO to enable BC on Portland PCs. This article offers a general overview. Here is a nice step-by-step article that I wish I’d gotten to earlier. Here’s how BranchCache speeds up the Exchange OAB download. Windows 7 clients need a hotfix for BranchCache to work with files open in read/write mode. Here’s the exhaustive TechNet overview of BranchCache. Here’s how to open Performance Monitor and watch the BranchCache counters. Links to info on SMB 3.0. BranchCache verification and monitoring from TechNet. What’s new in BranchCache for Windows 8/2012. This blog post explains some not-so-well documented aspects of BranchCache.
gpresult /r showed me that sea-dc1 had indeed picked up the GPO that I created.
Disabled the firewall on two Windows 7 PCs and installed the Excel Viewer. Opened perfmon and loaded the BranchCache counters, then zeroed them out by running netsh branchcache reset, then re-enabled BranchCache by running gpupdate. A moment later, ran netsh branchcache show status all to verify that it was ready.
Comparisons with a 1.5MB XLS file, opened in Excel Viewer (after running hashgen -f c:\shared on sea-dc1 to ensure that hashes were ready immediately):
First Windows 7 PC: 14.5 seconds on first try (and PerfMon BC counters ticked upward); 3.7 seconds on second try (opened from the local Offline Files Cache).
Second Windows 7 PC: 5.6 seconds on first try (and PerfMon BC counters ticked upward) because it loaded from the BC cache on the first PC. Nice!
Oddly, I can’t find any TechNet sources (only forum posts and blogs) where Microsoft officially acknowledges that BranchCache is an extension of (and thus dependent on) the Offline Files feature.
However, it’s true. If you disable Offline Files, group policy (at least on Windows 7) does not automatically start the BranchCache service…and if you start the BC service yourself, the BranchCache PerfMon counters go from “—” to “0.000″, but won’t budge again.
Moving on to hosted cache…you can’t enable it on a writeable domain controller. You run Enable-BCHostedServer -RegisterSCP and then Get-BCStatus.
Two improvements to BranchCache in Server 2012 and Windows 8: 1) workstations can be configured via group policy to act in distributed mode, but reconfigure themselves if a hosted cache server is later implemented in their subnet; 2) you don't need a SSL certificate on your hosted cache server anymore because data encryption is improved.
I wanted to simulate a WAN environment because workstations decide whether or not to use BranchCache by checking network latency. By default, it kicks in if latency is above 80 milliseconds. Hyper-V 3.0 can simulate low bandwidth environments, but not low latency. So, tried the NetworkSimulator, but it wouldn't install on Server 2012. Next, tried the popular live-Linux ISO WANem, which is based on Knoppix. The 3.0 beta gave wildly unpredictable latency, ranging from 2ms to over 3,000ms, with or without any configuration in it. So, switched to the 2.3 version...which worked out perfectly. This blog post encouraged me. I can now set *exactly* how much network latency I want to have between two virtual machines!
10/17:
Spent 4.5 hours studying tonight, mostly on "Why can I ping both interfaces of my WAN simulator from either side, but not through it?".
It was because WANem (and everything else in 192.168.0.x) was forwarding packets to my D-Link router at 192.168.0.1...which consulted its internal routing table and thought the return path for 192.168.5.x must be on the Internet, ignorant that 192.168.0.10 on the LAN should handle it. This I deduced, but confirmed it using Hyper-V 3.0's new port mirroring feature. The solution was to add a routing statement to each server - route add 192.168.5.0 mask 255.255.255.0 192.168.0.10.
10/19:
Fell into a deep time sink when a newly minted domain controller - pdx-dc1.portland.contoso.local - couldn't replicate Active Directory because its replication partner lacked a manual routing statement (needed for this lab setup). After that was fixed, the slow pace of replication imposed by my simulated 1.54Mb link with 90ms of latency meant that incomplete AD replication made WinRM unable to communicate using the server's FQDN, causing the DHCP installation wizard to fail. Nice, huh?
Some troubleshooting take-aways from the above paragraph:
net stats srv shows uptime
winrm get winrm/config displays the configuration of WinRM
winrm id -r:pdx-dc1.portland.contoso.local failed until AD replication was done
dcdiag showed when replication was still in progress
Here’s a list of the machines involved in today’s study:
- dc1.contoso.local (idle)
- sea-dc1.seattle.contoso.local (BranchCache enabled file server)
- pdx-dc1.portland.contoso.local (idle)
- pdx-pc1.portland.contoso.local (Windows 7 workstation)
- pdx-pc2.portland.contoso.local (Windows 7 workstation)
- pdx-pc3.portland.contoso.local (Windows 8 workstation)
- pdx-pc4.portland.contoso.local (Windows 8 workstation)
When I first opened up the GPMC on pdx-dc1, was puzzled to find that it only presented the top-level domain. To fix, right-click “Domains” and select “Show Domains”. While researching, I learned that applying GPOs across domains is generally frowned upon. You can’t apply a GPO to a generic Active Directory container, such as the default “Computers” folder.
On sea-dc1 (file server), installed BranchCache for Network Files (under the File Server Role), then enabled the GPO “Allow hash publication for all shared folders” so BC doesn’t have to be manually enabled for each share. Configured a GPO to enable BC on Portland PCs. This article offers a general overview. Here is a nice step-by-step article that I wish I’d gotten to earlier. Here’s how BranchCache speeds up the Exchange OAB download. Windows 7 clients need a hotfix for BranchCache to work with files open in read/write mode. Here’s the exhaustive TechNet overview of BranchCache. Here’s how to open Performance Monitor and watch the BranchCache counters. Links to info on SMB 3.0. BranchCache verification and monitoring from TechNet. What’s new in BranchCache for Windows 8/2012. This blog post explains some not-so-well documented aspects of BranchCache.
gpresult /r showed me that sea-dc1 had indeed picked up the GPO that I created.
Disabled the firewall on two Windows 7 PCs and installed the Excel Viewer. Opened perfmon and loaded the BranchCache counters, then zeroed them out by running netsh branchcache reset, then re-enabled BranchCache by running gpupdate. A moment later, ran netsh branchcache show status all to verify that it was ready.
Comparisons with a 1.5MB XLS file, opened in Excel Viewer (after running hashgen -f c:\shared on sea-dc1 to ensure that hashes were ready immediately):
First Windows 7 PC: 14.5 seconds on first try (and PerfMon BC counters ticked upward); 3.7 seconds on second try (opened from the local Offline Files Cache).
Second Windows 7 PC: 5.6 seconds on first try (and PerfMon BC counters ticked upward) because it loaded from the BC cache on the first PC. Nice!
Oddly, I can’t find any TechNet sources (only forum posts and blogs) where Microsoft officially acknowledges that BranchCache is an extension of (and thus dependent on) the Offline Files feature.
However, it’s true. If you disable Offline Files, group policy (at least on Windows 7) does not automatically start the BranchCache service…and if you start the BC service yourself, the BranchCache PerfMon counters go from “—” to “0.000″, but won’t budge again.
Moving on to hosted cache…you can’t enable it on a writeable domain controller. You run Enable-BCHostedServer -RegisterSCP and then Get-BCStatus.
10/12/13
Study: Office 365
10/6: I want to get some hands-on practice with Active Directory Federation for Office 365. To get started, here is an overview of the Office 365 plans:
- Small Business ($5/month): hosted email.
- Small Business Premium ($12.50/month): all the desktop programs + hosted email.
- ProPlus ($12/month): all the desktop programs; used with on-premise email.
- Midsize Business ($15/month): same as SBP, but with a 300 user cap instead of 25; AD Federation.
- Exchange Online Plan 1 ($4/month): hosted email; AD Federation.
- Enterprise 1 ($8/month): hosted email + a bit more; AD Federation.
- Enterprise 3 ($20/month): EVERYTHING – hosted email, desktop programs, etc.
(link to difference between ProPlus and Small Business Premium)
Office 365 is not for terminal servers (you need a VLK instead).
Of course, you have to be able to endure the terrible aesthetics of Office 365.
10/11: Signed into a 30 day free trial of Office 365 for midsize businesses. Associated my domain name of trimideas.com. Spent about 50 minutes watching a TechEd video about ways to authenticate to O365: use their built-in onmicrosoft.com domain, manually build out accounts, use a directory sync tool, or use ADFS. I was going to use onramp.office365.com, but it seems to be an all or nothing approach - since I didn’t have my MX records pointing to it, it wasn’t interested in talking.
Signed up for an $11/year certificate from GeoTrust through NameCheap.com. Followed outstanding instructions at MessageOps to configure ADFS.
For small businesses, there’s very little value in ADFS w/ Office 365 since ADFS offers precisely two benefits: your own AD server remains the single point of authentication (so employee access can be swiftly terminated) and SSO (with a few caveats). You use the directory sync tool to populate your user accounts (and, optionally, password hashes) into the Azure Cloud. On workstations, the Windows 7 Credential Manager can cache your login and nearly eliminate any pain surrounding a lack of SSO from the end-user’s view. Thus ended my interest in ADFS.
Tried out the Office 365 app for iPhone. It seems decent and unremarkable. Also looked at Windows Azure – you can get a 30 day trial with $200 to spend on their services in that time. They offer virtual machines, websites, databases, etc.
Installing the Windows Azure Active Directory Sync Setup tool on a Server 2012 member server was difficult. Eventually resolved it by deleting the “SQL Server” directory in both “Program Files” directories. Bizzare, huh? After that, it worked fine.
Watched a good video about troubleshooting Windows 8.
- Small Business ($5/month): hosted email.
- Small Business Premium ($12.50/month): all the desktop programs + hosted email.
- ProPlus ($12/month): all the desktop programs; used with on-premise email.
- Midsize Business ($15/month): same as SBP, but with a 300 user cap instead of 25; AD Federation.
- Exchange Online Plan 1 ($4/month): hosted email; AD Federation.
- Enterprise 1 ($8/month): hosted email + a bit more; AD Federation.
- Enterprise 3 ($20/month): EVERYTHING – hosted email, desktop programs, etc.
(link to difference between ProPlus and Small Business Premium)
Office 365 is not for terminal servers (you need a VLK instead).
Of course, you have to be able to endure the terrible aesthetics of Office 365.
10/11: Signed into a 30 day free trial of Office 365 for midsize businesses. Associated my domain name of trimideas.com. Spent about 50 minutes watching a TechEd video about ways to authenticate to O365: use their built-in onmicrosoft.com domain, manually build out accounts, use a directory sync tool, or use ADFS. I was going to use onramp.office365.com, but it seems to be an all or nothing approach - since I didn’t have my MX records pointing to it, it wasn’t interested in talking.
Signed up for an $11/year certificate from GeoTrust through NameCheap.com. Followed outstanding instructions at MessageOps to configure ADFS.
For small businesses, there’s very little value in ADFS w/ Office 365 since ADFS offers precisely two benefits: your own AD server remains the single point of authentication (so employee access can be swiftly terminated) and SSO (with a few caveats). You use the directory sync tool to populate your user accounts (and, optionally, password hashes) into the Azure Cloud. On workstations, the Windows 7 Credential Manager can cache your login and nearly eliminate any pain surrounding a lack of SSO from the end-user’s view. Thus ended my interest in ADFS.
Tried out the Office 365 app for iPhone. It seems decent and unremarkable. Also looked at Windows Azure – you can get a 30 day trial with $200 to spend on their services in that time. They offer virtual machines, websites, databases, etc.
Installing the Windows Azure Active Directory Sync Setup tool on a Server 2012 member server was difficult. Eventually resolved it by deleting the “SQL Server” directory in both “Program Files” directories. Bizzare, huh? After that, it worked fine.
Watched a good video about troubleshooting Windows 8.
10/1/13
Study: Active Directory Federation Services
Someday I want to take the Desktop Virtualization course at TrainSignal. Then I want to take the SQL Server 2008 Administration course and associated exam.
AD FS is essentially an enhanced one-way AD Trust with selective authentication all wrapped up in SSL for transport over the Internet. Side note: Server 2012 doesn't include a “System Restore” feature like desktops do.
To summarize what I’ve learned about AD FS – you have two Active Directory domains (or it could be another identity source) that trust each other. The side which provides the initial validation of a user’s identity (e.g. Cash Cow Capital Group) is the “claims provider” or “security token service” (STS); the side which provides a program or service is the “relying party” (e.g. Globomantics’ secure web server). The application must be “claims aware” – i.e. designed specifically to be aware of claims – or at least built with .Net, such as OWA, so the “Claims to Windows Token service” can assist it. All the examples that I’ve seen are web services, but I think that Office 365 offers AD Federation for your Outlook…but that makes sense because nowadays Outlook does all its “talking” over an HTTPS connection to the server so it can work from anywhere.
Back to the topic…so on the claims provider (aka STS), you configure the “Relying Party Trust” where you can configure three types of rules: you must have an “Issuance Authorization Rule” which defines who is allowed; an optional “Issuance Transform Rule” which customizes the claim data (i.e. user attributes) that you’ll send back to the relying party; and a “Delgation Authorization Rule”…which remains a mystery to me.
On the relying party (which serves up data to users authenticated by the STS) you configure a “Claims Provider Trust” where you state that you accept them as a trusted STS, then create an “Acceptance Transform Rule” which takes the claim(s) handed over by the STS and (optionally) converts it into different values before handing it up to the application.
I’m not at all sure that I understand the difference between Issuance Authorization & Issuance Transform rules, but for the 70-412 exam I don’t think it’s a big deal. Found a write-up by Join.Me/LogMeIn on how to do AD FS with them. The Transcender practice test has a question about AD FS endpoints which was discouraging to me because the TrainSignal training didn’t even touch on that and I certainly haven’t investigated their specific purposes.
AD FS is essentially an enhanced one-way AD Trust with selective authentication all wrapped up in SSL for transport over the Internet. Side note: Server 2012 doesn't include a “System Restore” feature like desktops do.
To summarize what I’ve learned about AD FS – you have two Active Directory domains (or it could be another identity source) that trust each other. The side which provides the initial validation of a user’s identity (e.g. Cash Cow Capital Group) is the “claims provider” or “security token service” (STS); the side which provides a program or service is the “relying party” (e.g. Globomantics’ secure web server). The application must be “claims aware” – i.e. designed specifically to be aware of claims – or at least built with .Net, such as OWA, so the “Claims to Windows Token service” can assist it. All the examples that I’ve seen are web services, but I think that Office 365 offers AD Federation for your Outlook…but that makes sense because nowadays Outlook does all its “talking” over an HTTPS connection to the server so it can work from anywhere.
Back to the topic…so on the claims provider (aka STS), you configure the “Relying Party Trust” where you can configure three types of rules: you must have an “Issuance Authorization Rule” which defines who is allowed; an optional “Issuance Transform Rule” which customizes the claim data (i.e. user attributes) that you’ll send back to the relying party; and a “Delgation Authorization Rule”…which remains a mystery to me.
On the relying party (which serves up data to users authenticated by the STS) you configure a “Claims Provider Trust” where you state that you accept them as a trusted STS, then create an “Acceptance Transform Rule” which takes the claim(s) handed over by the STS and (optionally) converts it into different values before handing it up to the application.
I’m not at all sure that I understand the difference between Issuance Authorization & Issuance Transform rules, but for the 70-412 exam I don’t think it’s a big deal. Found a write-up by Join.Me/LogMeIn on how to do AD FS with them. The Transcender practice test has a question about AD FS endpoints which was discouraging to me because the TrainSignal training didn’t even touch on that and I certainly haven’t investigated their specific purposes.