Exchange 2010 - search the database for an email

Client: "We need an email sent to so-and-so back in June 2012.  We can't find it.  Can you?"

On the client's Exchange 2010 SP1 server, I assigned myself the "Discovery Management" role under https://server/ecp/?p=AdminRoleGroups.  When I created a search at https://server/ecp/?exsvurl=1&p=MailboxSearches, it threw an error shown in the screenshot.

This was solved by following these instructions on a TechNet blog to fix an issue peculiar to Exchange 2010 SP1, and then rebuild the index.

Also learned about some built-in troubleshooting scripts.  This Technet article talks about the full-text index catalog.  This Technet article is all about "Understanding Multi-Mailbox Search".  This article says that the built-in Exchange search is great for finding content, but not so good for legal searches because the data is easily contaminated.

Hands-on: IPAM

The first thing to note about IPAM is that you can't put it on a domain controller.  It can oversee multiple domains within a single forest.  IPAM is a feature, not a role.  After installing it on IPAM1.seattle.contoso.local, I installed the client on a domain controller and tried to manage it, but got the delightful error shown in this screenshot.  This was because I was signed in with an enterprise admin account from a parent domain - whoami /groups to the rescue.  Signing in with a domain admin account eliminated the error.  Otherwise, I think that adding my account to the WinRMRemoteWMIUsers group on the IPAM server would've worked too.

Moving back to the IPAM server itself, I set the provisioning method to GPO (instead of manual) and noted that you have to run Invoke-IPAMGPOProvisioning in each domain that you want to manage.

After provisioning, you tell it what domains you want to manage (you still have to use Powershell to create the GPOs in each domain).  I don't know what effect this step actually has.  Next, you get to "Step 4" which is to manually kick off the ServerDiscovery task in Task Scheduler.  Interestingly, you can't view the details of what this task actually does.

Next up, ran the Invoke-IPAMGPOProvisioning commandlet.  This creates three GPOs in the specified domain, letting the IPAM server read event logs, get through the Windows firewall, and create scheduled tasks on the servers that you choose to manage through the IPAM console.

When you go back to the IPAM console and tell it that a discovered server should be managed, that server is added to the newly created GPOs that are attached at the domain level.

Next, you right-click a discovered server and select "Retrieve all server data".  This immediately fires up several tasks in the Task Scheduler:
- Audit: collects events from domain controllers, NPS, and DHCP servers.
- Address Expiry: tracks IP address expiry state and logs notifications.
- ServiceMonitoring: collects DNS zone status events from DNS servers.
- AddressUtilization: collects IP address space usage data from DHCP servers.
- ServerConfiguration: collects configuration info from DHCP and DNS servers.

I found the explanation of these groups in a Microsoft Word document, but it's also listed in the "Description" field of the task.

There's also the ServerAvailability task which "collects status information from DHCP and DNS servers" and the ServerDiscovery task which "automatically discovers domain controllers, DHCP servers and DNS servers in the domains you select".

This is all that I'm going to do with IPAM for now.

Hands-on: require a user certificate in IIS for authentication

Went through a tutorial on certificates in IIS. Installed two certificate role services – the CA and Web Enrollment. I’m not entirely clear on the difference between an Enterprise CA and a Stand-Alone CA, so am installing an Enterprise CA in my lab. Enabled SSL on the default website. Enrolled a user certificate via the web site. Found that out-of-the-box, I was missing some authentication methods, so added ‘em via the Roles & Services wizard. Successfully enabled the requirement to authenticate to Active Directory via basic authentication over SSL *and* posess a user certificate.

Hands-on: BranchCache

10/14:

Two improvements to BranchCache in Server 2012 and Windows 8: 1) workstations can be configured via group policy to act in distributed mode, but reconfigure themselves if a hosted cache server is later implemented in their subnet; 2) you don't need a SSL certificate on your hosted cache server anymore because data encryption is improved.

I wanted to simulate a WAN environment because workstations decide whether or not to use BranchCache by checking network latency. By default, it kicks in if latency is above 80 milliseconds. Hyper-V 3.0 can simulate low bandwidth environments, but not low latency. So, tried the NetworkSimulator, but it wouldn't install on Server 2012. Next, tried the popular live-Linux ISO WANem, which is based on Knoppix. The 3.0 beta gave wildly unpredictable latency, ranging from 2ms to over 3,000ms, with or without any configuration in it. So, switched to the 2.3 version...which worked out perfectly. This blog post encouraged me. I can now set *exactly* how much network latency I want to have between two virtual machines!

10/17:

Spent 4.5 hours studying tonight, mostly on "Why can I ping both interfaces of my WAN simulator from either side, but not through it?".

It was because WANem (and everything else in 192.168.0.x) was forwarding packets to my D-Link router at 192.168.0.1...which consulted its internal routing table and thought the return path for 192.168.5.x must be on the Internet, ignorant that 192.168.0.10 on the LAN should handle it. This I deduced, but confirmed it using Hyper-V 3.0's new port mirroring feature. The solution was to add a routing statement to each server - route add 192.168.5.0 mask 255.255.255.0 192.168.0.10.

10/19:

Fell into a deep time sink when a newly minted domain controller - pdx-dc1.portland.contoso.local - couldn't replicate Active Directory because its replication partner lacked a manual routing statement (needed for this lab setup). After that was fixed, the slow pace of replication imposed by my simulated 1.54Mb link with 90ms of latency meant that incomplete AD replication made WinRM unable to communicate using the server's FQDN, causing the DHCP installation wizard to fail. Nice, huh?

Some troubleshooting take-aways from the above paragraph:

 

ncpa.cpl opens Network Connections

net stats srv shows uptime

winrm get winrm/config displays the configuration of WinRM

winrm id -r:pdx-dc1.portland.contoso.local failed until AD replication was done

dcdiag showed when replication was still in progress

WinRM troubleshooting guide

Here’s a list of the machines involved in today’s study:

• dc1.contoso.local (idle)
• sea-dc1.seattle.contoso.local (BranchCache enabled file server)
• pdx-dc1.portland.contoso.local (idle)
• pdx-pc1.portland.contoso.local (Windows 7 workstation)
• pdx-pc2.portland.contoso.local (Windows 7 workstation)
• pdx-pc3.portland.contoso.local (Windows 8 workstation)
• pdx-pc4.portland.contoso.local (Windows 8 workstation)

Portland and Seattle are separated by a simulated 1.5Mb T1 with (a high) 90ms of latency.

When I first opened up the GPMC on pdx-dc1, was puzzled to find that it only presented the top-level domain. To fix, right-click “Domains” and select “Show Domains”. While researching, I learned that applying GPOs across domains is generally frowned upon. You can’t apply a GPO to a generic Active Directory container, such as the default “Computers” folder.








On sea-dc1 (file server), installed BranchCache for Network Files (under the File Server Role), then enabled the GPO “Allow hash publication for all shared folders” so BC doesn’t have to be manually enabled for each share. Configured a GPO to enable BC on Portland PCs. This article offers a general overview. Here is a nice step-by-step article that I wish I’d gotten to earlier. Here’s how BranchCache speeds up the Exchange OAB download. Windows 7 clients need a hotfix for BranchCache to work with files open in read/write mode. Here’s the exhaustive TechNet overview of BranchCache. Here’s how to open Performance Monitor and watch the BranchCache counters. Links to info on SMB 3.0. BranchCache verification and monitoring from TechNet. What’s new in BranchCache for Windows 8/2012. This blog post explains some not-so-well documented aspects of BranchCache.

gpresult /r showed me that sea-dc1 had indeed picked up the GPO that I created.

Disabled the firewall on two Windows 7 PCs and installed the Excel Viewer. Opened perfmon and loaded the BranchCache counters, then zeroed them out by running netsh branchcache reset, then re-enabled BranchCache by running gpupdate. A moment later, ran netsh branchcache show status all to verify that it was ready.

Comparisons with a 1.5MB XLS file, opened in Excel Viewer (after running hashgen -f c:\shared on sea-dc1 to ensure that hashes were ready immediately):

First Windows 7 PC: 14.5 seconds on first try (and PerfMon BC counters ticked upward); 3.7 seconds on second try (opened from the local Offline Files Cache).

Second Windows 7 PC: 5.6 seconds on first try (and PerfMon BC counters ticked upward) because it loaded from the BC cache on the first PC. Nice!

Oddly, I can’t find any TechNet sources (only forum posts and blogs) where Microsoft officially acknowledges that BranchCache is an extension of (and thus dependent on) the Offline Files feature.

However, it’s true. If you disable Offline Files, group policy (at least on Windows 7) does not automatically start the BranchCache service…and if you start the BC service yourself, the BranchCache PerfMon counters go from “—” to “0.000″, but won’t budge again.

Moving on to hosted cache…you can’t enable it on a writeable domain controller. You run Enable-BCHostedServer -RegisterSCP and then Get-BCStatus.

Study: Office 365

10/6: I want to get some hands-on practice with Active Directory Federation for Office 365. To get started, here is an overview of the Office 365 plans:

- Small Business ($5/month): hosted email.
- Small Business Premium ($12.50/month): all the desktop programs + hosted email.
- ProPlus ($12/month): all the desktop programs; used with on-premise email.
- Midsize Business ($15/month): same as SBP, but with a 300 user cap instead of 25; AD Federation.
- Exchange Online Plan 1 ($4/month): hosted email; AD Federation.
- Enterprise 1 ($8/month): hosted email + a bit more; AD Federation.
- Enterprise 3 ($20/month): EVERYTHING – hosted email, desktop programs, etc.

(link to difference between ProPlus and Small Business Premium)
Office 365 is not for terminal servers (you need a VLK instead).
Of course, you have to be able to endure the terrible aesthetics of Office 365.

10/11: Signed into a 30 day free trial of Office 365 for midsize businesses. Associated my domain name of trimideas.com. Spent about 50 minutes watching a TechEd video about ways to authenticate to O365: use their built-in onmicrosoft.com domain, manually build out accounts, use a directory sync tool, or use ADFS. I was going to use onramp.office365.com, but it seems to be an all or nothing approach - since I didn’t have my MX records pointing to it, it wasn’t interested in talking.

Signed up for an $11/year certificate from GeoTrust through NameCheap.com. Followed outstanding instructions at MessageOps to configure ADFS.

For small businesses, there’s very little value in ADFS w/ Office 365 since ADFS offers precisely two benefits: your own AD server remains the single point of authentication (so employee access can be swiftly terminated) and SSO (with a few caveats).  You use the directory sync tool to populate your user accounts (and, optionally, password hashes) into the Azure Cloud. On workstations, the Windows 7 Credential Manager can cache your login and nearly eliminate any pain surrounding a lack of SSO from the end-user’s view. Thus ended my interest in ADFS.

Tried out the Office 365 app for iPhone. It seems decent and unremarkable. Also looked at Windows Azure – you can get a 30 day trial with $200 to spend on their services in that time. They offer virtual machines, websites, databases, etc.

Installing the Windows Azure Active Directory Sync Setup tool on a Server 2012 member server was difficult. Eventually resolved it by deleting the “SQL Server” directory in both “Program Files” directories. Bizzare, huh? After that, it worked fine.

Watched a good video about troubleshooting Windows 8.

Study: Active Directory Federation Services

Someday I want to take the Desktop Virtualization course at TrainSignal. Then I want to take the SQL Server 2008 Administration course and associated exam.

AD FS is essentially an enhanced one-way AD Trust with selective authentication all wrapped up in SSL for transport over the Internet. Side note: Server 2012 doesn't include a “System Restore” feature like desktops do.

To summarize what I’ve learned about AD FS – you have two Active Directory domains (or it could be another identity source) that trust each other. The side which provides the initial validation of a user’s identity (e.g. Cash Cow Capital Group) is the “claims provider” or “security token service” (STS); the side which provides a program or service is the “relying party” (e.g. Globomantics’ secure web server). The application must be “claims aware” – i.e. designed specifically to be aware of claims – or at least built with .Net, such as OWA, so the “Claims to Windows Token service” can assist it. All the examples that I’ve seen are web services, but I think that Office 365 offers AD Federation for your Outlook…but that makes sense because nowadays Outlook does all its “talking” over an HTTPS connection to the server so it can work from anywhere.

Back to the topic…so on the claims provider (aka STS), you configure the “Relying Party Trust” where you can configure three types of rules: you must have an “Issuance Authorization Rule” which defines who is allowed; an optional “Issuance Transform Rule” which customizes the claim data (i.e. user attributes) that you’ll send back to the relying party; and a “Delgation Authorization Rule”…which remains a mystery to me.

On the relying party (which serves up data to users authenticated by the STS) you configure a “Claims Provider Trust” where you state that you accept them as a trusted STS, then create an “Acceptance Transform Rule” which takes the claim(s) handed over by the STS and (optionally) converts it into different values before handing it up to the application.

I’m not at all sure that I understand the difference between Issuance Authorization & Issuance Transform rules, but for the 70-412 exam I don’t think it’s a big deal. Found a write-up by Join.Me/LogMeIn on how to do AD FS with them. The Transcender practice test has a question about AD FS endpoints which was discouraging to me because the TrainSignal training didn’t even touch on that and I certainly haven’t investigated their specific purposes.

Study notes from September

Working toward the 70-412 exam:

A nifty DHCP icon reference is at http://technet.microsoft.com/en-us/library/gg722802. No such reference is available for DNS icons, but I did learn that if you sign a DNS zone (called DNSSEC), the icon for the zone gets a padlock on it.

Windows Server Backup 2012 can do system state, specific folders, and bare metal restores. Microsoft’s cloud backup (part of Azure) only supports folders at this time. Active Directory site links are bridged by default (which means they can all talk to each other directly). A bridge is a collection of 2+ site links (not the sites themselves). A site link is a collection of 2+ sites that can replicate to each other. To backup a certificate authority, you must be a member of the CA Administrators group, or the Backup Operators group. Reviewed Active Directory functional levels.

Studied up on Active Directory Federation Services tonight: watched the first 10 minutes of a video that explains it in simple terms, and the first 38 minutes of a really outstanding video that explains the concept and big idea of identity federation. By the time I started reading this TechNet article about claim rule set types, my eyes had glazed over and I decided that I really need to try this out hands-on. Wikipedia nicely explains claims-based identity. I did learn that you can see computer certificates by function, rather than logical store, in the MMC by simply toggling the view.

Learned about IPAM in this hour long video by Microsoft evangelist Rick Claus at TechEd Australia 2012. There’s also several “Jump Start 2012″ training videos with Rick and another guy, such as this one about IIS 8 (dynamic blacklisting, central SSL store), DHCP failover, and IPAM.

Microsoft’s Second Shot program is back – yay!

Hyper-V migration options in Server 2012: No shared storage is required for a “live storage” migration, but a similar processor architecture is required, VHD or VHDX hard disks are required (no pass-through disks), and the Hyper-V hosts must be members of the same AD domain. A “quick” migration can be used in clusters with shared storage – the guest’s state is saved on a node that is then “failed” and the VM is resumed on a “surviving” node. This article lays it all out…and here’s a native-English write-up on Hyper-V migration which is very readable. Here’s an article about Hyper-V failover policy in a clustered environment. If you need to move a Hyper-V guest to a host with a different processor manufacturer (Intel to AMD or vice versa), you need to do an export/import of the VM (page 233 in the 70-417 book). Lastly, if you have a VM with multiple snapshots, you can apply whichever snapshot you want, then export the VM & import it as a new VM - thus giving you a single new VHD to work with, unencumbered by multiple snapshots.

Last night at the office I worked on getting the FreePBX distribution of Asterisk to play happily with the DPMA (Digium Phones Module for Asterisk). This was inspired by the failure of AsteriskNOW 3.0 to work on an HP Proliant rack-mount server. I needed to run yum install avahi to get the mDNS service that the DPMA uses to notify Digium phones of its presence; then deleted the .htaccess files in /var/www/html/admin and /var/www/html/admin/modules to allow the Digium phones to download firmware from the PBX. Also uploaded the client’s logo so it appears on each phone’s display.

Back to Server 2012: in NLB, a non-clustered application will fail over to the host with the lowest numerical host priority.

Active Directory replication – in AD Sites & Services, you can tell a server which partner to replicate with, trigger an immediate repliction, and edit the “options” attribute to tell two sites to replicate immediately (within 5 seconds) via change notification rather than waiting the 15 (minimum) to 180 (default) or more minutes for a compressed and scheduled replication. Instant replication on manually created site links. TechNet in-depth explanation of AD replication. Troubleshooting AD replication. Windows PE order of operations.

Trust types:
- External trusts are for AD + NT 4 or a domain not included in a forest trust.
- Realm trusts are for AD + a non-Windows Kerberos 5+ realm (e.g. Unix).
- Forest trusts are for AD forests.
- Shortcut trusts are for situations where a forest trust is present, but slow; the shortcut trust is between two domains so that the chain of authentication is shorter & faster.

The term “transitive” in this context means that subdomains are included; “non-transitive” means that only those particular domains trust each other. The term “selective authentication” means that only explicitly defined security principles from the other forest or domain are authenticated, not everybody in the entire forest or domain.

A Server 2012 cluster can use basic disks at the “witness disk”, but not dynamic ones (reinforcing my aversion to dynamic disks). You can use SMB 3.0 file shares as shared storage for Hyper-V servers in a failover cluster.

Study time: installed an RODC. Learned that PowerShell 3.0 is available for Windows 7 (it ships on Windows 8 & 2012). It includes “resolve-DNSname”, as an enhanced alternative to NSLookup. Read about DNS Security Extensions - they can replicate automatically to other domain controllers, but for stand-alone DNS servers, you must manually import “Trust Anchors”. You can require Windows 7/8 DNS clients to use DNSSec for specific DNS suffixes via group policy. DNSLint is a tool for troubleshooting DNS replication. Here’s an article that explains how RODCs handle DNS updates (they send the DNS client an SOA record for a writeable DNS server, then they ask that SOA server for just that single record, to update their own cache – since they can only replicate in, not out).