Troubleshooting System Center Operations Manager (SCOM) Server performance

Microsoft has write an overview for troubleshooting performance for SCOM.

For OpsMgr 2007 and R2 – Root management server (RMS)

Configuration update bursts are caused by management pack imports and by discovery data. When system performance is slow, the most likely bottlenecks are, first, the CPU and, second, the OpsMgr installation disk I/O.

The RMS is responsible for generating and sending configuration files to all affected Health Services.

For Workflow reloading (which is caused by new configuration on RMS), the most likely bottlenecks are the same: the CPU first, and OpsMgr installation disk I/O second. The RMS is responsible for reading the configuration file, for loading and initializing all workflows that run on it, and for updating the RMS HealthService store when the configuration file is updated on the RMS.

For local workflow activity bursts (which is when agents change their availability), the most likely bottleneck is the CPU. If you find that the CPU is not working at maximum capacity, the next most likely bottleneck is the hard disk. The RMS is responsible for monitoring the availability of all agents that are using RMS local workflows. The RMS also hosts distributed dependency monitors that use the disk.

Management server

During a configuration update burst (that is caused by MP import and discovery), the typical bottlenecks are, first, the CPU and, second, the OpsMgr installation disk I/O. The management server is responsible of forwarding configuration files from the RMS to the target agents.

For Operational data collection, bottlenecks are typically caused by the CPU. The disk I/O may also be at maximum capacity, but that is not as likely. The management server is responsible for decompressing and decrypting incoming operational data, and inserting it into the Operational Database. It also sends acknowledgements (ACKs) back to the agents or gateways after it receives operational data, and uses disk queuing to temporarily store these outgoing ACKs. Lastly, the management server will also forward monitor state changes (by using a disk queue) to the RMS for distributed dependency monitors.

Gateway

The gateway is both CPU-bound and I/O-bound. When the gateway is relaying a large amount of data, both the CPU and I/O operations may show high usage. Most of the CPU usage is caused by the decompression, compression, encryption, and decryption of the incoming data, and also by the transfer of that data. All data that is received by the gateway and from the agents is stored in a persistent queue on disk, to be read and forwarded to the management server by the gateway Health service. This can cause heavy disk usage. This usage can be significant when the gateway is taken temporarily offline and must then handle accumulated agent data that the agents generated and tried to send when the GW was still offline.

To troubleshoot the issue in this situation, collect the following information for each affected management server or gateway:

  • Exact Windows version, edition, and build number (for example, Windows Server 2003 Enterprise x64 SP2)
  • Number of processors
  • Amount of RAM
  • Drive that contains the Health Service State folder
  • Whether the antivirus software is configured to exclude the Health Service store

    Note For more information, click the following article number to view the article in the Microsoft Knowledge Base:

    975931

    (http://support.microsoft.com/kb/975931/ )

    Recommendations for antivirus exclusions that relate to Operations Manager

  • RAID level (0, 1, 5, 0+1 or 1+0) for the drive that is used by the Health Service State
  • Number of disks used for the RAID
  • Whether battery-backed write cache is enabled on the array controller

Troubleshooting SQL Server Performance

Operational Database (OperationsManager)

For the OperationsManager database, the most likely bottleneck is the disk array. If the disk array is not at maximum I/O capacity, the next most likely bottleneck is the CPU. The database will experience occasional slowdowns and operational “data storms” (very high incidences of events, alerts, and performance data or state changes that persist for a relatively long time). A short burst typically does not cause any significant delay for an extended period of time.

During operational data insertion, the database disks are primarily used for writes. CPU use is usually caused by SQL Server churn. This may occur when you have large and complex queries, heavy data insertion, and the grooming of large tables (which, by default, occurs at midnight). Typically, the grooming of even large Events and Performance Data tables does not consume excessive CPU or disk resources. However, the grooming pf the Alert and State Change tables can be CPU-intensive for large tables.

The database is also CPU-bound when it handles configuration redistribution bursts, which are caused by MP imports or by a large instance space change. In these cases, the Config service queries the database for new agent configuration. This ususally causes CPU spikes to occur on the database before the service sends the configuration updates to the agents.

Data Warehouse (OperationsManagerDW)

For the OperationsManagerDW database, the most likely bottleneck is the disk array. This usually occurs because of very large operational data insertions. In these cases, the disks are mostly busy performing writes. Usually, the disks are performing few reads, except to handle manually-generated Reporting views because these run queries on the data warehouse.

CPU usage is usually caused by SQL Server churn. CPU spikes may occur during heavy partitioning activity (when tables become very large and then get partitioned), the generation of complex reports, and large amounts of alerts in the database, with which the data warehouse must constantly sync up.

General troubleshooting

To troubleshoot the issue in this situation, collect the following information for each affected management server or gateway:

  • Exact Windows version, edition, and build number (for example, Windows Server 2003 Enterprise x64 SP2)
  • Number of processors
  • Amount of RAM
  • Amount of memory that is allocated to SQL Server
  • Whether SQL Server is 32-bit and is AWE enabled

    Note You can find most of this information in SQL Server Management Studio or in SQL Server Enterprise Manager. To do this, open the Properties window of the server, and then click the General and Memory tabs. The General tab includes the SQL Server version, the Windows version, the platform, the amount of RAM, and the number of processors. The Memory tab includes the memory that is allocated to SQL Server. In Microsoft SQL Server 2008 and in Microsoft SQL Server 2005, the Memory tab also includes the AWE option. To determine whether AWE is enabled in Microsoft SQL Server 2000, run the following command in the Microsoft SQL Query Analyzer:

    sp_configure ‘show advanced options’, 1
    RECONFIGURE
    GO
    sp_configure ‘awe enabled’

    The returned values for config_value and for run_value will be 1 if AWE is enabled.

    If OS is 32-bit and RAM is 4 GB or greater, check whether the /pae or /3gb switches exist in the Boot.ini. file. These options could be configured incorrectly if the server was originally installed by having 4 GB or less of RAM, and if the RAM was later upgraded.

    For 32-bit servers that have 4 GB of RAM, the /3gb switch in Boot.ini increases the amount of memory that SQL Server can address (from 2 to 3 GB). For 32-bit servers that have more than 4 GB of RAM, the /3gb switch in Boot.ini could actually limit the amount of memory that SQL Server can address. For these systems, add the /pae switch to Boot.ini, and then enable AWE in SQL Server.

    On a multi-processor system, check the Max Degree of Parallelism (MAXDOP) setting. In SQL Server 2008 and in SQL Server 2005, this option is on the Advanced tab in the Properties dialog box for the server. To determine this setting in SQL Server 2000, run the following command in SQL Query Analyzer:

    sp_configure ‘show advanced options’, 1
    RECONFIGURE
    GO
    sp_configure ‘max degree of parallelism’

    The default value is 0, which means that all available processors will be used. A setting of 0 is fine for servers that have eight or fewer processors. For servers that have more than eight processors, the time that it takes SQL Server to coordinate the use of all processors may be counterproductive. Therefore, for servers that have more than eight processors, you generally should set Max Degree of Parallelism to a value of 8. To do this, run the following command in SQL Query Analyzer:

    sp_configure ‘show advanced options’, 1
    GO
    RECONFIGURE WITH OVERRIDE
    GO
    sp_configure ‘max degree of parallelism’, 8
    GO
    RECONFIGURE WITH OVERRIDE
    GO

  • Drive letters that contain data warehouse or Ops and Tempdb files
  • Whether the antivirus software is configured to exclude SQL data and log files (Antivirus software cannot scan SQL database files. Trying to do this can degrade performance.)
  • Amount of free space on drives that contain data warehouse or Ops and Tempdb files
  • Storage type (SAN or local)
  • RAID level (0, 1, 5, 0+1 or 1+0) for drives that are used by SQL Server
  • If SAN storage us used: amount of spindles on each LUN that is used by SQL Server
  • In OpsMgr 2007 SP1: whether hotfix 969130 (data warehouse event grooming) or SP1 hotfix rollup 971541 is applied
  • If the converted Exchange 2007 managment pack is being used or has ever been used: amount of rows in the LocalizedText table in the Ops DB and in the EventPublisher table in the data warehouse database

    Note To determine the row amounts, run the following commands: 

    USE OperationsManager SELECT COUNT(*) FROM LocalizedText
    USE OperationsManagerDW SELECT COUNT(*) FROM EventPublisher

Counters to identify memory pressure

  • MSSQL$<instance>: Buffer Manager: Page Life expectancy – How long pages persist in the buffer pool. If this value is below 300 seconds, it may indicate that the server could use more memory. It could also result from index fragmentation.
  • MSSQL$<instance>: Buffer Manager: Lazy Writes/sec – Lazy writer frees space in the buffer by moving pages to disk. Generally, the value should not consistently exceed 20 writes per second. Ideally, it would be close to zero.
  • Memory: Available Mbytes – Values below 100 MB may indicate memory pressure. Memory pressure is clearly present when this amount is less than 10 MB.
  • Process: Private Bytes: _Total: This is the amount of memory (physical and page) being used by all processes combined.
  • Process: Working Set: _Total: This is the amount of physical memory being used by all processes combined. If the value for this counter is significantly below the value for Process: Private Bytes: _Total, it indicates that processes are paging too heavily. A difference of more than 10% is probably significant.

Counters to identify disk pressure

Capture these Physical Disk counters for all drives that contain SQL data or log files:

  • % Idle Time: How much disk idle time is being reported. Anything below 50 percent could indicate a disk bottleneck.
  • Avg. Disk Queue Length: This value should not exceed 2 times the number of spindles on a LUN. For example, if a LUN has 25 spindles, a value of 50 is acceptable. However, if a LUN has 10 spindles, a value of 25 is too high. You could use the following formulas based on the RAID level and number of disks in the RAID configuration:
    • RAID 0: All of the disks are doing work in a RAID 0 set
    • Average Disk Queue Length <= # (Disks in the array) *2
    • RAID 1: half the disks are “doing work”; therefore, only half of them can be counted toward Disks Queue
    • Average Disk Queue Length <= # (Disks in the array/2) *2
    • RAID 10: half the disks are “doing work”; therefore, only half of them can be counted toward Disks Queue
    • Average Disk Queue Length <= # (Disks in the array/2) *2
    • RAID 5: All of the disks are doing work in a RAID 5 set
    • Average Disk Queue Length <= # Disks in the array *2
    • Avg. Disk sec/Transfer: The number of seconds it takes to complete one disk I/O
    • Avg. Disk sec/Read: The average time, in seconds, of a read of data from the disk
    • Avg. Disk sec/Write: The average time, in seconds, of a write of data to the disk

      Note The last three counters in this list should consistently have values of approximately .020 (20 ms) or lower and should never exceed.050 (50 ms). The following are the thresholds that are documented in the SQL Server performance troubleshooting guide:

      • Less than 10 ms: very good
      • Between 10 – 20 ms: okay
      • Between 20 – 50 ms: slow, needs attention
      • Greater than 50 ms: serious I/O bottleneck
    • Disk Bytes/sec: The number of bytes being transferred to or from the disk per second
    • Disk Transfers/sec: The number of input and output operations per second (IOPS)

    When % Idle Time is low (10 percent or less), this means that the disk is fully utilized. In this case, the last two counters in this list (“Disk Bytes/sec” and “Disk Transfers/sec”) provide a good indication of the maximum throughput of the drive in bytes and in IOPS, respectively. The throughput of a SAN drive is highly variable, depending on the number of spindles, the speed of the drives, and the speed of the channel. The best bet is to check with the SAN vendor to find out how many bytes and IOPS the drive should support. If % Idle Time is low, and the values for these two counters do not meet the expected throughput of the drive, engage the SAN vendor to troubleshoot.

The following links provide deeper insight into troubleshooting SQL Server performance:

OpsMgr Performance counters

The following sections describe the performance counters that you can use to monitor and troubleshoot OpsMgr performance.

Gateway server role

  • Overall performance counters: These counters indicate the overall performance of the gateway:
    • Processor(_Total)\% Processor Time
    • Memory\% Committed Bytes In Use
    • Network Interface(*)\Bytes Total/sec
    • LogicalDisk(*)\% Idle Time
  • LogicalDisk(*)\Avg. Disk Queue LengthOpsMgr process generic performance counters: These counters indicate the overall performance of OpsMgr processes on the gateway:
    • Process(HealthService)\%Processor Time
    • Process(HealthService)\Private Bytes (depending on how many agents this gateway is managing, this number may vary and could be several hundred megabytes)
    • Process(HealthService)\Thread Count
    • Process(HealthService)\Virtual Bytes
    • Process(HealthService)\Working Set
    • Process(MonitoringHost*)\% Processor Time
    • Process(MonitoringHost*)\Private Bytes
    • Process(MonitoringHost*)\Thread Count
    • Process(MonitoringHost*)\Virtual Bytes
  • Process(MonitoringHost*)\Working SetOpsMgr specific performance counters: These counters are OpsMgr specific counters that indicate the performance of specific aspects of OpsMgr on the gateway:
    • Health Service\Workflow Count
    • Health Service Management Groups(*)\Active File Uploads: The number of file transfers that this gateway is handling. This represents the number of management pack files that are being uploaded to agents. If this value remains at a high level for a long time, and there is not much management pack importing at a given moment, these conditions may generate a problem that affects file transfer.
    • Health Service Management Groups(*)\Send Queue % Used: The size of persistent queue. If this value remains higher than 10 for a long time, and it does not drop, this indicates that the queue is backed up. This condition is cause by an overloaded OpsMgr system because the management server or database is too busy or is offline.
    • OpsMgr Connector\Bytes Received: The number of network bytes received by the gateway – i.e., the amount of incoming bytes before decompression.
    • OpsMgr Connector\Bytes Transmitted: The number network bytes sent by the gateway – i.e., the amount of outgoing bytes after compression.
    • OpsMgr Connector\Data Bytes Received: The number of data bytes received by the gateway – i.e., the amount of incoming data after decompression.
    • OpsMgr Connector\Data Bytes Transmitted: The number of data bytes sent by the gateway – i.e. the amount of outgoing data before compression.
    • OpsMgr Connector\Open Connections: The number of connections that are open on gateway. This number should be same as the number of agents or management servers that are directly connected to the gateway.

Management server role

Overall performance counters: These counters indicate the overall performance of the management server:

  • Processor(_Total)\% Processor Time
  • Memory\% Committed Bytes In Use
  • Network Interface(*)\Bytes Total/sec
  • LogicalDisk(*)\% Idle Time

LogicalDisk(*)\Avg. Disk Queue LengthOpsMgr process generic performance counters: These counters indicate the overall performance of OpsMgr processes on the management server:

  • Process(HealthService)\% Processor Time
  • Process(HealthService)\Private Bytes – Depending on how many agents this management server is managing, this number may vary, and it could be several hundred megabytes.
  • Process(HealthService)\Thread Count
  • Process(HealthService)\Virtual Bytes
  • Process(HealthService)\Working Set
  • Process(MonitoringHost*)\% Processor Time
  • Process(MonitoringHost*)\Private Bytes
  • Process(MonitoringHost*)\Thread Count
  • Process(MonitoringHost*)\Virtual Bytes

Process(MonitoringHost*)\Working SetOpsMgr specific performance counters: These counters are OpsMgr specific counters that indicate the performance of specifric aspects of OpsMgr on the management server:

  • Health Service\Workflow Count: The number of workflows that are running on this management server.
  • Health Service Management Groups(*)\Active File Uploads: The number of file transfers that this management server is handling. This represents the number of management pack files that are being uploaded to agents. If this value remains at a high level for a long time, and there is not much management pack importing at a given moment, these conditions may generate a problem that affects file transfer.
  • Health Service Management Groups(*)\Send Queue % Used: The size of the persistent queue. If this value remains higher than 10 for a long time, and it does not drop, this indicates that the queue is backed up. This condition is cause by an overloaded OpsMgr system because the OpsMgr system (for example, the root management server) is too busy or is offline.
  • Health Service Management Groups(*)\Bind Data Source Item Drop Rate: The number of data items that are dropped by the management server for database or data warehouse data collection write actions. When this counter value is not 0, the management server or database is overloaded because it can’t handle the incoming data item fast enough or because a data item burst is occurring. The dropped data items will be resent by agents. After the overload or burst situation is finished, these data items will be inserted into the database or into the data warehouse.
  • Health Service Management Groups(*)\Bind Data Source Item Incoming Rate: The number of data items received by the management server for database or data warehouse data collection write actions.
  • Health Service Management Groups(*)\Bind Data Source Item Post Rate: The number of data items that the management server wrote to the database or data warehouse for data collection write actions.
  • OpsMgr Connector\Bytes Received: The number of network bytes received by the management server – i.e., the size of incoming bytes before decompression.
  • OpsMgr Connector\Bytes Transmitted: The number of network bytes sent by the management server – i.e., the size of outgoing bytes after compression.
  • OpsMgr Connector\Data Bytes Received: The number of data bytes received by the management server – i.e., the size of incoming data after decompress)
  • OpsMgr Connector\Data Bytes Transmitted: The number of data bytes sent by the management server – i.e., the size of outgoing data before compression)
  • OpsMgr Connector\Open Connections: The number of connections open on management server. It should be same as the number of agents or root management server that are directly connected to it.
  • OpsMgr database Write Action Modules(*)\Avg. Batch Size: The number of a data items or batches that are eceived by database write action modules. If this number is 5,000, a data item burst is occurring.
  • OpsMgr DB Write Action Modules(*)\Avg. Processing Time: The number of seconds a database write action modules takes to insert a batch into database. If this number is often greater than 60, a database insertion performance issue is occurring.
  • OpsMgr DW Writer Module(*)\Avg. Batch Processing Time, ms: The number of milliseconds for data warehouse write action to insert a batch of data items into a data warehouse.
  • OpsMgr DW Writer Module(*)\Avg. Batch Size: The average number of data items or batches received by data warehouse write action modules.
  • OpsMgr DW Writer Module(*)\Batches/sec: The number of batches received by data warehouse write action modules per second.
  • OpsMgr DW Writer Module(*)\Data Items/sec: The number of data items received by data warehouse write action modules per second.
  • OpsMgr DW Writer Module(*)\Dropped Data Item Count: The number of data items dropped by data warehouse write action modules.
  • OpsMgr DW Writer Module(*)\Total Error Count: The number of errors that occurred in a data warehouse write action module.

Root management server role

Overall performance counters: These counters indicate the overall performance of the root management server:

  • Processor(_Total)\% Processor Time
  • Memory\% Committed Bytes In Use
  • Network Interface(*)\Bytes Total/sec
  • LogicalDisk(*)\% Idle Time

     

LogicalDisk(*)\Avg. Disk Queue LengthOpsMgr process generic performance counters: These counters indicate the overall performance of OpsMgr processes on the root management server:

  • Process(HealthService)\% Processor Time
  • Process(HealthService)\Private Bytes (Depending on how many agents this root management server is managing, this number may vary and could be several hundred Megabytes.)
  • Process(HealthService)\Thread Count
  • Process(HealthService)\Virtual Bytes
  • Process(HealthService)\Working Set
  • Process(MonitoringHost*)\% Processor Time
  • Process(MonitoringHost*)\Private Bytes
  • Process(MonitoringHost*)\Thread Count
  • Process(MonitoringHost*)\Virtual Bytes
  • Process(MonitoringHost*)\Working Set
  • Process(Microsoft.Mom.ConfigServiceHost)\% Processor Time
  • Process(Microsoft.Mom.ConfigServiceHost)\Private Bytes
  • Process(Microsoft.Mom.ConfigServiceHost)\Thread Count
  • Process(Microsoft.Mom.ConfigServiceHost)\Virtual Bytes
  • Process(Microsoft.Mom.ConfigServiceHost)\Working Set
  • Process(Microsoft.Mom.Sdk.ServiceHost)\% Processor Time
  • Process(Microsoft.Mom.Sdk.ServiceHost)\Private Bytes
  • Process(Microsoft.Mom.Sdk.ServiceHost)\Thread Count
  • Process(Microsoft.Mom.Sdk.ServiceHost)\Virtual Bytes

     

Process(Microsoft.Mom.Sdk.ServiceHost)\Working SetOpsMgr specific performance counters: These counters are OpsMgr specific counters that indicate the performance of specific aspects of OpsMgr on the root management server:

  • Health Service\Workflow Count: The number of workflows that are running on this root management server.
  • Health Service Management Groups(*)\Active File Uploads: The number of file transfers that this root management server is handling – i.e., configuration and management pack uploads to agents. If this value remains higher for a long time, and it does not drop, this indicates that not much discovery or management pack is being imported at the moment, and that there could be a problem in file transfer.
  • Health Service Management Groups(*)\Send Queue % Used: The size of the persistent queue.
  • Health Service Management Groups(*)\Bind Data Source Item Drop Rate: The number of data items dropped by the root management server for database or data warehouse data collection write actions. When this counter value is not 0, the root management server or database is overloaded because it can’t handle the incoming data item fast enough or because a data item burst is occurring. The dropped data items will be resent by agents. After the overloaded or burst situation is finished, these data items will be inserted into the database or into the data warehouse.
  • Health Service Management Groups(*)\Bind Data Source Item Incoming Rate: The number of data items received by the root management server for database or data warehouse data collection write actions.
  • Health Service Management Groups(*)\Bind Data Source Item Post Rate: The number of data items that the root management server wrote to the database or to the data warehouse for database or data warehouse data collection write actions.
  • OpsMgr Connector\Bytes Received: The number of network bytes received by the root management server – i.e., the size of incoming bytes before decompress.
  • OpsMgr Connector\Bytes Transmitted: The number of network bytes sent by the root management server – i.e., the size of outgoing bytes after compression.
  • OpsMgr Connector\Data Bytes Received: The number of data bytes received by the root management server – i.e., the size of incoming data after decompression.
  • OpsMgr Connector\Data Bytes Transmitted: The number of data bytes sent by the root management server – i.e., the size of outgoing data before compression.
  • OpsMgr Connector\Open Connections: The number of connections open on the root management server. It should be same as the number of agents or management servers that are directly connected to it.
  • OpsMgr Config Service\Number Of Active Requests: The number of configuration or management pack requests that are being processing by the Config service.
  • OpsMgr Config Service\Number Of Queued Requests: The number of queued config or management pack requests sent to the Config service. If it is high for a long time, the instance space or management pack space is changing too frequently.
  • OpsMgr SDK Service\Client Connections: The number of SDK connections.
  • OpsMgr DB Write Action Modules(*)\Avg. Batch Size: The number of a data items or batches that are received by database write action modules. If this number is 5,000, a data item burst is occurring.
  • OpsMgr DB Write Action Modules(*)\Avg. Processing Time: The number of seconds that a database write action modules takes to insert a batch into a database. If this number is often larger than 60, a database insertion performance issue is occurring.
  • OpsMgr DW Writer Module(*)\Avg. Batch Processing Time, ms: The number of milliseconds that it takes for a data warehouse write action to insert a batch of data items into a data warehouse.
  • OpsMgr DW Writer Module(*)\Avg. Batch Size: The average number of data items or batches that are received by data warehouse write action modules.
  • OpsMgr DW Writer Module(*)\Batches/sec: The number of batches received by data warehouse write action modules per second.
  • OpsMgr DW Writer Module(*)\Data Items/sec: The number of data items received by data warehouse write action modules per second.
  • OpsMgr DW Writer Module(*)\Dropped Data Item Count: The number of data items that are dropped by data warehouse write action modules)

OpsMgr DW Writer Module(*)\Total Error Count (This is number of errors happened in data warehouse write action modules.


Microsoft Exchange 2010 SP3 is released

Microsoft  Microsoft Exchange Server 2010 Service Pack 3 (SP3)                                        

 Earlier last year, Microsoft announced that Exchange 2010 Service Pack 3 would be coming in the first half of 2013. Later, we updated the timeframe to Q1 2013. Today, we’re pleased to announce the availability of Exchange Server 2010 Service Pack 3, which is ready to download.

Service Pack 3 is a fully slipstreamed version of Exchange 2010. The following new features and capabilities are included within SP3:

  • Coexistence with Exchange 2013:Customers who want to introduce Exchange Server 2013 into their existing Exchange 2010 infrastructure will need the coexistence changes shipping in SP3.NOTE: Exchange 2010 SP3 allows Exchange 2010 servers to coexist with Exchange 2013 CU1, which is also scheduled to be released in Q1 2013. Customers can test and validate this update in a representative lab environment prior to rolling out in their production environments as an important coexistence preparatory step before introducing Exchange Server 2013 CU1.
  • Support for Windows Server 2012: With SP3, you can install and deploy Exchange Server 2010 on computers that are running Windows Server 2012.
  • Support for Internet Explorer 10: With SP3, you can use IE10 to connect to Exchange 2010.
  • Customer Requested Fixes: All fixes contained within update rollups released before SP3 will also be contained within SP3. Details of our regular Exchange 2010 release rhythm can be found in Exchange 2010 Servicing.

In addition to the customer reported issues resolved in previous rollups, this service pack also resolves the issues that are described in the following Microsoft Knowledge Base (KB) articles:

Note: Some of the following KB articles may not be available at the time of publishing this post.

2552121 You cannot synchronize a mailbox by using an Exchange ActiveSync device in an Exchange Server 2010 environment

2729444 Mailboxes are quarantined after you install the Exchange Server 2010 SP2 version of the Exchange Server 2010 Management Pack

2778100 Long delay in receiving email messages by using Outlook in an Exchange Server 2010 environment

2779351 SCOM alert when the Test-PowerShellConnectivity cmdlet is executed in an Exchange Server 2010 organization

2784569 Slow performance when you search a GAL by using an EAS device in an Exchange Server 2010 environment

2796950 Microsoft.Exchange.Monitoring.exe process consumes excessive CPU resources when a SCOM server monitors Exchange Server 2010 Client Access servers

2800133 W3wp.exe process consumes excessive CPU and memory resources on an Exchange Client Access server after you apply Update Rollup 5 version 2 for Exchange Server 2010 SP2

2800346 Outlook freezes and high network load occurs when you apply retention policies to a mailbox in a mixed Exchange Server 2010 SP2 environment

2810617 Can’t install Exchange Server 2010 SP3 when you define a Windows PowerShell script execution policy in Group Policy

2787500 Declined meeting request is added back to your calendar after a delegate opens the request by using Outlook 2010

2797529 Email message delivery is delayed on a Blackberry mobile device after you install Update Rollup 4 for Exchange Server 2010 SP2

2800080 ErrorServerBusy response code when you synchronize an EWS-based application to a mailbox in an Exchange Server 2010 environment


The New Exchange Reaches RTM! [Exchange 15]

Today Microsoft reached an important milestone in the development of the new Exchange.

Moments ago, the Exchange engineering team signed off on the Release to Manufacturing (RTM) build. This milestone means the coding and testing phase of the project is complete and we are now focused on releasing the new Exchange via multiple distribution channels to our business customers. General availability is planned for the first quarter of 2013.

Microsoft has a number of programs that provide business customers with early access so they can begin testing, piloting and adopting Exchange within their organizations:

  • Microsoft will begin rolling out new capabilities to Office 365 Enterprise customers in our next service updates, starting in November through general availability.
  • Volume Licensing customers with Software Assurance will be able to download Exchange Server 2013 through the Volume Licensing Service Center by mid-November. These products will be available on the Volume Licensing price list on December 1.

Since announcing the Preview of the new Exchange back in July, the EHLO team has been actively blogging about the features and capabilities of the new Exchange. Microsoft is excited to start getting the finished product into the hands of our customers!

For those who are interested in learning more about the new Exchange, check out the series of posts that have been published over the past couple months:


Update Rollup 2 for Exchange 2010 Service Pack 2 (KB2661854)

Date Published: 16/04/2012

Microsoft has released the following update rollup 2 for Exchange Server 2010 SP2:

This update contains a number of customer-reported and internally found issues since the release of SP1 RU2. See KB2661854: Description of Update Rollup 2 for Exchange Server 2010 Service Pack 2 for more details.

Note: Some of the following KB articles may not be available at the time of publishing this post.

We would like to specifically call out the following fixes which are included in this release:

  • KB2696913 You cannot log on to Outlook Web App when a proxy is set up in an Exchange Server 2010 environment
  • KB2688667 High CPU in W3WP when processing recurrence items who fall on DST cutover
  • KB2592398 PR_INTERNET_MESSAGE_ID is the same on messages resent by Outlook
  • KB2630808 EwsAllowMacOutlook Setting Not Honored
  • KB2661277 Android/Iphones stuck with 451 during Cross forest proxy in datacenter
  • KB2678414  Contact name doesn’t display company if name fields are left blank

Note that this fix will not cause the CAS to CAS OWA proxying incompatibility with Exchange 2007 as discussed here. No additional updates are required on Exchange 2007 for proxying to work once Exchange 2010 SP2 RU2 is installed.

General Notes:

For DST Changes: http://www.microsoft.com/time.

Note for Forefront Protection for Exchange users  For those of you running Forefront Protection for Exchange, be sure you perform these important steps from the command line in the Forefront directory before and after this rollup’s installation process. Without these steps, Exchange services for Information Store and Transport will not start after you apply this update. Before installing the update, disable ForeFront by using this command: fscutility /disable. After installing the update, re-enable ForeFront by running fscutility /enable.


Update Rollup 6 for Exchange Server 2010 SP1 (KB2608646)

Date Published: 28/10/2011

Microsoft has released the following update rollup for Exchange Server 2010 SP1:

Update Rollup 6 for Exchange Server 2010 SP1 (KB2608646) Download the rollup here.

This update contains a number of customer-reported and internally found issues since the release of SP1. See ‘KB 2608646: Description of Update Rollup 6 for Exchange Server 2010 Service Pack 1′ for more details.

This update contains a number of customer reported and internally found issues since the release of RU5. In particular we would like to specifically call out the following fixes which are included in this release:

  • 2627769 Some time zones in OWA are not synchronized with Windows in an Exchange Server 2010 environment
  • 2528854 The Microsoft Exchange Mailbox Replication service crashes on a computer that has Exchange Server 2010 SP1 installed
  • 2544246 You receive a NRN of a meeting request 120 days later after the recipient accepted the request in an Exchange Server 2010 SP1 environment
  • 2616127 “0×80041606″ error code when you use Outlook in online mode to search for a keyword against a mailbox in an Exchange Server 2010 environment.
  • 2549183 “There are no objects to select” message when you try to use the EMC to specify a server to connect to in an Exchange Server 2010 SP1 environment

 

Availability of this update on Microsoft Update is planned for late November.

General Notes

An issue with management of RBAC roles when RU6 is partially deployed in the organization: Due to changes shipped in this update, certain warnings can be displayed when managing RBAC roles, if RU6 is not yet deployed to all servers in the organization. Please see the following KB article for more information:

Managing RBAC roles might display warnings or errors if Exchange 2010 SP1 RU6 is partially deployed in the organization
http://support.microsoft.com/kb/2638351

!Note for Forefront users:

For those of you running Forefront Protection for Exchange, before installing the update, stop all Forefront services.


Update Rollup 5 for Exchange Server 2010 Service Pack 1 (KB 2582113)

Date Published: 23/8/2011

Microsoft has released the following update rollup for Exchange Server 2010 SP1:

Update Rollup 5 for Exchange Server 2010 SP1 (KB2582113) Download the rollup here.

This update rollup is highly recommended for all Exchange Server 2010 customers.

For a list of changes that are included in this update rollup, see KB2582113.

 

Here is a list of the fixes included in update rollup 5 for Exchange 2010 SP1:

  • 2275156 (http://support.microsoft.com/kb/2275156/ ) The inline
    contents disposition is removed when you send a “Content-Disposition: inline”
    email message by using EWS in an Exchange Server 2010 environment
  • 2499044 (http://support.microsoft.com/kb/2499044/ ) You cannot save
    attachments in an email message by using OWA if the subject line contains
    special characters in an Exchange Server 2010 environment
  • 2509306 (http://support.microsoft.com/kb/2509306/ ) Journal reports
    are expired or lost when the Microsoft Exchange Transport service is restarted
    in an Exchange Server 2010 environment
  • 2514766 (http://support.microsoft.com/kb/2514766/ ) A RBAC role
    assignee can unexpectedly run the Add-ADPermission command on an Exchange Server
    2010 server that is outside the role assignment scope
  • 2529715 (http://support.microsoft.com/kb/2529715/ ) Slow network or
    replication issues after you change the number of virus scanning API threads in
    Microsoft Exchange Server 2010
  • 2536704 (http://support.microsoft.com/kb/2536704/ ) Mailbox users who
    are migrated by using ILM 2007 cannot use the Options menu in OWA in an Exchange
    Server 2010 environment
  • 2537094 (http://support.microsoft.com/kb/2537094/ ) French
    translation errors occur when you edit a response to a meeting request by using
    OWA in an Exchange Server 2010 SP1 environment
  • 2554604 (http://support.microsoft.com/kb/2554604/ ) A RBAC role
    assignee can unexpectedly manage certificates that are outside the role
    assignment scope in an Exchange Server 2010 environment
  • 2555800 (http://support.microsoft.com/kb/2555800/ ) You cannot use
    the GetItem operation in EWS to retrieve properties of an email message in an
    Exchange Server 2010 environment
  • 2555850 (http://support.microsoft.com/kb/2555850/ ) You cannot
    delete a mailbox folder that starts with a special character in its name by
    using Outlook in an Exchange Server 2010 environment
  • 2556096 (http://support.microsoft.com/kb/2556096/ ) The columns in
    the .csv logging file are not lined up correctly when you perform a discovery
    search on a mailbox in an Exchange Server 2010 environment
  • 2556107 (http://support.microsoft.com/kb/2556107/ ) The columns in
    the .csv logging file are not lined up correctly when you perform a discovery
    search on a mailbox in an Exchange Server 2010 environment
  • 2556133 (http://support.microsoft.com/kb/2556133/ ) A device that
    uses Exchange ActiveSync cannot access mailboxes in an Exchange Server 2010
    environment
  • 2556156 (http://support.microsoft.com/kb/2556156/ ) Extra.exe crashes
    when it performs RPC activity checks against an Exchange Server 2010 server
  • 2556352 (http://support.microsoft.com/kb/2556352/ ) “ChangeKey is
    required for this operation” error message in Outlook for Mac 2011 in an
    Exchange Server 2010 environment
  • 2556407 (http://support.microsoft.com/kb/2556407/ ) Certain
    client-only message rules do not take effect on email messages that are saved as
    drafts in an Exchange Server 2010 environment
  • 2559926 (http://support.microsoft.com/kb/2559926/ ) “There are no
    items to show in this view.” error message when you try to view a folder by
    using Outlook in an Exchange Server 2010 environment
  • 2572958 (http://support.microsoft.com/kb/2572958/ ) The
    “Test-OutlookConnectivity -Protocol HTTP” command fails with an HTTP 401 error
    in an Exchange Server 2010 environment

Note to Forefront users:

If you don’t disable Forefront before installing a rollup or service pack, and enable afterwards, you run the risk of Exchange related services not starting. You can disable Forefront by going to a command prompt and navigating to the Forefront directory and running FSCUtility /disable. To enable Forefront after installation of a UR or SP, run FSCUtility /enable.


An Update on Exchange Server 2010 SP1 Rollup Update 4

Microsoft has published this article to give you an update about Exchange 2010 SP1 Rollup 4.

————————————————————————————————–

The Exchange Sustained Engineering team recently made the decision to recall
the June 22, 2011 release of Exchange 2010 SP1 Rollup 4. This was not an action
we took lightly and we understand how disruptive this was to customers. We would
like to provide you with some details that will give you a deeper understanding
of what actually happened and, more importantly, what improvements we are making
to prevent this in the future.

 

     

    Q: What actually triggered the recall?

    A: While fixing a bug that prevented deleted public folders from being
    recovered, we exposed an untested set of conditions with the Outlook client.
    When moving or copying a folder, Outlook passes a flag on a remote procedure
    call that instructs the Information Store to open deleted items which haven’t
    been purged. Our fix inadvertently caused the RPC to skip all content that
    wasn’t marked for deletion because we were not expecting this flag on the call
    from Outlook on the copy and move operations.

       

      Q: Why didn’t you test this scenario?

      A: The short answer is we thought we did. We didn’t realize we missed a key
      interaction between Exchange and Outlook. The Exchange team has well over
      100,000 automated tests that we use to validate our product before we ship it.
      With the richness and number of scenarios and behaviors that Exchange supports,
      automated testing is the only scalable solution. We execute these tests in
      varying scenarios and conditions repeatedly before we release the software to
      our customers. We also supplement these tests with manual validation where
      necessary. The downside of our tests is that they primarily exercise the
      interfaces we expose and are designed around our specifications. They do test
      positive and negative conditions to catch unexpected behavior and we did execute
      numerous folder copy and move tests against the modified code which all passed.
      What we did not realize is that our tests were not emulating the procedure call
      as executed by Outlook.

         

        Q: Exchange has been around a while, why did this happen
        now?

        A: In Exchange 2010 we introduced a feature called RPC Client
        Access
        . This functionality is responsible for serving as the MAPI endpoint for
        Outlook clients. It allowed us to abstract client connections away from the
        Information Store (on Mailbox servers) and cause all Outlook clients to connect
        to the RPC Client Access service.

         

        As part of our investigation, we discovered that there was some specific code
        added to the Exchange 2003 Information Store to handle the procedure call from
        Outlook using the extra flag. This code was also carried forward into Exchange
        2007. But when the Exchange team added the RPC Client Access service to Exchange
        2010, that code was not incorporated into the RPC Client Access service because
        it was mistakenly believed to be legacy Outlook behavior that was no longer
        required. That, unfortunately, turned out not to be the case. The fact that we
        were not allowing a deleted public folder to be recovered was masking this new
        bug completely.

           

          Q: Are there other similar issues lurking in RPC Client
          Access?

          A: We do not believe so. The RPC Client Access functionality has been
          well-tested at scale and proven to be reliable for the millions of mailboxes
          hosted in on-premises deployment and in our own Office 365 and Live@EDU
          services.

             

            Q: What are you doing to prevent similar things from
            happening in the future?

            A: We have conducted a top-to-bottom review of the process we use to triage,
            develop and validate changes for Rollups and Service Packs and are making
            several improvements. We have changed the way we evaluate a customer requested
            fix to ensure that we more accurately identify the risk and usage scenarios that
            must be validated for a given fix. Recognizing the diversity of clients used to
            connect to Exchange, we are increasing our client driven test coverage to
            broaden the usage patterns validated prior to release. Most notably, we are
            working even closer with our counterparts in Outlook to use their automated test
            coverage against each of our releases as well. We are also looking to increase
            coverage for other clients as well.

            ————————————————————————————————–


            Update Rollup 4 (UR4) for Exchange Server 2010 SP1 removed from downloads

            Microsoft has discovered an issue impacting some customers who have installed
            Exchange 2010 SP1 RU4 into their Exchange environment and as a result have
            removed SP1 RU4 from Download Center and recommend customers do not proceed with
            any planned deployments of SP1 RU4.

             

            A small number of customers have reported when the Outlook client is used to
            move or copy a folder that subfolders and content for the moved folder are
            deleted. After investigation we have determined that the folder and item
            contents do not appear in the destination folder as expected but may be
            recovered from the Recoverable Items folder (what was previously known as
            Dumpster in older versions of Exchange) from the original folder. This behavior
            occurs due to a customer requested change in SP1 RU4 which allowed deleted
            Public Folders to be recovered. Outlook and Exchange are not correctly
            processing the folder move and copy operations causing the folder contents to
            appear to be deleted. OWA and Exchange Web Services clients are not affected by
            this change and process the folder move or copy actions correctly.

             

            Microsoft will be providing a fix in Exchange 2010 SP1 RU5, scheduled for release in
            August, which prevents the content loss in the target location during the
            move/copy process. In addition, we are also working with the Outlook development
            team to examine their code for proper behavior and identify if a fix is
            necessary from the client. If you have already deployed SP1 RU4, Microsoft recommend
            obtaining an Interim Update that resolves this issue.

             

            If you are a customer seeing this issue or would like to receive the Interim
            Update, please contact Microsoft Customer Support.  When contacting Microsoft Customer Support, you can
            reference KB 2581545 (please note that this article will be available at a later
            date on support.microsoft.com).  When installing the Interim Update, you need to
            install this on all Client Access and Mailbox servers that have SP1 RU4
            installed.


            Update Rollup 4 (UR4) for Exchange Server 2010 SP1 released

            Microsoft has released the following update rollup for Exchange Server 2010:

            • Update Rollup 4 for Exchange Server 2010 SP1 (2509910)

            If you’re running Exchange Server 2010 SP1, you need to apply Update Rollup 4 for Exchange 2010 to address the issues listed below.

            Remember, you only need to download the latest update for the version of Exchange that you’re running.

            Here is a list of the fixes included in update rollup 4:

            1. 2537099 “80040154″ error message when you try to configure external Client Access namespaces on an Exchange Server 2010 server
            2. 2536700 Outlook stops responding when you try to copy a folder to its subfolder by using Outlook in online mode in an Exchange Server 2010 SP1 environment
            3. 2536517 The Microsoft Exchange RPC Client Access service crashes intermittently on an Exchange Server 2010 server
            4. 2536494 It takes a long time to return results when you perform an Advanced Find search on a mailbox by using Outlook in online mode in an Exchange Server 2010 SP1 environment
            5. 2535648 The EMC takes a long time to open in an Exchange Server 2010 environment
            6. 2535130 Performance in Outlook or in OWA decreases when you use IMAP4 to access the contacts folder in an Exchange Server 2010 environment
            7. 2535105 There is no option to disable the Availability service in an Exchange Server 2010 environment
            8. 2533543 Event ID 2153 is logged on each database availability group member in an Exchange Server 2010 environment
            9. 2533538 You cannot look up the free/busy information of a user who is located on an Exchange Server 2010 organization from another Exchange Server 2010 organization
            10. 2533451 A RBAC role assignee can unexpectedly run the “Update-FileDistributionService” command on an Exchange Server 2010 server that is outside the role assignment scope
            11. 2519359 “Changes to the rule cannot be saved.” error message when you try to create a reply rule by using Outlook in an Exchange Server 2010 environment
            12. 2518850 You cannot receive email messages on a mobile phone by using ActiveSync in an Exchange Server 2010 environment
            13. 2517088 Public folder conflict resolution does not work as usual in an Exchange Server 2010 environment
            14. 2515259 “The items could not be copied.” error message when you run the Get-MailboxSearch cmdlet in an Exchange Server 2010 SP1 environment
            15. 2514709 Event ID 1001 after you successfully the install Exchange Server 2010 Unified Messaging server role
            16. 2514574 The Exchange RPC Client Access service crashes in an Exchange Server 2010 environment
            17. 2513723 The “New-MailboxImportRequest” cmdlet does not import all messages in a .pst file in the ANSI format in an Exchange Server 2010 environment
            18. 2512023 “GetUserOofSettings”, “SetUserOofSettings” and “GetUserAvailability” operations do not support Exchange Impersonation on the Exchange Server 2010 SP1 schema
            19. 2511897 You cannot send an email message to a mailbox for a brief period when you move the mailbox by using online move in an Exchange Server 2010 environment
            20. 2507463 You cannot move a mailbox that contains a corrupted Search Folder in an Exchange Server 2010 environment
            21. 2506820 The free/busy information does not display of a user whose mailbox is located on an Exchange Server 2003 server
            22. 2506049 The hierarchy of a new public folder database on an Exchange Server 2010 SP1 server is not replicated
            23. 2505968 The EdgeTransport.exe process crashes when you apply a rule that contains a bad email address in an Exchange Server 2010 environment
            24. 2504453 You cannot retrieve statistical information about a public folder by using the “Get-PublicFolderStatistics” cmdlet in an Exchange Server 2010 SP1 environment
            25. 2503337 Comments of your meeting response message is missing when you decline a meeting request in an Exchange Server 2010 environment
            26. 2501070 A RBAC role assignee can stop queue processing on an Exchange Server 2010 Hub Transport server or an Exchange Server 2010 Edge Transport server that is outside the role assignment scope
            27. 2500903 A space is missing in the subject line of a “Tentative” meeting response in an Exchange Server 2010 environment
            28. 2500648 “There are no items to show in this view.” error message when you try to view a folder in Outlook in an Exchange Server 2010 environment
            29. 2495167 You cannot recover a deleted public folder by using Outlook or MFCMAPI in an Exchange Server 2010 environment
            30. 2495010 The EdgeTransport.exe process consumes 100% CPU usage on an Exchange Server 2010 Edge Transport server or an Exchange Server 2007 Edge Transport server
            31. 2493393 You cannot use ECP to perform a wipe on a mobile phone in an Exchange Server 2010 SP1 environment
            32. 2492068 “The item cannot be saved to this folder.” error message when try to post an item to a mail-disabled public folder in an Exchange Server 2010 SP1 environment
            33. 2491354 You cannot view the free/busy information of users in a mixed Exchange Server 2007 and Exchange Server 2010 environment
            34. 2490134 A deferred delivery email message is not delivered by using Outlook 2007 in online mode in an Exchange Server 2010 environment
            35. 2489964 An update enables range 0x-0x1F characters in the display name of an Exchange Server 2010 user account
            36. 2489938 The “Connect-ExchangeServer” function does not change the target Exchange server in Exchange Server 2010
            37. 2489130 A RBAC role assignee can unexpectedly change mailbox properties that are outside the management role group scope in an Exchange Server 2010 environment
            38. 2488643 Outlook downloads duplicated POP3 email messages in an Exchange Server 2010 environment
            39. 2479188 The iCal parts of an email message contain invalid entries when they are sent from an Exchange Server 2003 mailbox to an Exchange Server 2010 mailbox
            40. 2477273 The DomainController parameter does not work when you use the “MoveMailbox.ps1″ script to move mailboxes in an Exchange Server 2010 environment
            41. 2471964 A NDR is sent to the sender when you move an email message to a personal folder file in an Exchange Server 2010 SP1 or a later version environment
            42. 2467619 A user who manages a distribution group cannot remove another user whose mailbox is disabled in an Exchange Server 2010 environment
            43. 2465292 “MAPI_E_FAILONEPROVIDER (0x8004011D)” error message when you access an Exchange Server 2010 mailbox by using a MAPI application
            44. 2446908 ESE event descriptions are missing in Event Viewer when the Eseutil utility is called on an Exchange Server 2010 SP1 server
            45. 2394554 An email message is not delivered if it contains unsupported encoded characters in the subject line in an Exchange Server 2010 environment
            46. 2491951 You cannot install Exchange Server 2010 SP1 if the NetBIOS domain name of the domain controller contains an ampersand (&) character
            47. 2507066 Administrator audit logging is disabled unexpectedly during an Exchange Server 2010 SP1 installation

            Download the rollup here.

            Note to Forefront users:

            If you don’t disable Forefront before installing a rollup or service pack, and enable afterwards, you run the risk of Exchange related services not starting. You can disable Forefront by going to a command prompt and navigating to the Forefront directory and running FSCUtility /disable. To enable Forefront after installation of a UR or SP, run FSCUtility /enable.


            Update!! – Exchange Server Deployment Assistant for Exchange 2007 and Office 365 Coexistence

            The Exchange Server Deployment Assistant has been enhanced to include support for configuring
            rich coexistence for organizations interested in maintaining some users
            on-premises with Exchange 2007 and some users hosted by Microsoft Office 365
            Beta for enterprises in the cloud. This scenario is in addition to the existing
            support for configuring rich coexistence between on-premises Exchange 2003
            organizations and Office 365 Beta. The coexistence information is only available
            in English at this time and additional rich coexistence support for Exchange
            2010 will be added the Deployment Assistant soon.

            Rich coexistence (“hybrid deployment”) offers organizations the ability to
            extend the feature-rich experience and administrative control they have with
            their existing on-premises Microsoft Exchange organization to the cloud. It
            provides the seamless look and feel of a single Exchange organization between an
            on-premises organization and a cloud-based organization. In addition,
            coexistence can serve as an intermediate step to moving completely to a
            cloud-based Exchange organization. This approach is different than the simple
            exchange migration (“cutover migration”) and staged Exchange migration options
            currently offered by Office 365 Beta outlined in E-Mail
            Migration Overview
            .

            http://technet.microsoft.com/en-us/exdeploy2010


          • About Me


            Rene

            My name is René van Maasakkers.
            I work as a Microsoft consultant in the Netherlands and this is my Messaging blog.


          • Archive

          • Rene van Maasakkers
            iDream theme by Templates Next | Powered by WordPress