Hi All
We are now coming to the end of our Exchange 2010 upgrade project and everything has gone pretty well so far.
From time to time our administrators are asked to complete searches in Exchange. This could be for a number of reasons e.g Data Protection request or a disciplinary query.
In Exchange 2007 we used to complete multi mailbox searches using a third party tool which would use MAPI to search every users' mailbox for items which matched our search terms. This process was extremely slow (90 hours to cover 2000 mailboxes) and not particular reliable.
In Exchange 2010 Microsoft released a feature called Multi-Mailbox search. This allowed administrators to search all of the mailboxes using the search index which is built into Exchange. This means the searches in Exchange 2010 are much much quicker. It also means that this search will look at attachments which is excellent, although does create far more results to review.
MSExchange Link for more information on Multi-Mailbox search
We did hit one issue whilst testing this search after all of the mailboxes had been migrated. The search would fail against a large number of mailboxes. It would retry 6 times and eventually fail and move onto the next folder. This was bad news as the rest of the folder might contain extremely important information.
Log Name: Application
Source: MSExchange MailboxSearchDate: 10/2/2013 2:05:26 PMEvent ID: 21000Task Category: (21)Level: ErrorKeywords: ClassicUser: N/AComputer: Server01.domain.comDescription:Batch copy for MailboxSearch 'Test 2' on mailbox 'Joe Blogs' was retried '5' times but failed on all instances. The last exception encountered was: 'Microsoft.Exchange.Data.Storage.PartialCompletionException: Move/Copy messages failed. ---> Microsoft.Mapi.MapiExceptionPartialCompletion: MapiExceptionPartialCompletion: Unable to copy message(s). (hr=0x40680, ec=0)Diagnostic context: Lid: 21750 Lid: 18122 StoreEc: 0x4DA Lid: 25510 Lid: 25270 StoreEc: 0x4DA Lid: 19830 Lid: 25290 StoreEc: 0x4DA Lid: 55847 EMSMDBPOOL.EcPoolSessionDoRpc called [length=45] Lid: 43559 EMSMDBPOOL.EcPoolSessionDoRpc returned [ec=0x0][length=118][latency=0] Lid: 23226 --- ROP Parse Start --- Lid: 31418 --- ROP Parse Done --- Lid: 16465 Lid: 24657 StoreEc: 0x4DA at Microsoft.Mapi.MapiExceptionHelper.ThrowIfErrorOrWarning(String message, Int32 hresult, Boolean allowWarnings, SafeExInterfaceHandle iUnknown, Exception innerException) at Microsoft.Mapi.MapiUnk.ThrowIfErrorOrWarning(String message, Int32 hr) at Microsoft.Mapi.MapiFolder.CopyMessages(CopyMessagesFlags flags, MapiFolder destFolder, Byte[][] entryIds) at Microsoft.Exchange.Data.Storage.CoreFolder.<>c__DisplayClass24.<InternalMoveOrCopyItems>b__1e() at Microsoft.Exchange.Data.Storage.CoreFolder.ExecuteMapiGroupOperationMethod(String operationAttempted, StoreObjectId[] sourceObjectIds, MapiGroupOperation mapiGroupOperationCall) --- End of inner exception stack trace --- at Microsoft.Exchange.Data.Storage.CoreFolder.ExecuteMapiGroupOperationMethod(String operationAttempted, StoreObjectId[] sourceObjectIds, MapiGroupOperation mapiGroupOperationCall) at Microsoft.Exchange.Data.Storage.CoreFolder.ExecuteMapiGroupOperationInternal(StoreObjectId[] sourceObjectIds, MapiGroupOperationInternalMethod method)'.
Source: MSExchange MailboxSearch
Date: 10/2/2013 2:05:26 PM
Event ID: 21003
Task Category: (21)
Level: Warning
Keywords: Classic
User: N/A
Computer: Server01.domain.com
Description:
Batch copy for MailboxSearch 'Test 2' on mailbox 'Joe Bloggs' failed on folder 'DPATEST' failed even after retries, and it will be skipped. The exception encountered was: 'Microsoft.Exchange.Data.Storage.PartialCompletionException: Move/Copy messages failed. ---> Microsoft.Mapi.MapiExceptionPartialCompletion: MapiExceptionPartialCompletion: Unable to copy message(s). (hr=0x40680, ec=0)
Diagnostic context:
Lid: 21750
Lid: 18122 StoreEc: 0x4DA
Lid: 25510
Lid: 25270 StoreEc: 0x4DA
Lid: 19830
Lid: 25290 StoreEc: 0x4DA
Lid: 55847 EMSMDBPOOL.EcPoolSessionDoRpc called [length=45]
Lid: 43559 EMSMDBPOOL.EcPoolSessionDoRpc returned [ec=0x0][length=118][latency=0]
Lid: 23226 --- ROP Parse Start ---
Lid: 31418 --- ROP Parse Done ---
Lid: 16465
Lid: 24657 StoreEc: 0x4DA
at Microsoft.Mapi.MapiExceptionHelper.ThrowIfErrorOrWarning(String message, Int32 hresult, Boolean allowWarnings, SafeExInterfaceHandle iUnknown, Exception innerException)
at Microsoft.Mapi.MapiUnk.ThrowIfErrorOrWarning(String message, Int32 hr)
at Microsoft.Mapi.MapiFolder.CopyMessages(CopyMessagesFlags flags, MapiFolder destFolder, Byte[][] entryIds)
at Microsoft.Exchange.Data.Storage.CoreFolder.<>c__DisplayClass24.<InternalMoveOrCopyItems>b__1e()
at Microsoft.Exchange.Data.Storage.CoreFolder.ExecuteMapiGroupOperationMethod(String operationAttempted, StoreObjectId[] sourceObjectIds, MapiGroupOperation mapiGroupOperationCall)
--- End of inner exception stack trace ---
at Microsoft.Exchange.Data.Storage.CoreFolder.ExecuteMapiGroupOperationMethod(String operationAttempted, StoreObjectId[] sourceObjectIds, MapiGroupOperation mapiGroupOperationCall)
at Microsoft.Exchange.Data.Storage.CoreFolder.ExecuteMapiGroupOperationInternal(StoreObjectId[] sourceObjectIds, MapiGroupOperationInternalMethod method)'.
Source: MSExchange Mid-Tier Storage
Date: 10/2/2013 2:05:33 PM
Event ID: 3003
Task Category: Discovery
Level: Error
Keywords: Classic
User: N/A
Computer: Server01.domain.com
Description:
Search 'SearchStatus\2e0e5de4-30a3-46db-8bff-0dd831a341c7' has following error:
An error occurred when searching Joe Bloggs. The message is 'Move/Copy messages failed.'.
Log Name: Application
Log Name: Application
Initially we couldn't work out why this was occurring, my colleague (John Manley), looked into this further and worked out that the Discovery Search mailbox did not have items over 100mb. This was a big find and once we had a common reason for the search failing it made the next steps easier.
Our company has a much smaller global transport limit configured in Exchange, but this is typically covering messages which were sent and received before this limit was imposed.
We tested the search with the Transport Limit off, but the same problem occurred. We moved the Discovery Search mailbox to its own DB but the problem still occurred.
Eventually we found that although the MaxSendSize and MaxReceiveSize for every user in the Exchange Organisation was unlimited, the Discovery Search mailbox was set to 100mb.
We set this mailbox to be unlimited. This change takes a number of hours to propagate through Exchange, but once replicated the searches begain to return results greater than 100mb......yay!
Hopefully this will save someone a few painstaking hours of trawling log files etc.
Sam
Google +
This is the only article I could find that offered a solution to this problem. Thank you!
ReplyDelete