So one of our customer's SharePoint portal goes live. Everyone is happy and there are celebrations all around until an error crops up in production environment which had never occurred in any of the 3 environments (developer, integration, QA).
One of the core functionality of the SharePoint Portal; a custom webpart which places an order for certain product is failing inconsistently. The point of failure was the line of code which was submitting an order on the 3rd party web service consumed by our webpart. The stack trace was as follows
System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure. at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Threading.ExecutionContext.runTryCode(Object userData) at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result) at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.ConnectStream.WriteHeaders(Boolean async) --- End of inner exception stack trace --- at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request) at System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request) at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
After further diving through the event logs and SharePoint ULS logs we narrowed down to the actual problem as logged in the ULS logs and also in event logs below.
Note: I have removed the details of the actual certificate in the above stack trace
This made me to do some research and stumbling upon quite a few blogs; the most interesting one as below
http://jeroenvanree.wordpress.com/2012/01/16/event-id-8311-the-root-of-the-certificate-chain-is-not-a-trusted-root-authority/
The above blog did explain the issue & the resolution. Which we did follow but the errors were still coming. It was only after further analysis that I realized that the 3rd party webservice consumed by the webpart has a chain of certificates assigned like screenshot on the right.
While reading Jeroen's blog above I interpret that I need to only upload the Primary root cert and we are taken care of but it is also required for us to upload the entire chain excluding the first one in the chain (in this case webservice cert).
Once the intermediate cert was uploaded in the Manage Trust store of SharePoint everything worked as a charm with no run time exceptions and nothing logged in Event / ULS logs.
I decided to write a post on this to make everyone aware that don't just upload the primary root but you will need to upload all the cert in the chain excluding the first one.
P.S. This would have been very obvious error and easy analysis if all my environments consistently threw System.Net.WebException. I am still to figure out why only certain web front ends are throwing the error while other execute the code without returning run time error but log in the Event Logs.
One of the core functionality of the SharePoint Portal; a custom webpart which places an order for certain product is failing inconsistently. The point of failure was the line of code which was submitting an order on the 3rd party web service consumed by our webpart. The stack trace was as follows
System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure. at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Threading.ExecutionContext.runTryCode(Object userData) at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result) at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.ConnectStream.WriteHeaders(Boolean async) --- End of inner exception stack trace --- at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request) at System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request) at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
After further diving through the event logs and SharePoint ULS logs we narrowed down to the actual problem as logged in the ULS logs and also in event logs below.
An operation failed because the following certificate has
validation errors:\n\nSubject Name: CN={}, OU={}, OU={}, OU={},
O={}\nIssuer Name: CN={} OU={}, O={}, C=US\nThumbprint: {}\n\nErrors:\n\n The root of the certificate
chain is not a trusted root authority
Note: I have removed the details of the actual certificate in the above stack trace
This made me to do some research and stumbling upon quite a few blogs; the most interesting one as below
http://jeroenvanree.wordpress.com/2012/01/16/event-id-8311-the-root-of-the-certificate-chain-is-not-a-trusted-root-authority/
The above blog did explain the issue & the resolution. Which we did follow but the errors were still coming. It was only after further analysis that I realized that the 3rd party webservice consumed by the webpart has a chain of certificates assigned like screenshot on the right. While reading Jeroen's blog above I interpret that I need to only upload the Primary root cert and we are taken care of but it is also required for us to upload the entire chain excluding the first one in the chain (in this case webservice cert).
Once the intermediate cert was uploaded in the Manage Trust store of SharePoint everything worked as a charm with no run time exceptions and nothing logged in Event / ULS logs.
I decided to write a post on this to make everyone aware that don't just upload the primary root but you will need to upload all the cert in the chain excluding the first one.
P.S. This would have been very obvious error and easy analysis if all my environments consistently threw System.Net.WebException. I am still to figure out why only certain web front ends are throwing the error while other execute the code without returning run time error but log in the Event Logs.
No comments:
Post a Comment