[RESOLVED] WCF large collections

[dannystommen]

A while ago I posted a threadw about WCF client configuration (http://www.codeguru.com/forum/showthread.php?t=464674)

At that moment I needed to retrieve a list of 480 object from the WCF host. After editing the client's app.config, everything worked fine.

Now I have a different list (of the same type) which has 1800 object in it. Now I get this CommunicationException everytime again. When I reduce this list to 500 objects, it works fine, so somehow, the list is to big.

I changed the binding to very large numbers, but still I can't retrieve the list.
The binding look like next:

                <binding name="UserTcpBinding" closeTimeout="00:01:00" openTimeout="00:01:00"
                    receiveTimeout="00:10:00" sendTimeout="00:01:00" transactionFlow="false"
                    transferMode="Buffered" transactionProtocol="OleTransactions"
                    hostNameComparisonMode="StrongWildcard" listenBacklog="10"
                    maxBufferPoolSize="50000000" maxBufferSize="50000000" maxConnections="10"
                    maxReceivedMessageSize="50000000">
                    <readerQuotas maxDepth="50000000" maxStringContentLength="50000000" maxArrayLength="50000000"
                        maxBytesPerRead="50000000" maxNameTableCharCount="50000000" />
                    <reliableSession ordered="true" inactivityTimeout="00:10:00"
                        enabled="false" />
                    <security mode="Transport">
                        <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
                        <message clientCredentialType="Windows" />
                    </security>
                </binding>

Is there anyway to resolve this by modifying the app.config. Or do I need to return the list by a callback function in several times? (so first send back the first 500 object of the list, then the 2nd 500 and so on)

[wildfrog]

What kind of CommunicationException? What's its inner exception? Message?

- petter

[Arjay]

To help debug, include exception details in the service.

<behavior name="myBehavior">
          <serviceMetadata httpGetEnabled="true" />
          <serviceDebug includeExceptionDetailInFaults="true" />
</behavior>

[dannystommen]

Arjay: I have includeExceptionDetailInFaults set to true

Wildfrog: it's a CommunicationObjectFaultedException
- InnerException: null
- Message: "The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.
- Source: mscorlib
- Static members: Non-public Members: _COMPlusExceptionCode = -532459699

[Arjay]

Try searching google for "COMPlusExceptionCode 0xE0434F4D" to see if that would give you any ideas.

[dannystommen]

I googled more then half an hour now, but I can not find a good answer.

The only thing I think it has to with it, is serialization/deserialization of the collection. But I don't know if that is correct.
And if it is correct, what is causing this and how to solve it?

[Arjay]

I'm not sure if you did this already, but you need to increase the default on the service (as well as the client).

<bindings>
<wsHttpBinding>
<binding name="wsHttpBindingConfig" maxReceivedMessageSize ="50000000" maxBufferPoolSize="50000000" >
<readerQuotas maxArrayLength="50000000" />
</binding>
</wsHttpBinding>
</bindings>

[dannystommen]

First I did this only at the client, but after increasing the default on the service I still get the same error;

Client app.config

                <binding name="UserTcpBinding" closeTimeout="00:01:00" openTimeout="00:01:00"
                    receiveTimeout="00:10:00" sendTimeout="00:01:00" transactionFlow="false"
                    transferMode="Buffered" transactionProtocol="OleTransactions"
                    hostNameComparisonMode="StrongWildcard" listenBacklog="10"
                    maxBufferPoolSize="50000000" maxBufferSize="50000000" maxConnections="10"
                    maxReceivedMessageSize="50000000">
                    <readerQuotas maxDepth="32" maxStringContentLength="50000000"
                        maxArrayLength="50000000" maxBytesPerRead="50000000" maxNameTableCharCount="50000000" />
                    <reliableSession ordered="true" inactivityTimeout="00:10:00"
                        enabled="false" />
                    <security mode="Transport">
                        <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
                        <message clientCredentialType="Windows" />
                    </security>
                </binding>

            <endpoint address="net.tcp://tcwork11:10001/UserService/service"
                binding="netTcpBinding" bindingConfiguration="UserTcpBinding"
                contract="srUserService.UserService" name="UserTcpBinding">
                <identity>
                    <userPrincipalName value="tcwork11\Danny" />
                </identity>
            </endpoint>

Service app.config

  <system.serviceModel>
    <bindings>
      <netTcpBinding>
        <binding name="MaxBinding" maxReceivedMessageSize="50000000" maxBufferPoolSize="50000000" maxBufferSize="50000000">
          <readerQuotas maxArrayLength="50000000" maxStringContentLength="50000000" maxBytesPerRead="50000000" maxNameTableCharCount="50000000"/>
        </binding>
      </netTcpBinding>
    </bindings>
    <behaviors>
       //some code here
    </behaviors>
    <services>
      <service behaviorConfiguration="UserBehavior" name="YieldManagerPlus.Service.Services.sUser.UserService">
        <endpoint address="service" binding="netTcpBinding" bindingConfiguration="MaxBinding" contract="YieldManagerPlus.Service.Services.sUser.IUserService" name="UserTcpBinding" />

     //some code here

[Arjay]

Shouldn't behaviorConfiguration be set to MaxBinding instead of UserBehavior?

[dannystommen]

No, that gives a ConfigurationErrorsException: There is no service behavior named 'MaxBinding'. (because it's not an behavior but a binding)

[boudino]

Don't forget, that if a communication exception has occured, the channel remain in faulted state and you cannot use it any more, you have to create a new channel.

[dannystommen]

Yes I know that, but I still don't know why this exception is thrown.

[dannystommen]

Finally resolved.

Some gave me the advice of turning on WCF Tracing (http://msdn.microsoft.com/en-us/library/ms732023.aspx).

I traced the client, but nothing special to find. Just the same exception as already was thrown.

Also traced the service. When the service is running, it does not throw an execption, but the trace did log an exception:
There was an error while trying to serialize parameter YieldManagerPlus.Service.Services.sUser:GetAllAPIentityResult. The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota. '. Please see InnerException for more details.

I added into client app.config:

<behaviors>
  <endpointBehaviors>
    <behavior name="MaxBehavior">
      <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
    </behavior>
  </endpointBehaviors>
</behaviors>

Now it works fine!


Only thing that I don't understand, why does the trace log that exception, but the exception is never thrown at the service application?

[TheCPUWizard]

I heate repeating myself...but...

Making MAJOR increases to the configuration values for WCF communications is very likely to create subtle problems at some point in the future.

Services should (almost always) be configuraed to operate on messages that are within (or at least close) to these limits.

In many cases, the best pattern is to have the server perform multiple transfers using a "reverse channel". This provides:

1) Lower memory resource utilization for both the server AND the client.

2) Faster processing, as each "chunk" can be processed while the next one is still "in-flight".

Of the 20+ enterprise class WCF solutions I have deployed, I have followed the practice of using config files with the limits set to (approx) 80% of their default values. This allows me to implement the architecture so that the production systems can run with the default values.

[boudino]

Can you explain the term "reverse channel", please? Everything I googled out refered to telecommunications, but it says nothing about software usage the term.