break up large TCP/IP packets

[bkelly]

Environment: Windows XP Pro and Win 7, Visual Studio 2008, MFC, C++

I have an message application that fetches telemetry data from some hardware, processes it, and sends the data to a display application on a nearby computer. (The display cannot accept UDP.) The packet rate is always at least three per millisecond and sometimes ten. The message app extracts what I call messages from the telemetry stream and sends them out one message to a packet. Some of the messages sent out are longer than the maximum size (MTU). I presume, pretty sure, the OS and Winsock breaks them up into multiple packets and re-assembles them before delivery to the display application.

The application knows which packets will be too large. Because of the nature of the messages it would not be real difficult to create two or more packets for the really long messages and send them out separately. The long messages may contain over 2000 parameters and the protocol between the message app and the display app requires eight bytes per parameter plus standard TCP/IP overhead. That is more than 16,000 bytes for a single message.

Both computers have dual CPUs with quad core each, and there are two gigabyte switches between the two, one on the “telemetry” side of the room and one on the “server” side of the room. With one app running the network usage as seen by either computer is about 2.5% so no sweat on bandwidth.

It would take a moderate programming effort, but once running it would be almost no impact during run time. For the large packets there would be a short for loop calling the send function as needed.

Sometimes there will be two or more copies of my application output data from different streams at one time. Each will be sending from three to ten packets per millisecond.

Question:
Would it be worth the effort to break up the larger packets into smaller ones so Winsock and the OS does not have to do this?

[Alex F]

Are you talking about IP or UDP?

[Igor Vartanov]

No matter what protocol you're talking about, do you really have any issue? From the "many characters" post I cannot see any problem clearly stated. So, what is your reason to change anything?

[Skizmo]

Some of the messages sent out are longer than the maximum size (MTU).

You don't have to worry about the MTU size... the network stack is doing that work.

Would it be worth the effort to break up the larger packets into smaller ones so Winsock and the OS does not have to do this?

No.

[bkelly]

No.

We are not having problems at the moment, but am concerned that when we transition to multiple streams, we will. Thanks for the reply.

[MikeAThon]

Question:
Would it be worth the effort to break up the larger packets into smaller ones so Winsock and the OS does not have to do this?

No. TCP does a terrific job on packetizing, and the algorithms have been fine-tuned for years based on years of research. You would not be able to do as good a job as TCP.

In fact, if you are concerned about future capabilities for expansion, you might even consider exactly the oppposite: Combine many messages into each call to send(), and also increase SO_SNDBUF and SO_RCVBUF on both the sending and receiving sides.

[bkelly]

Hello MikeAThon,

The concept that TCP does a good job packetizing make great sense. The Microsoft people has been dealing with that for quite a long time and should have it down by now.

Combining messages is an interesting thought. There are two fixed type messages that are send out once per millisecond that could be combined without two much effort. There are a couple of gotchyas, but it could be done with moderate effort and complexity.

Then there are what I call messages. Embedded in the telemetry stream are messages that arrive in indeterminate order and are of varying length. Depending on the phase of operations the source device throws in what ever messages it deems needed. Right now, for each message there is a fixed packet. All the overhead is set up during a startup configuration phase (a little more than half the total volume of bytes is overhead) and only the data values change during run time.

I suppose I could create a master structure them memcpy several structures together. but that will take quite a bit of code adding both complexity and additional run time. Right now the code is rather efficient and handing the data off to the TCP_Manager class quite quickly. I just don't know if it would be worth the time to combine messages. But it could be.

Things appear to be running OK right now. I suspect that if I start making that change it might cross the line from being efficient and into premature optimization. That can be a broad fuzzy line.

Thanks for taking the time to discuss this with me.