sendto sending UDP packets twice

[Peter Robertson]

I have a Windows 7 application that uses sendto to send UDP packets to a DSP.

Fairly often, the first packet or the first two packets being send out are duplicated. I have confirmed that for each pair of packets,
I am executing a single sento call.

WireShark shows them being sent out twice, the only differences being the header and UDP checksums:
26 14.006331000 192.168.0.4 192.168.0.10 UDP 62 Yes Source port: 9 Destination port: 9
27 14.006637000 192.168.0.4 192.168.0.10 UDP 62 Yes Source port: 9 Destination port: 9
28 14.011313000 192.168.0.4 192.168.0.10 UDP 94 Yes Source port: 9 Destination port: 9
29 14.011392000 192.168.0.4 192.168.0.10 UDP 94 Yes Source port: 9 Destination port: 9

The short time intervals between the original packet and its copy suggest that this is happening within Windows.

Any ideas what may be happening here?

[Peter Robertson]

Additional information:

I have now got it into a state where every call to sendto results in two identical packets being issued (except the final one in the set where extra zeros are added to the ehnd).

[OReubens]

this isn't unusual for UDP since UDP has no flow control.

you can get duplicate packes for several reasons.
- the origin is multihomed and a packet is sent out from each source address on the origin (this indicates a flaw in your routing, though it's possible it's unsolvable, or may even be intended)
- depending on setup, you could be getting a packet over IPV4 and another over IPV6 (this indicates a flaw in your setup)
- there may be multiple paths from source to target, indicating loops in your routing, again, this is potentially a flaw, or intended.
- ...

[Peter Robertson]

Thank you for your reply, but I don't see what flow control has to do with it.

I am sending the packets, and I send them only once each. The connection to the DSP is direct; there are no routers in the way.
There is only one ARP mapping between the IP address being used and the device's MAC address.

[ahoodin]

How do you *know* the data being sampled and sent just has simply not changed yet?

[Peter Robertson]

How do you *know* the data being sampled and sent just has simply not changed yet?

First, no data are being sampled; the contents of the packets being sent come from a file on the PC of fixed and known contents.
Second, I add a sequence number to each packet being sent; Wireshark shows that number going up in every other packet leaving the PC.

[OReubens]

As I said, UDP has no flow control. so on the reciving end you have to be prepared to accept duplicates, handle missing packets and packets arriving out of order.

This is why TCP was developed, it's basically UDP with flow control, making sure that from the connection POV, you get a continuous, ordered stream. But that comes at a price ofc, that flow control takes CPU effort so TCP is slower.

there are many reasons for duplicate packets, particularly in UDP unicast. This is how UDP works, nothing much you can do, it even happens in a perfecly correctly configured network, and can happen a lot for various badly configured network issues.

https://wiki.wireshark.org/DuplicatePackets

[Peter Robertson]

As I said, UDP has no flow control. so on the reciving end you have to be prepared to accept duplicates, handle missing packets and packets arriving out of order.

I find it hard to believe that on a network of two processors, directly connected with neither end doing anything except the process in hand, a single call to sendto can validly send the same packet twice.

[ahoodin]

Just throw out the second packet, you said you have a sequence number, so you know that packet was already processed. Its either that or better yet, you can implement modbus over UDP, or add some other protocol overtop like SCTP over UDP. OReubens is right about this. UDP is the wild wild west of uncontrolled network communications, raw speed and throughput like a wild mustang. Great if you want a fast stream of data such as a live video feed, but not so great if you want flow control and other features, since it is up to you to implement them. UDP definitely has its uses, but without work, precision isn't one of them.

Have a look at SCTP, there is stuff already out there that you can bolt on. Link to your project and run with it.

[Andy(P)]

I'm encountering exactly the same problem in a very similar setup.
A point to point ethernet cable from the laptop to the embedded platform (Zynq 7030).
Python is being use to send/receive message on the PC, wireshark on the PC shows all messages from PC --> Embedded platform are duplicated (identical payload, between 2us and 15us difference in time stamps).
As the packet rate and size goes in, this is going to be a problem for given the data rates, so I'd rather not just 'discard' the unwanted duplicates.

with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
s.bind(('', 5704))
while(more_to_send == TRUE)
<code to populate bytes_to_send>
s.sendto(bytes_to_send, (HOST, UDP_PORT))
data = s.recv(1024)

43204 80.146000 192.168.1.20 192.168.1.10 UDP 566 5704 → 5604 Len=524 (payload)
43205 80.146003 192.168.1.20 192.168.1.10 UDP 566 5704 → 5604 Len=524 (duplicate at payload time + 3us)
43206 80.149112 192.168.1.10 192.168.1.20 UDP 48 5604 → 5704 Len=6 (ack)
43207 80.149306 192.168.1.20 192.168.1.10 UDP 566 5704 → 5604 Len=524 (payload)
43208 80.149311 192.168.1.20 192.168.1.10 UDP 566 5704 → 5604 Len=524 (duplicate at payload time +5us)

Was there an conclusion from Peter Robertson?

Andy

[2kaud]

Yes - See OReubens posts #3, #7. Your UDP receive code has to allow for duplicate and missing packets. This happens all the time with UDP. if this causes a problem, then you'll need to use TCP.

[Andy(P)]

Thanks 2kaud. I guess I'm trying to understand why python would send two identical packets back to back, down the point to point link to the embedded board. The concern is not the ability to cope with it (apart from the used bandwidth), but understanding why it should happen. I'm erring towards thinking it is a python bug.