I'm working on a UDP game-server for an xna game (no, i'm not interested in using the networking tools build in to xna). I have it working but i need ways to improve it's performance.

currently i have my data sorted like this:
class clientInfo
        public EndPoint endPoint; // client identifier, since udp is connectionless

        public int lastPacketReceived = 0; // sequence number of last packet received
        public int lastPacketSend = 0; // sequence number of last packet send
        public List<packetInfo> sendPackets = new List<packetInfo>();
    class packetInfo
        public int sequenceNumber; // the packets equence number
        public long sentTicks; // time when the packets was send
        public int retryCount; // how many times the packet was resend
        public byte[] packet; // the data in the packet
I have a a list<clientInfo> that represents my clients.

The main thread of my server listens for data on a udp port and pases the received data to a thread in the ThreadPool where it get's processed. I also have another tread in the ThreadPool that handles the acknowledgement of packets that where send, resend packets when needed and drops clients when it has to try to many times.

The problem with this structure is that i have to lock almost the entire threads because they are both working on the same list of clients and the same lists of packets (connecting and disconnecting clients, adding end removing send packets, increasing retry counts, etc..). I guess this makes the threating almost useless... How can i fix this? Is there a better way to organize the data to fix this problem or do i need to manage the threads differently?

Another problem is that when there are many clients the server seems to start dropping packets. Is there a better way of listening for packets? I read somewhere that there may be a period between the EndReceiveFrom and the next BeginReceiveFrom when the program isn't listening. Maybe i'l have to make multiple threats listen for new data?

I hope you guy's can help me with this, it seems to be rather hard to find information on this topic. If you need more info to help me i'll be happy to share. I didn't post the entire code because it's rather long a little messy ATM .

thanks in advance,