Vista - WinSock quesion

Hello,
I have a VS DotNet 2005 application listening for tcp/ip packets on a
particular port (say port 5000). This works well as the app can collect
packets from remote devices and send a response to the remote device.
Unfortunately, I am having trouble initiating a send to a remote device and
having the return packet come back on the listener port (port 5000). The
issue seems to be that when I send a packet, the packet goes to the last
received packet's address which is not what I want, even when the remote
IPEndPoint is set to the correct device. The socket appears to ignore the
remote IPEndPoint. I was thinking about setting up the sending part of the
app on another port, but I am questioning this approach. What am I missing?
What should I do here?

Thanks,
Steven