It is what setup programs was designed for. Create a setup which will include your exe, dll, other files needed for your program.