essentially... yes. Although it doesn't necessarily need to be on a separate thread, it is possible to implement the webserver portion on your main GUI thread without too much problems. and without causing the UI to hang/stutter.
If those apps can service webpages accessible using a webbrowser, than that is what they do. There are quite a few technologies to achieve this though, from accessing raw sockets, to class libraries, to high level libraries such as WinHttp and the HTTP server api.