How to display a window on top without having it always top most?

I have a window (let's call this window "A") that always needs to be displayed on top of the main application window when "A" is first displayed in the application.
But after "A" is displayed we want to be able to select the main application window so that the main application window can be displayed on top of "A".

I've tried setting the owner window as the main application window but "A" is always displayed as top most all the time.

If I don't set the owner window and TopMost = false for window "A", then sometimes the main application window is displayed on top of "A" when "A" is first displayed which is also not what we want.

What do I do?
Thanks!