I'm working on an alternate shell which has to run on top of Windows Explorer, without admin rights. This shell includes a tray, but the tray module won't work when Explorer is running because Explorer receives the tray icons' messages. (This isn't a problem with my shell's tray module - when I kill explorer.exe and run my shell, its tray module works as expected.)
I'm wondering if there's a way to safely pause Explorer while my shell is running? I don't want to kill Explorer because of the potential for messing up a user's system, and if my shell crashes, that leaves the user without a shell until they restart their system. If I understand correctly, suspending the explorer.exe process would have the same result if my shell goes down. I need a way of keeping Explorer inactive, but in a way that only holds while my shell is running. Is that even possible?
P. S. If there's a way of getting my app to receive tray messages without taking down Explorer, I'd be delighted to take that route instead.
EDIT: I did some more research and it looks like I could subclass the tray window and handle the necessary messages. That also looks relatively risky in terms of crashing Explorer, especially on 64-bit systems. Is anyone familiar with the procedure for doing this? (I'm familiar with subclassing, but not subclassing such things as Explorer. I'm also not familiar with the pitfalls of subclassing 64-bit apps.)
Hooking also appears to be an option, but I'd rather not set off any antivirus apps if I can avoid it, and hooking seems to be a major red flag for heuristic-based AVs.
Last edited by computerfreaker; April 17th, 2011 at 08:30 PM.
Under the operating conditions which I need to work with, my shell has to run on top of Explorer. I know it's not at all ideal, but it's what I have to work with.
I've found some measure of success with SetWindowsHookEx, using WH_GETMESSAGE, in a DLL; I just have a glitch or two to work out and it should be good to go. Whether it works on Vista/Seven remains to be seen, but it's looking pretty decent on my XP system.
I'm reasonably happy with the relative simplicity of SetWindowsHookEx, but the possibility of tripping AVs still bothers me. I'm thinking about switching to CreateRemoteThread/LoadLibrary just to avoid this, since rumor has it that API combo is unlikely to be flagged. Setting up CreateRemoteThread/LoadLibrary looks like a heck of a lot of work, though. Is it worth the extra time and aggravation?