You might look into custom file compression - you create a special type of file compression that only your program knows the pattern of... (encryption)

Something along the lines of simply changing the file size by removing certain characters in a certain order - and replacing them with your custom file handler to access it...

This is just speculation because I have no idea how to do it - but I know that it is plausible. I believe that would solve your dilemma of users simply associating the file with the necessary program to open the protected file with just a different extension.

If it was too easy then everyone would do it.