Binary Compatability

I am having a problem creating some components with binary compatability.

The components have the compatable version set and the compatable file stored in a different directory.

Sometimes when I make a change to the component and recompile it, it causes the main application to fail with an automation error, even though there were no "you are breaking binary compatability" warnings when the component was rebuilt.

Recompiling the main app fixes the problem, but I don't see why I should have to be doing this.

This is under VB6 Service Pack 3

Any ideas?