The main contenders for this purpose seem to be Fedora Directory Server, OpenDS, and Apache Directory Server
Seeing as you don't even mention Samba, I assume you are trying to avoid drop-in replacements for AD?