In this case, why in your opinion hasn't it been more popular before?
I've wondered the same thing. However, lots of features of HTTP, HTML, CSS, and other technologies were not heavily used even when browsers supported them fully. Simple things like CSS-based drop-down menus was a brilliant new thing a few years ago, but CSS has been around since IE 3.0.
I guess the whole XmlHttpRequest architecture was built, not advertised, and it just took a while for developers to use it for all its worth.
In my opinion it wasn't popular before because it is not essential like, for example, pop-up windows.
It is not very easy to use Ajax correctly and many developers misuse it.
Also, in my opinion, Ajax is not going to spread much more than it already is, because it is still not crucial and because most "developers" lack of sufficient knowledge to implement Ajax decently, so they prefer to think it is alien technology discovered by NASA.
I use Ajax successfully in my CMS administration area to enhance usability and if you look at the 4.63kb ajax.js file i use, it's very simple.
I disagree about AJAX being nonessential. The problem with a lot of web sites built even today is that they refresh the whole page to change something relatively small. Whether CGI, ASP, or ASP.NET, this still happens. Why? Because it's convenient to the developer.
In the future, it'll be easier to use AJAX to the point where you can update only the part of the screen you want, provided the browser supports the action. It'll become essential when network bandwidth and CPU load are negatively impacted by web applications that want to do excessive postbacks.