Not the idea, but implementing it so that it actually makes sense to use it. At least in my head. Sure an animal is the base, dog, cow, rhino are the derived classes. But to me the part I hard time getting was what the heck can be really basic piece(s) of data in an animal that the derived classes can be built on. Things like that caught me and still do.
Perhaps a different example would make things clearer. Consider a Shape base class that has an area() virtual function, and two separate derived classes, Circle and Rectangle. These two types of Shapes need to use different equations to compute their area(), but both of them have a concept of area. So if you wanted to write a function that needed the area of a shape, but didn't care what kind of shape it was other than that, you only need to write one function (taking a Shape* parameter), instead of needing to write two separate functions (one for Circle and one for Rectangle) which are largely identical except for the type.
Incidentally, the above inheritance example could be re-posed as a template example without too much trouble. The major difference between inheritance and templates is: If the actual type of an object (eg, Rectangle or Circle) can be known at compile time, use templates; if it cannot be known until runtime, use inheritance.