Is covariance and contravariance applicable only to delegates in C #?
Not really; language-level variance can also be applied to interfaces, for example IEnumerable<out T>
(similarly inalso good).
It should also be noted that arrays of reference types are also covariant:
string[] orig = {"abc","def"};
object[] sameArray = orig;
Can we have covariance and contravariance in normal class hierarchies?
No; this does not apply to classes / structures (although you can, of course, implement a covariant interface and force this interface).