This work considers the problem of learning with missing data. Two main classes of approaches are considered. The first class consists of sequential algorithms in which the missing values are first imputed by using an imputation method and then a learning algorithm is applied. This sequential approach is shown to be non-robust for certain scenarios. The second class of algorithms is more robust as they allow exploitation of side information (location of missing values) from the imputation block, which enhances the performance. In particular, an online updation scheme is proposed which is computationally efficient.