Knowledge Bank

We propose a novel framework for semantically segmenting images at the pixel-level given a dataset labeled only at the image-level. The intention of this model is to remove the expensive, time consuming, and unreliable process of densely labeling image datasets at the pixel-level. To accomplish this, our algorithm lays a framework to mesh techniques from unsupervised learning with the same deep convolutional neural network architectures that produce state-of-the-art results on fully-supervised datasets. The first pivotal contribution that separates our proposed algorithm from existing methods is that we avoid hallucinating a per pixel ground truth. We achieve this by maintaining a per pixel confidence distribution across classes and leveraging an expectation maximization framework to optimize these distributions using the image-level labels. Secondly, we propose a dataset score metric to measure how a tractable a given dataset is for the weakly supervised setting. We demonstrate that our proposed algorithm allows us to accurately segment high entropy problems typically intractable for weak supervision.