As bilirubin begins to build up, it deposits on the fatty tissue under the skin causing the baby's skin and whites of the baby's eyes to appear yellow.