The authors gave a presentation at an Institute of Medicine Roundtable on Health Literacy workshop titled “Health Literacy and Palliative Care” in July 2015. The presentations highlighted the experience of the health care system for those who need palliative care and the difficulty in accessing it. The authors believe the take-home message from the day was that the health literacy problem in our health system resides within the health care system. The system’s inability to find out, understand, and respond to what matters most to patients and their families is exemplified in Bev’s moving and infuriating story of her husband’s experience after a diagnosis of pancreatic cancer in upstate New York. Joe Alves’s diagnosis of inoperable metastatic cancer brought the couple into a world with which they were not familiar and had little time or energy to learn to navigate. The saddest thing about Bev and Joe’s story, as told the shaded boxes below, is how typical it is. This commentary is offered to point to the system changes that are necessary.