Astronomy is entering an era of data-driven discovery, due in part to modern
machine learning (ML) techniques enabling powerful new ways to interpret
observations. This shift in our scientific approach requires us to consider
whether we can trust the black box. Here, we overview methods for an
often-overlooked step in the development of ML models: building community trust
in the algorithms. Trust is an essential ingredient not just for creating more
robust data analysis techniques, but also for building confidence within the
astronomy community to embrace machine learning methods and results.