Therapeutic response in cancer depends critically on the state of cancer cells and additional cells in the tumor microenvironment, both of which evolve over time. New single-cell and spatial molecular technologies enable unprecedented characterization of these states across molecular and cellular scales, but are challenging to interpret due to the high-dimensional nature of these data. New computational methodologies are essential to interpret these data. We demonstrate how the Bayesian non-negative matrix factorization method, CoGAPS, enables us to learn patterns associated with immunotherapy response and resistance from single cell data. While cellular composition is important, the spatial distribution of cells in the tumor microenvironment further mediate response and resistance to therapies. Emerging spatial molecular technologies provide a powerful tool to model these interactions. We demonstrate how CoGAPS further models intra-tumor heterogeneity of the tumor microenvironment and tumor cells from Visium spatial transcriptomics data. Finally, further integration of the molecular features learned from multi-omics data with mathematical modeling has the power to leverage the intra- and inter-tumor heterogeneity these data uncover to predict mechanisms of immunotherapy response and resistance.