I report the tension between the interpretability and the solvability of neural network applications of physics differential equations. From my viewpoint as the director of Machine Learning Physics (MLPhys) Initiative in Japan, I study the necessity of the interpretability of neural networks in various fields in theoretical physics such as condensed matter physics, lattice QCD and gravitational physics. This is based on a summary discussion of MLPhys International Conference 2026 which was held in Okinawa, Japan in July 2026.