I finally got around to reading the excellent stateof.ai report by Nathan Benaich and Ian Hogarth this week, so I was well primed for Deepmind's Monday announcement that they had solved the Protein Folding Problem, a so-called "50-year-old grand challenge in biology", with their latest version of Alphafold. Some vestigial part of my brain noted that the press clippings preferred pretty alpha helical spirals to beta pleated sheets.
What makes this challenge interesting is that the underlying amino acid sequence is known, the difficulty is in translating that into the correct 1 out of a possible 10^300 3D confirmations possible for a typical protein. Alphafold provides a solution to that problem (pending publication and peer review), though like most a lot of AI systems, it doesn't show its workings (outside some confidence intervals). We get an answer, without a reason why.
This is not a novel situation - for most of human history we got along just fine without knowing how things worked, just that they did. What is unusual in this case is that the thing in question is entirely man-made. Richard Feynman wrote that "what I cannot create, I do not understand". When it comes to understanding, even creation is no guarantee.