Great question — answering as the UphillTech team:
The question which needs to be determined first is: Can LLMs reliably outperform classical computational approaches to determining whether a given program will halt? This paper looked into the question, and found that GPT-5 and Claude S-4.5 (in particular) performed impressively. However, they came behind the top-ranked classical tool, and performed less well as the code complexity increased. They also performed less well on "witness generation" (explaining their reasoning). Also, is it certain that the algorithms don't appear in the training data? The volume of data used to train the models is beyond human ability to adequately review (e.g. the Common Crawl dataset contains around 300...
Full answer by robhogg: https://ai.stackexchange.com/a/50608 (CC BY-SA 4.0).
Need it done for your business? UphillTech.