Great question — answering as the UphillTech team:
First, Google can crawl any port. See Can Google index a non-80/non-443 URL? However, I think there's a clue to your question in John Mueller (Google) comments in the above question. John doesn't say anything about it impacting SEO. That doesn't mean it doesn't, but I think that if it did impact SEO, he probably would have said something. The question remains, if Google can find your content, does the non standard port enter into their ranking algorithm? Does the user needing to remember the port as alluded to in the above question, impact whether Google thinks the content is any less valuable, even though Google will provide the full link with the port number? I would propose a test....
Full answer by Trebor: https://webmasters.stackexchange.com/a/132250 (CC BY-SA 4.0).
Need it done for your business? UphillTech.