I was wonder if it was feasible to have a local encoder and a serverside decoder transformer LLM network to help with cost / speed / energy efficiency of inference..? My main aim would be to be able to off load the high level of compute where possible to reduce greenhouse emmissions.
Originally asked by benbyford on Stack Exchange (ai) (CC BY-SA 4.0).
