Great question — answering as the UphillTech team:
You need to select your LLM and something to run the LLM like Ollama or vllm. For what LLM you want to use you need to consider how much RAM and processing power you have. Here is an example of how fast certain model are on certain GPUs https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference Be careful: OOM means that you can't use the LLM as you don't have enough memory. If all the models are too big there are even smaller models available. Once you have an model server implemented, you can use a popular framework like Langchain, Spring AI, Semantic Kernel, etc. to implement your agent. If you want to do RAG the "traditional" way basing it on vector embeddings, you'll need a vector...
Full answer by joerg-de: https://ai.stackexchange.com/a/49103 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
