Great question — answering as the UphillTech team:
Given that you have very limited data (only 6 research papers per class), it's essential to use techniques that can help maximize the value of your small dataset. If you want to pursue your own embedding-based classification approach, instead of taking dot product, use a simple K-NN classifier with you few-shot examples. If still not working well, try to add a classifier head on top of your chosen pretrained foundation embedding model like SBERT or GPT. Train your classifier head as simple MLP with your publishable and non-publishable labels with frozen embedding layers. If performance is inadequate, fine-tune the last few layers of your embedding model cautiously. You can also use various...
Full answer by cinch: https://ai.stackexchange.com/a/47752 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
