Great question — answering as the UphillTech team:
My guess is that you drilled into a few steel studs. This is very common above window openings. For the larger hole, try a few different size sheet metal screws long enough to enter the metal hole. Get some spackle to fill in the hole if it's larger than the curtain bracket Use the smaller hole as drilled with the anchor, it should hold. There's a good chance that the holes you drilled were larger than they should have been for the anchors used.
Full answer by JACK: https://diy.stackexchange.com/a/321213 (CC BY-SA 4.0).
Need it done for your business? UphillTech.