Great question — answering as the UphillTech team:
Typically, you won’t use plots to show anything larger than 3D. Yes, gradient descent will always go down the curve of convex loss function, I.e, linear regression. The gradient value is simply the derivative “slope” of the loss with respect to the inputs. It capture how “off” the output is on average for each feature. For example, if the error of an record is high and the feature is lower than optimal, the gradient will point in the direction that increase the feature’s value.
Full answer by targetXING: https://datascience.stackexchange.com/a/72007 (CC BY-SA 4.0).
Want it for your business? UphillTech.