A risk premium is the additional expected return an investor requires for holding a risky asset instead of a risk-free one. Its two ingredients are a baseline, the risk-free rate, which is the return on an asset whose payment is treated as certain (in practice a short-term U.S. Treasury security), and the extra return layered on top to compensate for the chance that the risky asset disappoints. Written out, the expected return on a risky asset equals the risk-free rate plus its risk premium.
The word "expected" is doing the heavy lifting. A risk premium is a forward-looking demand, agreed to before the outcome is known. It is the reason a rational investor is willing to hold something that might fall in value at all: without the prospect of a higher average payoff, there would be no reason to accept the uncertainty. It does not mean the risky asset will in fact beat the safe one over any particular stretch. If it always did, there would be no risk to be paid for.