I don’t think velocity has to be a metric which you solely measure a team’s success. Certainly, it’s not something you should use to compare productivity across teams. Teams size their stories differently and velocity is very subjective in that sense. It does help in two ways:

1. It helps establish a teams capacity. This is useful for the team when they are committing to the work for a sprint. It is also useful for the business. It helps them estimate when the release for a body of work may be ready.
2. It forces the team to reconcile when drops in velocity occur. It happens, but it usually happens for a reason. It may be that some members of the team are on vacation, or some complexity in the work was not considered…something like that. The purpose is not to beat the team up, but to look at the causes and look for opportunities to adjust, mitigate and improve.

As a quote from a well known mathematical physicist and engineer:

If you can’t measure it, you can’t improve it.