They can't cancel the registration. They can only cancel the payment. CBSubs is not able to accept payments or create baskets without a user object so CBSubs always takes affect after registration.

Understood but then how does the prospective subscriber go and make payment then? There is no obvious way to tell them this or show them - so I end up with multiple emails and messages. Is there a way when they come back to the registration page to show an outstanding payment and get them to complete it?

If they attempt to login and have an unpaid basket it will ask them to pay. If they attempt to login and have no subscription, but one is required they'll be presented the plans that are available to them so they can subscribe.

Also by the way, its worth noting that CB Subs default message on the payment screen is "Cancel Payment and Subscription" so if what you are saying is true that it only cancels the payment then that default should be changed as it gives the impression that no user recording (not the plan, but a record of the person applying, since GDPR regulations would assume that there is no legitimate interest in this data being retained if an applicant decides to cancel before purchasing a subscription) has taken place.

The default string is in no way confusing. It cancels the payment and the subscription which is what they're attempting to do (pay for a subscription). Regarding GDPR there is still interest in their data as they had interest enough to move on to the payment process and could be cancelling to purchase something else or to come back to it. There's no way of knowing their exact intentions. If they want to delete their account either Joomlas frontend user delete functionality or the functionality provided by CB Privacy is sufficient to allow users to remove themselves.

Perhaps a "delete user record" routine can be run when an applicant presses that message?

We have no such functionality because it's dangerous. An accidental click costs the user all the time they spent registering and loses you the customer. It should not be possible for someone to delete their account so easily. If you want such functionality it is likely doable with CB Auto Actions.