There is a page explaining how to add/deduct userpoints from a user for each message using a custom module (http://drupal.org/node/646512) so here we will cover how to do this with the Privatemsg-Rules integration. Make sure to have the Userpoints rules integration sub-module enabled and lets get started.

To add or deduct points from either the author or the recipient of the message
Event: Privatemsg - A message is sent
Action: Userpoints - Grant points to a user
Arguments configuration - User: [Author of the message] or [Recipient of the message.]
Enter the amount of points to add to the user, or enter a negative number to deduct points.

Very simple =)

Additionally, you may add conditions to check if the user (author or recipient) has a specific role and a configurable message on the page to show the user after the message is sent.