1)the condition i am using is not working :-
[cb:userdata field="avatar" avatar="nophoto_n|tnnophoto_n\.png$/]

That's not a valid substitution. You'll need to use the below to condition if the avatar field has an image uploaded.

Display: Field conditional self
Field: Custom > Value
Value: [cb:userfield field="avatar" /]
Translate Value: No
Operator: Is REGEX
Value: /(nophoto_n|tnnophoto_n)\.png$/
Translate Value: No
Mode: Hide

2)what should be the right substitution for this, (i am wanting the Firstname of the user to whom the PM is being sent ) ?

You'd just substitute in the first name using [firstname] for example. The user object is going to be the user id specified in &users so substitutions will default to their user object.

1)I tried the substitution but there is no difference in behaviour for users who are showing their avatar or who are not showing their avatar. The link shows for both type of users.

Is the avatar field it self hidden? This could result in the substitution failing as the field won't be reachable. You can try the below, but it may not be as accurate as it's checking the database value directly.

Display: Field conditional self
Field: Custom > Value
Value: [avatar]
Translate Value: No
Operator: Is REGEX
Value: /(nophoto_n|tnnophoto_n)\.png$/
Translate Value: No
Mode: Hide

Is the avatar field hidden it self and if it is how is it being hidden? Another field being conditioned? CB Privacy? The problem with the above substitutions is they return nothing if the field can't be accessed except the direct database substitution [avatar]. You can test this by adding a Custom HTML field with the below substitutions.

[avatar]
[cb:userfield field="avatar" /]

This will show you what values are being output so you can adjust your condition as needed. If another field is hiding the avatar and being conditioned then just reverse the condition for you link field.