Joining a group lets you participate in that group. You can't post, comment, etc.. unless you join that group. Open and Approval groups have publicly view-able content. Invite groups are private. More group types will be coming in a future release; including a Page type that has no user management at all.

You will not see activity in CB Activity on your profile for groups you didn't join unless your connection posted something in that group and you can access that group. Recent Activity is not your activity. It is All activity you have permission to see. If you only want to see activity relevant to you then use My Activity menu endpoint.

The new Filters feature coming with CB Activity 5.0.0 is maybe of some interest as it filters the stream to only include the specified assets and can also filter posts by user. You can take a look at this feature in the announcement post below.

www.joomlapolis.com/forum/6-news-and-publicity/240952-what-am-i-working-on#312215

So is there no way to see activity for groups that a user is joined.

There is no way to filter CB Activity to only groups you have joined, correct. There is only a way to filter it to groups you or your connections posted content to and not random people you are not connected with.

I understand that they have access to that activity content, but it would he awesome if I could filter the activity so that it's related to only the group's that the user is joined too.

It's possible this can be implemented by GJ it self. It is not possible for CB Activity to implement this as it has no clue what GJ even is.

forge.joomlapolis.com/issues/7567