If you're viewing a Recent Activity stream (which is just all activity) or a GJ asset like groupjive.group.% then it's going to show all activity the viewing user has permission to see so yes it will show activity from groups they have not joined. You however should not see activity on your profile for groups you have not joined unless it's a group you can see and one of your connections is interacting with that group.