Field permissions are based off Tab permissions. To do what you're wanting you'd need to have 2 tabs, 1 for each usergroup, and place the fields of choice on those respective tabs. Ensure both tabs are in "not shown in profile" positions. Once done you'll need a 3rd tab that will have delimiter fields using substitutions to render those fields as tab permission is for viewing AND editing, so you need to display via delimiter for other usergroups to see the contents.