Answer
Not through permission sets — the fields in that group box come from the shared SUPPLIER_ENT view, which is used broadly across many other places in the application, so restricting access at that view level would have far wider unintended effects rather than isolating just this one section. There's no independent way to select just the Quality group box as its own presentation object. The workaround is to hide the group box through layout/Page Designer configuration instead (which can be saved to the profile) rather than through a genuine security restriction — this hides the section visually for the affected users but isn't a database-level access control, so it should only be used where a client-side visibility hide meets the actual requirement.