Answer
Yes — find the specific Selector definition backing that search list in Page Designer and simply remove the unwanted column (like Site) from it; this is a straightforward column-removal on the Selector itself, not a separate visibility-condition setting.