"Visible" refers to who can see that the group exists.
If a particular group is configured to be Visible to the Organization Membership, then every active user with access to the groups area of the applications will see that group on the All Groups page and in the All Groups portion of the Groups menu (the display when you click on the Groups navigation item next to the home icon).
If a particular group is configured not to be Visible to the Organization Membership, but is configured to be Visible to specific groups, then only members of those groups will see the group listed on the All Groups page and in the All Groups portion of the Groups menu.
Note: This is different than having access to the contents/work of the group. That is still controlled by the group roster and sharing configuration. The Visible setting simply controls visibility to the group's existence.