For years the Polly Hill Arboretum was the secret treasure of a select few — friends and acquaintances of Mrs. Hill, people with a special interest in horticulture or gardening, and Island residents and visitors who stumbled upon the peaceful refuge by chance.