The Harvard Elementary School, also called the Brown Building, stood on Mass Avenue near the site of the current Hildreth School building.