PROVIDENCE, R.I. (AP) — Twenty-one members of the Vanderbilt family have written to board members of a preservation group that owns The Breakers, saying its management is no longer fulfilling its commitment to the public trust and is exploiting the Gilded Age family mansion, sometimes referred to as America’s Downton Abbey.

The nonprofit Preservation Society of Newport County purchased The Breakers from the Vanderbilt family in the 1970s and runs it as a museum that is one of New England’s top tourist destinations. The 70-room mansion is a National Historic Landmark and is considered the crown jewel of Newport, a city filled with seaside mansions.

The objections center on a plan to build a visitors center on the grounds of The Breakers, which the letter’s signatories, including designer Gloria Vanderbilt, say would degrade the property and its historic grounds. They want to put it across the street or elsewhere in Newport.

The Preservation Society wants to build the center as a way to provide better amenities, such as upgraded restrooms and a place to buy sandwiches. The Breakers draws 400,000 visitors every year, and the society’s 11 properties draw 900,000 visitors collectively.

Zoning officials in the city gave the project the green light in January.

While some Vanderbilts have objected publicly and privately to the visitors center in the past, the letter, dated Saturday, is the most concerted effort to date by the Vanderbilts to stop the project and take on the leadership of the Preservation Society.

In the letter, the family members write that The Breakers’ legacy is disappearing and say they believe the Preservation Society “is no longer a trustworthy steward of its flagship property.”

“It has betrayed its nonprofit mission to preserve historic Newport,” they write.

Andrea Carneiro, a society spokeswoman, said in an email that the group spends around $1.5 million per year preserving, restoring and maintaining the building and has demonstrated “exemplary stewardship.” She said the group purchased the mansion “with no restrictions, pledges or promises” to the Vanderbilt family, and she pointed to its latest project, restoring the mansion’s underground boiler room, as evidence of its preservation work.

The family members also write that they will not donate money or Vanderbilt family objects to the society “until the current leadership climate is changed.”

Among those who signed the letter was Paul L. Szapary, who previously served on the board of the Preservation Society, and who, along with his sister, owns many objects currently on display at The Breakers, including furniture, artwork and dinner plates.

“We are hoping that by expressing a united front we can maybe convince some people to take a closer look at this plan,” he said.

When asked to address the family members’ plans to withhold donations and their complaints that The Breakers is being exploited, Carneiro referred to her emailed statement that did not specifically address the questions.

However, the email said financial support for the Preservation Society has reached “record levels,” including an annual fund that raised more than $1 million last year.