Jeffrey Madison remembers playing board games to the light of a kerosene lamp at a time when many of his neighbors still didn’t have electricity. His childhood home stood in the sweep of the Gay Head Light, which flashed its red and white signal across the headlands.

“I can see it over the top of the trees in the house that I’m renting here,” said Mr. Madison, who returned home this summer after 20 years to become the town administrator.

Much has changed since his days of camping on the beach with friends and attending the Aquinnah School (now the town library) across from town hall. The lighthouse was moved away from the edge of a cliff two years ago, and with taller trees, Mr. Madison said the light doesn’t shine as far inland as it once did.

Some changes are a matter of perception. “It seems to be a longer ride from down Island to up Island than I remember,” Mr. Madison said. “I’m sure that the Island hasn’t grown. I just want things to happen, so going from here to there takes a longer time than I recall.”

Change is nothing new for Mr. Madison, who in his 68 years has lived many lives: Island real estate developer, restaurant owner, fireplace builder, Wampanoag tribal official, town selectman, Cape Cod attorney and now administrator of a town whose name itself was changed in 1998, from Gay Head to Aquinnah.

He still calls it Gay Head, occasionally catching himself mid-sentence. Aquinnah is a Wampanoag word that means “land under the hill.” Mr. Madison was elected as a Gay Head selectman in the 1970s, a position he held for about 15 years. During those years, he played a role in gaining federal recognition for the Wampanoag Tribe of Gay Head (Aquinnah). The recognition in 1987 ended a bitter 13-year land claims dispute and opened the door to federal funds and other benefits for the tribe.

He served as director of economic development for the tribe for about five years in the 1990s. Among other things, he worked on early plans for a casino in southeastern Massachusetts, years before casino gambling became legal in the commonwealth. More recent efforts by the tribe to build a class II gambling facility in Aquinnah have stirred heated debate.

“I am against the casino issue, and I started the casino issue,” Mr. Madison said this week. But he also defends the tribe’s right to move forward with its plans for Aquinnah — a key point of contention in recent years. “They should have their sovereign rights,” he said. “Sadly, one of the big mistakes that was made in the course of settling the land claim was not stressing the importance of sovereignty.”

“I sadly am responsible for that,” he added of his role as selectman at the time. “[I] was too easily persuaded to give up those sovereign rights. It’s something that I’ll live with and I’ll die with.”

These days, he said his relationship with the tribe has cooled, given what he calls a tribal government focused more on its own initiatives. “There are people who don’t know and aren’t willing to give credit or accept the opinions of those of us who did grow up here and raised our families here and maintained our tribal existence,” he said. “Now it’s more of a what’s-in-it-for-me association.”

Despite the ups and downs over the years, he didn’t think the past would stand in the way of his role as town administrator in the Island’s smallest town. “I’m trying not to be weighted by the baggage,” he said, adding that his long experience, including as a businessman, has mellowed his daily interactions. “I want to let things filter down to this job, rather than be an extrovert and go out there and try and force my will.”

The Madison family has a long history of leadership in Aquinnah. Mr. Madison’s father Luther Madison was a tribal medicine man, as well as a selectman and assessor for the town. Luther’s father Napolean Madison was also a medicine man and town leader, and founded the well-known Aquinnah Shop in the 1940s. “To that extent, I’m carrying on a tradition,” Mr. Madison said, although he pointed out that town administrators serve at the discretion of the selectmen and take a back seat in decision making.

He left Aquinnah in 1997 to attend law school, later joining the staff at Wynn and Wynn, a law firm based on the Cape. He returned to the Island frequently, including for the relocation of the Gay Head Light in 2015, when he stood with a crowd of onlookers as the tower inched its way to safety on a spring morning.

Now back in his ancestral homeland, he said the changes are sweeping but not absolute. He said the biggest change has been the influx of new residents, most of whom are not members of the tribe, unlike during his years as selectman. But the town still ranks among the smallest towns in the state, which he called a blessing and a curse.

“Issues tend to get magnified because of the size of the community,” Mr. Madison said, noting an ongoing dispute involving lease lots in Menemsha, a personnel conflict involving the town assessor and other issues. Arguments at Aquinnah’s annual town meetings seem to escalate regardless of the agenda.

“There has never been a lack of things to discuss, let’s put it that way,” Mr. Madison said. But he argued that every town has its share of drama. “I don’t think that Gay Head is unique — or that Aquinnah is unique — in that respect,” he said. Aquinnah’s small population still offers a sense of community that can be hard to find. But he said large tracts of conservation and tribal land and strict limits on development make it difficult to maintain a steady tax rate and also pay the bills. His priorities this year include working with the Martha’s Vineyard Commission to identify properties in town that could still be developed for the sake of adding to the tax base.

“Wouldn’t it be nice if Katama were void of condos and mega houses,” he said. “But it defrays the cost of the town operations. You could probably make the same argument for areas around Sengekontacket, or the bluffs in East and West Chop. Those were gorgeous places. . . . Now it is what it is. But it gives strength to the ability of the town to manage and to grow.”

He plans to present ideas for development to the selectmen in the coming year. “I think they are looking for me to do that. I won’t shrink from it,” he said. He added that any final decisions would come down to voters and the selectmen.

During his job interview this summer, he said that among his long list of careers, town administrator might be his last. “It probably is,” he said this week. “But who knows. I thought being an attorney would be my last career.”

He looks forward to seeing how the town continues to change in the future. “I’ll be here for a while,” he said.