On Chappaquiddick, concern is growing over what neighbors are calling a hotel-like development on Sampson’s Hill, a residential neighborhood on a high point in the center of the remote island located off the eastern end of Edgartown.

The property owner, Stephen Olsson, is advertising his luxury vacation rental homes on the Internet as places that can sleep up to 34 people in a compound that includes a clubhouse, pool, gardens, hot tub and jacuzzi.

All of Chappaquiddick is zoned as residential, and this week a group of Chappaquiddick residents appeared before the Edgartown planning board to complain about the development, which they say runs afoul of town zoning bylaws.

“You’re plunking a hotel in the middle of a single-family neighborhood,” said Ronald Monterosso, a Chappaquiddick resident who lives nearby. Mr Monterosso said the problem lies in the town’s transient rental bylaw which needs to be overhauled.

Reached by telephone on Thursday at his residence in Manchester, N.H., Mr. Olsson declined comment.

Public property records show that in 2011 Mr. Olsson bought Don Crocker’s home at 72 Chappaquiddick avenue. Mr. Olsson already owned abutting lots at 5 and 7 Chapel avenue. Construction is now underway to turn the 3.4-acre former Crocker property into a vacation rental compound that sleeps 20, and the property is advertised as such on the Web site weneedavacation.com. Mr. Olsson’s Chapel avenue home is already a vacation rental and is advertised as sleeping 14 people in five bedrooms, and having a pool, gardens, hot tub and jacuzzi. The rental fees are $8,000 to $15,000 a week.

The town planning board told the complaining neighbors and residents on Tuesday that they have little or no jurisdiction over the matter.

The board of health may be a different story. Edgartown health agent Matt Poole told the Gazette this week that the board of health had initiated some kind of enforcement action against the property owner that will take effect this week, although he could not elaborate. “We are aware of it, and we are taking action relating to board of health jurisdiction,” Mr. Poole said.

Mr. Poole said there are concerns that the homes have more bedrooms than allowed under the septic rules for that area of Chappaquiddick.

A letter sent to Mr. Olsson by 16 neighbors and e-mailed to the Gazette this week outlined concerns about the extent of the project, including noise levels and vehicles on the narrow, unpaved roads that wind up to Sampson’s Hill.

“I think it’s setting a dangerous precedent,” Edith W. Potter, a longtime Chappaquiddick resident, said yesterday. While Mrs. Potter said she didn’t know all the details about the plan, “I just don’t like the idea of it,” she said, noting that it sounded like a commercial use in a residential area.

“He’s making a complex,” said Camron Adibi, whose parents own a nearby home. He said the development has sparked a number of concerns, including overuse of private dirt roads and possible zoning bylaw violations.

“How did this project fall through all these major cracks?” Mr. Adibi asked. “It has an impact on all the neighbors.”

“I’d say it’s gone too far,” he said. “Forty people in a little neighborhood in the middle of Chappy?”