When jazz crooner Jerri Wells is finally coaxed up to the front of Oak Bluffs’ Offshore Ale by Eddie (Pepé Caron) Larkosh for a rendition of Do You Know What It Is to Miss New Orleans? she does not stick to the script for long. She delivers a few bars of the prescribed number then, like some sort of thief sidling past a security guard, hums her own improvised segue and ducks into the second verse of A ll Of Me, the Billie Holiday version, leaving the band to scramble after her.

Mark Williams and fellow boatbuilder Bill Sauerbrey, who have personally sailed from Centerville for this gig, come for the improvisation and stay for the beer and nuts. “It’s great to watch people who are entertaining themselves as much as us,” says Bill between sips of ale. “We tend to think ‘The sun’s out, let’s go listen to some jazz.’”

The Sunday Jazz Fellowship has convened here on Sunday afternoons for two-and-a-half years. Founder and multi-instrumentalist Mr. Larkosh encourages guest appearances each weekend, and the line-up changes regularly. Currently Jameison Sennott plays the bass, Albert (Abby) Dreyer is on keyboards, Charlie Terry plays guitar and shares drumming duties with Mr. Larkosh, who also plays harmonica and leads the band through standards from the likes of Cole Porter, Gershwin, Rogers and Hart. A steady stream of singers duet with the nattily-dressed showman through the afternoon.

“I came out to play in Edgartown Yacht Club as a drummer in 1964,” says Mr. Larkosh, 72, standing at the bar after his set. Mr. Larkosh has a bristly white beard and wears a bowtie with a black corduroy blazer and a badge he sewed on from his last trip to Europe. “I spent the summer here with my wife and child, and when we left we got as far as East Providence then decided to come back.” Mr. Larkosh got a job as a science teacher moving between schools in a mobile science lab, and when he wasn’t working, he played jazz. The ‘Pepé Caron’ moniker is a mix of his grandfather’s and mother’s names.

Mr. Larkosh has been performing since the age of 15 and has two idols. One is Louis Armstrong, strains of whose voice can be heard issuing from this improbable source. Then there is Toots Tillman, a Belgian master harmonica player. “Tillman’s the original item, he’s not like some of these fancy dancy players. But he does what I could never hope to do, so what’s the point? Which is why I’ve taken up the piano.”

Mr. Larkosh saw his idol play at the Regatta Ballroom in Cambridge about seven years ago, travelling up and spending the night in his Volkswagen just to watch him play. He remembers spotting Toots Tillman in an empty bar after the show and introducing himself: “Mr. Tillman sir, I said, I would like to shake your hand,” Alas, a dueling harmonica scene did not ensue. “No, Jesus! I wouldn’t ever get my harmonica out within ear shot of the guy.”

On the other hand, Mr. Larkosh is very happy for people to play with him. “We get any one whose coming through to play,” he says. “We’ve had some really capable players here.”

Jerri Wells, who tapped along for nearly two hours before stepping up to sing herself, is one such luminary. Ms. Wells began her singing career at the Apollo Theater in 1934, and had the distinction of being the first black female to sing on American cruise ships, a gig which took her to Paris several times. Between fraternizing with the band after the show, she runs through her complicated life story at a gallop, often bursting into giggles and, unintelligibly for this reporter, French. Moving to the Vineyard when her husband of 52 years died in 1997, she has three kids, multiple degrees and performed at the jazz festival in Hartford, Conn., last year.

“Everybody is welcome,” continues Mr. Larkosh, expanding on the Jazz Fellowship ethos. “However humble their instrument — washboard, spoons — just along as they can support time. We do it for the joy of making music. You learn the sheet music, and then you put it to one side and play, from the inside out.”