Don’t we all feel an extra measure of sympathy for that category of victims of Bernie Madoff’s vast fraudulent investment scheme, the ones whose names were tagged with the descriptor philanthropist?

Well, at least in some cases, we should not. Some of them were far from being hapless victims. Some made, and then hid, millions, or in at least one case, billions of dollars, says Lucinda Franks, a Pulitzer-Prize-winning investigative journalist.

And on Thursday night, the former New York Times and New Yorker writer who now reports the Madoff case for the Daily Beast, will speak about what she’s learned of the massive scam, and some of the not-so-innocent victims, at the Chilmark Community Center.

The Madoff scandal was the largest ever fraud of its kind, a Ponzi scheme widely cited as being worth $50 billion or more. But it may be that the total losses will never be clear.

What is clear to Ms. Franks, though, on the basis of what her contacts have told her, is that there has been a fair measure of sympathy wasted on some of his investors.

Back in April, in the Beast, she named Norman Levy, “a real-estate mogul and philanthropist who has previously been portrayed as a victim.”

In fact, she went on to write, he appeared to be part of a powerful international coterie of accomplices who helped shuffle billions through tax havens around the world.

And he was not the only one. Now, few people really believe that Mr. Madoff acted alone in his huge scam. A lot of speculation has focused on his family — wife Ruth, sons Andy and Mark and brother Peter — as well as some people involved in so-called feeder funds which directed investors to Madoff.

But Ms. Franks suggests something even wider than that: a large network of people who either knew of Madoff’s criminal activity, or willfully avoided knowing.

“To some degree,” she said yesterday, “each member of the family participated in the Ponzi scheme, some knowingly, some of them quasi-unknowingly.

“They knew certain aspects that were illegal, but they didn’t ask any questions about where the money was coming from or who he was sending the money out to.”

At the very least, she said, it amounted to a don’t ask, don’t tell type dodge. She thought charges against other family members were likely, as investigators untangle the complex web of transactions.

“But beyond that — and this is something I haven’t written yet — is that this was a global scam and there were a number of prominent philanthropists and entertainers, writers, artists, who had money enough to invest early on, did so on the promise that if they promoted Madoff Securities they would get 10 to 30 times their money back within a very short period of time.

“Some of these people are household names,” she said.

And, yes, some lost money later in the piece, but not all or even most of it. And they do not merit sympathy.

“Absolutely not,” she said. To the extent that they became victims, she said, it was due to criminal activity that they “sort-of knew about.” She continued:

“They knew they couldn’t get 10 or 30 times their investment unless something illegal was going on.

“And some of the leading philanthropists whose names have been mentioned as victims of the crime actually made up to $2 billion, in one case, early on. And then hid the money so it could not be subject to clawback.”

So how does she claim to know this?

Well, her first break was having known one of the senior investigators since childhood.

“I have a very, very high-up source in federal law enforcement, who has been my deep throat,” she said.

“It’s very funny; I have to call him and use an alias, between 6:30 and 7:30 a.m. or 7:30 and 9 p.m.”

He got her started; then a couple of other major sources came to her after reading her early stories. One of them is a woman she portrays as a kind of double agent, close to Ruth Madoff, but also a big loser in the Ponzi scheme.

“There’s a lot of pathos to this story,” Ms. Franks conceded.

“Ruth Madoff has not been charged with any crime, but she has been shunned. She had a lot of trouble finding a new apartment when she was forced to move from her old million dollar penthouse. She can’t get a hairdresser’s appointment, can’t get a restaurant reservation.

“She’s been shunned by all but a few of her closest friends, one of whom is my source. And she’s talked to me about members of the family as they have been questioned.

“There’s also a woman who is very close to the case and knows about some things when they happen or before they happen,” she said.

So what is happening, and what might we expect to happen?

“The last I heard, the trustees had only collected a few million dollars, but there could be a lot more coming,” Ms. Franks said, adding:

“I think the U.S. Attorney’s office is moving very slowly; they’ve had a lot of trouble getting hard evidence.”

But a deal struck with Madoff’s chief financial officer, Frank DiPasquale, under which he would be charged with lesser crimes in return for revealing secrets, should speed things up, she said.

She also said there was now an unprecedented level of cooperation between various tax haven countries and the United States.

“Gibraltar for instance has frozen all the assets on a bank account under the Madoff name. Anywhere in the world they see the name, the authorities freeze the money,” Ms. Franks said.

Ms. Franks hinted she might drop another name or two of those dubious philanthropists involved with Mr. Madoff when she speaks on Thursday night.

“I’m going to tell some of the story behind the story that I haven’t printed yet,” she said.

And, she guessed, given the summer demographics of the Vineyard, she may draw an engaged audience.