Lately, a debate has erupted among Congress, bankers and the media about whether the Federal Reserve Bank of New York has accumulated too much power since the financial crisis and needs to be reined in.

Outgoing Dallas Fed President Richard Fisher has suggested the New York Fed be stripped of its authority over many of the largest banks because it is seen as too cozy with them. San Francisco Fed President John Williams has said he doesn’t believe there needs to be any changes.

In Congress, interest in reform is coming from both sides of the aisle. Alabama Republican Sen. Richard Shelby, the new chairman of the Senate Banking, Housing and Urban Affairs Committee, has said Fisher’s proposals deserve consideration. Democrat Jack Reed of Rhode Island has a bill to require Senate confirmation of the New York Fed’s president.

Fed Chairwoman Janet Yellen, meanwhile, has testified that she does not believe changes are necessary. In a speech last week, she detailed measures established in 2010, including stress tests to ensure banks have adequate capital, that have centralized supervision in Washington.

Echoing Yellen’s themes, a Wall Street Journal story the day after her speech cited a previously unreported Fed document to describe how centralization had diminished the New York Fed’s authority. The document describes a now 5-year-old structure whereby supervision decisions by New York and other regional reserve banks that monitor large firms are vetted by a committee headed by Board Governor Daniel Tarullo.

Critics on both the right and left are skeptical that the New York Fed’s power has slipped much because of its unique role. The New York Fed supervises 13 of the 16 banks deemed the most systemically important. It has a larger voice in monetary policy than any of the regional Fed banks. Of the 12 reserve banks, New York is the only one that conducts its own enforcement investigations.

“Has the New York Fed really lost this authority, power, sway? I am doubtful,” said Simon Johnson, a senior fellow at the Peterson Institute for International Economics.

“This large institution group at the Board has been there for a while,” added Paul Kupiec, referring to the Large Institution Supervision Coordinating Committee, the panel headed by Tarullo that was set up to review high-level supervision decisions and plot policy direction.

Kupiec, a resident scholar who specializes in banking at the American Enterprise Institute, a conservative think tank, said the steps taken so far by the Federal Reserve have likely done little to assuage lawmakers’ concerns with the New York Fed’s disproportionate clout. “I don’t think it changes anything about the Congressional reform,” he said.

A spokesman for the Federal Reserve Board declined to comment on any diminishment of the New York Fed’s power, or on proposals to restructure the Fed to reduce New York’s influence.

Last year, ProPublica and This American Life reported on clashes within the New York Fed after it placed specialized examiners inside the “too-big-too fail” institutions it supervises under new responsibilities given to it by Congress after the 2008 financial crisis.

One of the examiners, Carmen Segarra, was fired in 2012 after her superiors pressed her to change a critical examination finding regarding Goldman Sachs. The New York Fed has said its dismissal of Segarra had nothing to do with her examination of Goldman. Concerned by her interactions with supervisors and colleagues at the New York Fed, Segarra took the unusual step of recording her meetings and conversations, making about 46 hours of tapes in all.

The recordings offered an unprecedented window into the institution’s culture at a time when the post-crisis reforms – some of the same ones cited by Yellen – were purportedly taking hold in 2011 and 2012. They offered little evidence that Washington was forcing New York to take a tougher line. Instead, Segarra’s supervisors appeared to defer to Goldman.

At a Senate hearing that followed the Segarra revelations, Bill Dudley, the president of the New York Fed, divulged that the Large Institution Supervision Coordinating Committee’s operating arm had vetted Segarra’s finding on Goldman. But Segarra has said she was never contacted by anyone from the operating committee or asked to provide any of her documentation.

Ahead of the hearing, the Fed said it would launch a review, as would its inspector general, into how effectively the opinions of frontline examiners were filtering up to the large institution committee. The Fed has since begun to move its examiners out of the largest banks.

While centralizing some authority, the Fed has given the New York branch a larger footprint on the committee than the other 11 regional banks.

It’s the only regional bank to have three staffers on the committee, according to a list of the group’s 16 members obtained by ProPublica. Other regional banks only have one staffer. The New York Fed also occupies five of the 20 seats that make up the operating group; that’s the same number of seats as all the other regional reserve banks combined.

The secrecy that shrouds bank supervision makes it difficult to judge how the large institution committee functions in practice, or the extent of the Federal Reserve Board’s involvement in supervisory decisions. The Fed has a long history of fighting public records requests that could provide insight into how it supervises banks. Fed employees, by rule and custom, seldom speak publicly about their jobs.

Yellen’s pronouncements do not seem to have dampened the ardor for reform in Congress.

Reed, who sits on the banking committee chaired by Shelby, worries that changes made since 2010 to shift power from New York to Washington might not prove durable since they are not enshrined in law.

“I recognize and appreciate the steps the current Board of Governors for the Federal Reserve has taken to improve supervision of some of the nation’s largest financial institutions,” Reed said in a statement to ProPublica. “But the current Governors won’t be in power forever, and there is not guarantee their recent reforms will be in place, 5, 10, or 15 years from now.”