A draft settlement between states and mortgage companies that would let the nation’s biggest banks pay out billions to compensate for a raft of foreclosures has public interest groups across the political spectrum hopping mad over a deal they say was forged behind closed doors and is being strong-armed by the Obama administration.

The draft proposal, which was sent to state officials Monday for approval, is supposed to overhaul the mortgage industry and help homeowners. In it, the country’s five largest mortgage lenders offer to pay out as much as $25 billion to cover new terms for homeowners driven out by foreclosure. But people who lost their homes are unlikely to get them back or see much financial benefit from the deal.

Advocates and watchdogs on both sides of the political aisle are fighting the deal, although for very different reasons.

On the left, complaints about the agreement being pushed by the administration center on whether the federal government is letting the banks off the hook for a foreclosure crisis that critics say could’ve been avoided had the banks been more upstanding about their deals. However, President Obama’s supporters also credit him for his plans to launch an investigative unit on abusive lending to be led by the Justice Department, a move announced in the State of the Union Tuesday night.

The argument from the other side claims the administration is trying to fast track a deal that amounts to extortion of banks that followed federal government rules to expand access to loans.

Judicial Watch announced this week that it is suing the Department of Justice and the U.S. Department of Housing and Urban Development in an effort to get more information about the settlement offer and the audits that purport to prove the banks defrauded consumers.

Read more: