A number of banks are likely to return government bailout money soon, Treasury Secretary Timothy Geithner told Wall Street’s main trade group.

At the same time, he said, the financial services industry remains fragile, and can expect more regulation for some time. Geithner made his remarks in a conversation with TV host Charlie Rose at the annual meeting of the Securities Industry and Financial Markets Association.

Geithner didn’t give any details about the expected paybacks to the Troubled Asset Relief Program (TARP). “It will depend on the institution, but for major banks in the country I think that money will come back relatively quickly,” he said.

Geithner noted that institutions have already paid back $71 billion in TARP funds. But that leaves about $134 billion that was handed out to big bailout recipients such as Citigroup and Bank of America, as well as hundreds of community and regional banks.

The two big banks are keen to follow the example of JP Morgan Chase and Goldman Sachs in paying back the TARP money, so that they can get away from restrictions on pay and other government scrutiny.

Geithner also told Rose that fixing the broken financial regulatory system remains an urgent priority for the administration.

“We need a much stronger set of protections for consumers and investors,” the Treasury secretary said. “We need to make firms less vulnerable, to restrain risk.”

In addition to monitoring risk and bolstering capital at financial firms, the government wants to ensure that when a systemically important firm faces collapse, there is a structure in place to isolate it and dismantle it, Geithner said.

House Financial Services Chairman Barney Frank unveiled draft legislation for this so-called resolution authority that was hammered out over the weekend by his committee and the Treasury. Hearings are scheduled for Thursday on the legislation.

Assistant Treasury Secretary Michael Barr earlier this week said the legislation must enable bank regulators to replace the board and management of a failing systemically important institution as it moves swiftly to limit the damage to the financial system from a possible collapse like that of Lehman Brothers last year.