They are not lending much to home buyers, and they're lending even less to single-family home builders, but banks are doling out big money to apartment developers.
Lenders provided a total of $172.5 billion in new mortgages for apartment buildings with five or more units in 2013, according to a new report out Friday from the Mortgage Bankers Association. That's an 18 percent increase in volume from 2012 levels.
The report also said the pace of lending for apartments so far this year matches the pace in 2013.
"Multifamily lending hit a new record in 2013," said Jamie Woodwell, the association's vice president of research and economics. "A strong appetite for loans led banks to increases multifamily lending by 19 percent, life (insurance) companies to increase by 65 percent and the (commercial mortgage-backed securities) market to increase by 119 percent."
Read MoreHome builder confidence U-turns, drops 5 points
The bankers association reports increases in multifamily lending among both smaller and larger loan sizes and within most lender segments.
The top five multifamily lenders in 2013 by dollar volume were JP Morgan Chase, Wells Fargo, PNCReal Estate, CBRE Capital Markets, Inc., and KeyBank. This, as JP Morgan and Wells Fargo reported big drops in residential mortgage lending to consumers in their latest quarterly earnings, with dollar volumes down 48 and 40 percent, respectively, from the same quarter of 2013.
"These bigger banks, Wells Fargo, JP Morgan, they are starving for loans, and JPM and Wells Fargo, we're hearing, are very aggressive on these multifamily projects," said Paul Miller, a banking analyst with FBR. "Does it worry me? It's still a relatively small line item for these bigger banks. For a smaller bank, where it would be a big chunk of their portfolio, it does worry me."
While single-family housing starts are still sputtering into recovery, up just 1 percent in September from August, multifamily starts continue to surge ahead, up 18.5 percent, according to a report released Friday by the U.S. Census. At that pace, the number of new apartment units could top 368,000 for the year. In September alone, completions of apartments reached a 24-year high. All of this begs the question of whether the market is overheating, and whether lenders, therefore, are taking an ever riskier bet?
"We view the nation as relatively under-housed," said David Toti, an analyst with Cantor Fitzgerald. "At some point the supply-demand ratio could tip into troublesome territory, but I don't think we're nearly there yet."
Read MoreShould you refinance TODAY?
He does cite some concern, however, around the Washington, D.C., area, Denver and certain Texas markets, but does not see an oversupply of apartments on a national level. Multifamily supply may be approaching peaks, and gains in both starts and permits for multifamily are decelerating, now below three-year averages.
"Is it overheated? A lot of people say it is, but it's hard for us to say that because the world's changing. It's more difficult for millennials and first time home buyers to buy homes, so they're renting for longer," FBR's Miller said.
It really comes down to fundamentals. Renting, while increasingly expensive, is still quite strong. Rents are high, and vacancies remain very low. First-time home buyers are not surging back to the home buyer market, and survey after survey shows that while millennials might aspire to home ownership someday, they are still lagging previous generations in stepping up to buy. Student debt, tighter credit conditions and basic consumer confidence continue to plague potential buyers—and fuel the rental apartment market.