The online real estate marketplace expects core business revenue to grow 56% within the next five years, but their cash-burning plan to buy and sell homes could take just as long to realize $20 billion in annualized revenue.
Cramer thinks that's too long to wait and see if the program will work.
"I liked the old Zillow, even if it was slowing down a bit, as it was a lot less risky," the "Mad Money" host said. "I'm not yet sold on the new Zillow, even with a much better class of CEO."
Barton returned to Zillow as CEO after co-founder Spencer Rascoff stepped down in February after 10 years in the role. Zillow announced the home-flipping concept nearly a year ago and the stock has taken a number of tumbles since.
The company wants to buy 5,000 homes a month, which could be an expensive endeavor, Cramer said. Zillow, which has dominated its industry, faces further pressures because its core business is not growing as fast as it once did, he added.
Cramer thinks the pivot to investing in houses was ill-advised, citing British investment bank Barclays' downgrade of the stock from hold to sell last month. On Monday, the American investment bank Cowen upgraded the security from market perform to outperform, throwing its faith behind the new leadership. But Cramer said it will be a wait-and-see gamble.
"Maybe Rich Barton can orchestrate a phenomenal turnaround at Zillow," Carmer said. "I'm glad he's running things again. But I think this headlong rush into the house-flipping business could prove to be very risky and, even if it works, there will absolutely be some brutal speedbumps along the way."
In October, Cramer said the new venture is not what Zillow's shareholders signed up for.
"Investors ... don't want an interest-rate-sensitive company that owns homes; they bought Zillow because it's a high-margin, asset-light online real estate play with a fabulous multi-year growth story," he said then.
In February, RBC Capital Markets' internet analyst Mark Mahaney said Barton, who was behind the launches of Expedia and Glassdoor, is the right choice to get the job done. But his firm also thinks it's a risky move.
"It's just, it is a big swing, it's a big risk," Mahaney told CNBC. "I want to see him bring his magic back to the company before we get constructive on the shares."
Zillow's shares are up more than 16% in 2019, but is down nearly 30% since this point last year. The stock dipped 1.19% Tuesday.