J.C. Penney's sales per square foot have fallen precipitously over the past year, but Ullman added that the higher price point of some of the designer home goods would help boost the key metric.
"Of course, (home) is a big space commitment; it's expected to earn its way," he said.
Big Name Designers It's been a year since J.C. Penney revealed that Jonathan Adler, Michael Graves and Terence Conran would join previously-announced Martha Stewart as home design partners, and two months since the merchandise began showing up at stores and online.
In what J.C. Penney called its "biggest home launch ever," over 500 of its 1100 department stores now have new home "shops," and an additional 145 are selling the merchandise.
(Read More: Down, But Not Out: JC Penney CEO Meets Shareholders)
Graves is optimistic about the success his brand has seen at J.C. Penney during the soft rollout.
"We are really happy with selling 1,600 kettles last week, that's a phenomenal number, and we never did that at Target," he said.
The kettle he's referring to has been at the center of recent controversy – with some Twitter users saying an image of the tea pot on a billboard looked just like Adolf Hitler.
On Thursday, J.C. Penney kicked off its new home launch in a traditional way… with a sale. Select home department items have been reduced by up to 40 percent until June 16.
Some analysts expressed concern over the higher price points of some of the new designer home goods, but Ullman isn't concerned. He likened the home "attractions" (the term he uses instead of shops) to Sephora, the first shops he brought into J.C. Penney stores in 2006.
"Sephora today is the most successful thing in the store," he said. "It's the only thing that grew last year, and was profitable last year in the store."
"So I think many of these attractions will have the same effect. People are going to come for the core which they are used to at J.C. Penney, and they are going to start to notice these things - they are going to find out it's not as expensive as it looks."Connecting With Customers
The retailer has kicked the marketing into high gear for the home launch, trying to re-connect in a careful way with its core customer, while hoping to attract new shoppers as well.
"It's quite clear that we owe the customer time to regain her trust, and we have to speak to her at the same time. We can't be silent, but we can't just talk about things we can't deliver on," he said. "So it's a process of getting back in the business of speaking to her, in tone that she appreciates. We are not preaching, we are not teaching - we are more or less sharing."
(Read More: JC Penney C-Suite Feels Hit After Dismal Year)
Analysts and media had the opportunity to see the new home goods before most consumers – and investors are anxiously awaiting their response to the home revamp.
JPMorgan retail analyst Matthew Boss was impressed by the home shops.
"Personally, I was wowed by the stores in terms of the appearance, in terms of the product presentation, and the brands that are actually on the shelves," he said. "So I think when the consumers see this for the first time they are probably going to be transformed and not believe they are in a J.C. Penney."
The Significance of 'Home'
Former J.C. Penney CEO Ron Johnson called the home department revamp "pivotal" to the retailer's overall transformation. While Johnson is no longer at the helm, his successor - and predecessor- Ullman has seen the home launch to its ultimate fruition.
When Ullman re-joined the retailer on April 8, some of the new home shops and merchandise had just been launched. That puts Ullman is an interesting position: leading the company through a home launch vision that wasn't his.
"I'm responsible no matter what happens. That's what you sign up for," he said. "I have no difficulty being responsible. I'm responsible for the team, frankly. I bear the responsibility; I feel the responsibility for a lot of people's well being. That's what it's about. It's the reason I came back, people deserve a chance to be successful and I hope I can help in some way."
Ullman would not reveal how early sales of the new home goods were going, but on the first quarter conference call in May he told investors he was "encouraged by shopper reaction" since the retailer's "apology" and "thank you" commercials aired.
Martha, Martha, Martha
The fate of the Martha Stewart branded merchandise – which competitor Macy's claims to have exclusive rights to sell - still hangs in the balance.
The lawsuits Macy's filed against both J.C. Penney and Martha Stewart Living Omnimedia remains unresolved as the three legal teams wait for the judge to schedule a date for closing arguments.
(Read More: What's Behind the Rush Into the Low-Margin Grocery Business)
J.C. Penney is currently able to sell Martha Stewart branded products in the non-exclusive categories, but has to brand the products as "jcp EVERYDAY" for now.
However, New York Supreme Court Judge Jeffrey Oing's ultimate decision could change everything.
It's possible he could prevent J.C. Penney from selling any Martha Stewart merchandise in the categories Macy's is contesting - branded or unbranded.
A decision of that nature would leave only products in the so-called "non-exclusive" categories, like Martha Stewart branded disposable party goods, greeting cards and window dressings available for sale at J.C. Penney.
JPMorgan's Boss said that although losing the case would be the "worst case scenario" for JC Penney, it would not throw off the home department entirely.
"I don't think it would be a game changer, I don't think it would transform the overall appearance of the home shop, he said. "But Martha coming in is definitely one of the anchors, so it's something worth following."
When asked about the case at the J.C. Penney home launch, Martha Stewart told CNBC: "It's still ongoing. Until we know what Judge Oing finally determines, I have nothing to say."
(Read More: Target's Plan for Brides: Say Yes to the Dress)
Ullman added: "To try to figure out what the judge is thinking, it's probably too late for that. It's in his hands, and we will obviously react to whatever the decision is, and move on."
Stewart has been a very vocal supporter of former CEO Johnson's strategy. In addition to the merchandise partnership, J.C. Penney took a 16.6 percent stake in Martha Stewart Living Omnimedia.
When asked for her thoughts on the move back to sales and coupons, Stewart added: "I joined because Ron Johnson was there, with a new concept. It was a very appealing concept. Now it's changed and I have yet to see the success of it. So, we are looking forward to it."
She's not the only one… with many others betting that J.C. Penney can bring it home.
—By CNBC's Courtney Reagan. Follow her on Twitter
Questions? Comments? Email us at firstname.lastname@example.org.