"Even though the merchandise has become better and the store experience has improved and the inventory has been cut, they lost that connection with that core customer over the last 12 months," he added.
Analysts had widely criticized Johnson's decision to move away from sales and discounts as being out of touch with the JC Penney customer. During 2012, comparable store sales slid 25 percent as value-seeking customers turned away from the new strategy.
(Read More: Ron Johnson's Downfall: 'Hubris Finally Did Him In')
To win back the customer, Ullman plans to emphasize the value of its merchandise and set a goal of having a competitive offering by the crucial back-to-school season. For many retailers, back-to-school sales are the second most important shopping period after the holiday season.
"CEO Ullman spoke to BTS (back to school) as the test if the business is back in a way that the customer understands," Boss said. "This will likely be measured by sales trends, with some measure of growth in traffic. Back to school is competitive, and JC Penney will need to have a clear strategy in place and a message to their customers by then."
JC Penney's e-commerce performance was especially poor during Johnson's tenure and dropped 33 percent during fiscal year 2012 from the year-before period. While the company will be working to address this, it would be unrealistic to expect a fix in this category by back to school, the note added.
After JC Penney completes the transformation of its home goods department in May, Ullman plans to step back from Johnson's costly plan to transform the department store into a store containing many shops. The company will then regroup as it continues to craft a strategy to better resonate with customers, the note said.
(Read More: Was Ackman's Big Idea for JC Penney Too Big to Succeed?)
Ullman told analysts that it should "not be a question of how many shops JC Penney can put in, but how many the customer will value."
Analysts have also expressed concern about the company's balance situation. To fund its transformation, the company has used up a considerable amount of its cash reserves and ended the fourth quarter with $930 million in cash.
While Ullman told JPMorgan that it won't be 'going on a diet,' he said it would be reasonable to assume capital expenditures would undergo a haircut.
Boss expects to hear more about the balance sheet in the coming weeks.
"I expect that probably over the next four to six weeks we'll hear something further on the balance sheet, but to Ullman's credit it seems like he was only contacted over this past weekend to take the job, so I think it's a little early for him to comment on the road map ahead," Boss told CNBC's "Squawk on the Street."
JPMorgan currently has a neutral rating and a $15 price target on the company's stock.
Boss said "a clear strategy needs to be outlined on how JCP plans to regain lost customers, and drive future profitability" before he thinks the company will be a worthy investment. "While high-low pricing is a start, the company needs to communicate a clear roadmap on where they go from here in terms of sales, margins and stabilizing the balance sheet."
-By CNBC's Katie Little; Follow her @katie_little_
Questions? Comments? Email us at consumernation@cnbc.com.