Here's how Aldi and Walmart prices compare on pantry staples like milk and bread

Locations of Aldi and Walmart in New Jersey
Megan Leonhardt | CNBC

It's after the holidays and the shelves in your refrigerator and pantry may be looking a bit bare. Time to do a little grocery shopping!

If you're looking to stock up without breaking the bank, both Aldi and Walmart sell inexpensive pantry staples. That works for most people since, on the whole, shoppers are looking for good quality at a good price, says John Karolefski, store analyst and editor of Grocery Stories.

But which store offers the best deal on staples?

Using the grocery price comparison app Basket, CNBC Make It compared a dozen essential items such as milk, bread, eggs, olive oil and flour at the Aldi and Walmart stores in North Bergen, New Jersey. When available, we also analyzed both retailers' organic store brand prices.

Note: Aldi's Simply Organic canned diced tomatoes only sold as a 28 oz can

Aldi is cheaper overall on eight of the 12 products: It would cost under $17 to restock these kitchen essentials there.

"Aldi, happily, has a lot of good quality, good-tasting products at good prices," Karolefski says. "It's one of the reasons they've been so successful in the U.S."

Stocking up on the same products at Walmart is only a few bucks more than Aldi, though, for a total of just over $20, and the store has cheaper prices on bread, milk and eggs. Walmart offers shoppers a one-stop shopping experience, too: If you pop in, you can pick up groceries and an emergency toilet plunger, or a replacement pair of mittens for your forgetful kid.

Organic products

But what about quality? Last year, the Organic Trade Association found that over 82 percent of U.S. households buy organic groceries on a regular basis.

When it comes to organic products, both Aldi and Walmart offer great prices. In fact, Whole Foods, which is perhaps the best-known organic grocery store, is facing "steep competition, and their pricing is just not comparable," Cindy Livesey, author of "Living Rich with Coupons," tells CNBC Make It.

Aldi's SimplyNature brand edges out Walmart's Good Value Organic brand when it comes to price. And, besides being organic, frugal shopping expert Lauren Greutman reports that the SimplyNature products omit over 125 ingredients that experts have deemed questionable, including artificial flavors, high fructose corn syrup, trans fatty acids, nitrates and propylene glycol.

But Greutman notes that while Aldi and Walmart are generally the cheapest option for many essential products, it pays to do your homework and check out the weekly ads and coupons offered by other shops. Greutman found in her comparison that products like dish detergent and fresh chicken are generally cheaper with a coupon at a conventional grocery store.

Also, Greutman says, Walmart will generally price-match competitor ads — so if Aldi is advertising a cheaper price, take the ad into a nearby Walmart and you could get the lower price on your items.

Don't miss: 6 foods you should never buy at Whole Foods

Like this story? Subscribe to CNBC Make It on YouTube!

Where to buy Thanksgiving dinner for 8 for under $50
Where to buy Thanksgiving dinner for 8 for under $50