Two big problems in the job market

What is Fed Chair Janet Yellen still so worried about that she feels interest rates must remain so low for so long?

A sign  hangs in the window of a clothing store on June 6, 2014 in San Francisco, California.
Justin Sullivan / Staff | Getty Images News
A sign hangs in the window of a clothing store on June 6, 2014 in San Francisco, California.

The unemployment rate has come down from the crisis high of 10.0 percent (October 2009) to the current rate of 6.2 percent. We have generated an average of 230,000 jobs a month so far in 2014. Consumer sentiment has improved dramatically, the stock market is way up, the housing market has rebounded, and banks are finally beginning to scale up their lending activities again. What's not to like?

In a nutshell, the answer is twofold: 1) there are not enough people working; and 2) those that do have jobs are not making enough money.

Read MoreWhy America has a shortage of truck drivers

Let's look at payrolls first. It is true that the absolute number of "employed" people has now finally surpassed the previous record set in 2007. According to the Bureau of Labor Statistics, 146.4 million Americans were considered employed in July 2014, compared to 146.0 million in 2007. This is indeed an important milestone. However, the problem is that the Civilian Noninstitutional Population (which includes people residing in the U.S. who are 16 years or older and who are neither inmates of institutions nor active duty in the Armed Forces) rose by over 16 million during this time frame.

So yes, millions of active job-seekers still cannot find work, which is why the unemployment rate remains high. But many millions more have stopped looking for a job either out of frustration or due to some other reason. What does this mean? It means that we are well short of our potential with regard to the productive capacity of our labor force.

But perhaps the bigger issue with regard to the labor market is a trend that had been firmly established well before the financial crisis. Middle-class incomes have not been growing on an inflation-adjusted basis. In fact, median household income in 2012 was the lowest since 1995. We will not get a data point for 2013 from the Census Bureau until the middle of next month, but there is little reason for much optimism.

Many of the jobs that have been created since the financial crisis have been service-related jobs that don't pay as much as the manufacturing jobs in years past. In addition, many job-seekers are finding they are only able to find part-time work when they would really rather have a full-time job. And finally, the large pool of available labor has meant employers felt no obligation to increase wages for existing employees. These factors and others have weighed heavily on middle-class incomes over the past few years.

Read MoreHow to deal with Jackson Hole 'rate rage'

Lower inflation-adjusted incomes means less money to spend in an economy that is heavily dependent on consumer spending.

And this does not even consider the fact that the government's official estimates of inflation are very flawed. The expenses that really matter to middle-class folks - health care, education, housing, food and energy - are not given the proper weighting in the official estimates of inflation. But these are the expenses that consume the lion's share of middle-class paychecks. And as we have mentioned many times in the past, asset prices must be taken into consideration as the Fed formulates monetary policy. Why? Because a major expense for middle-class Americans is saving for retirement. If asset prices are heavily inflated, the costs of saving for retirement increases dramatically through lower expected returns in the future.

With lower incomes and higher expenses, it's no wonder that this has been one of the weakest economic recoveries on record. There is simply not enough demand for goods and services coming from the middle class. Wealthier folks have certainly been spending, but even the rich can't move the needle enough to drive a more robust economic recovery.

We have long held that the Fed's continuing monetary support amounts to pushing on a string while creating a whole new set of risks. In fact, Yellen herself seemed to acknowledge these sentiments in her speech last Friday when she said, "Judgments concerning the size of that gap (the difference between the current labor market and "maximum employment") are complicated by ongoing shifts in the structure of the labor market and the possibility that the severe recession caused persistent changes in the labor market's functioning."

If it is true that structural impediments are the cause for the ongoing labor-market weakness, then the Fed may be impotent at this point. And if the Fed is unable to restore the labor market to its pre-crisis glory, why is it unnecessarily adding to systemic risks?

Read MoreThe robots are coming! The robots are coming! Will they take your job?

Through its monetary policy, the Fed is effectively encouraging the irresponsible assumption of debt (as a response to the problem of too much debt), inflating asset bubbles and exacerbating the disparity in wealth and income between the rich and the middle class. In my opinion, that doesn't make much sense.

It's time for the Fed to get out of the way.

Commentary by Michael K. Farr, president of Farr, Miller & Washington and a CNBC contributor.