However, the path to normalizing rates is not an easy one, he said, arguing the next five years are going to be as complex to navigate as the past five or 10 years.
"There are lots of reasons for that: slowing global growth, aging population in the United States and advanced economies, high levels of debt to GDP in all advanced economies," he said.
Kaplan sees an interest rate hike in the not-too-distant future, provided gross domestic product numbers recover as he believes they will. "I think people should expect it's going to be a slow, patient, gradual normalization."
In a speech Monday, Kaplan said it would be too soon to hike interest rates this month, but he expressed an openness to a possible increase in June. He's not a voting member of the Fed's policymaking committee this year.
Many economists expect a June rate hike, but the futures market has been more cautious, looking to perhaps December as the next tightening.
The Fed raised rates for the first time in more than nine years in December. At the time, central bankers predicted four hikes for 2016, but after holding steady in March, policymakers projected only two increases this year.
The December rate hike was not a mistake, Kaplan told CNBC, saying negative interest rates in Europe and Japan is a fourth reason for the Fed to be cautious. "We not an island. Slowing global growth and also strong divergence has some effect on us."
Monetary policy can't fix everything, he said, calling for Washington to get past the gridlock of recent years and put forth fiscal policy.
He said there's a cost to excessively low rates such as squeezing savers looking for returns and forcing investors to take on more risk.
Kaplan took over at the Dallas Fed in the fall, following the retirement of Richard Fisher, who served 10 years at the helm. During his 23 years at Goldman Sachs, Kaplan served in various capacities, including vice chairman.