At the same time, he says the notion that a digital overlord will control us or that robots will take our jobs is nonsense.
Bezos shared his thoughts about artificial intelligence at the George W. Bush Presidential Center's Forum on Leadership in April.
"I think autonomous weapons are extremely scary," said Bezos.
The artificial intelligence tech that "we already know and understand are perfectly adequate" to create these kinds of weapons said Bezos, "and these weapons, some of the ideas that people have for these weapons, are in fact very scary."
Experts predict malicious AI attacks could include speech synthesis for impersonation; analysis of human behaviors, moods and beliefs for manipulation; automated hacking and physical weapons like swarms of micro-drones.
Bezos said there ought to be some oversight. "I don't know what the solution" should be, he said, "but, smart people need to be thinking about that."
It could perhaps be modeled after the Geneva Convention, suggested Bezos, which is a series of publicly recognized international laws protecting medical workers in conflict regions and prisoners of war.
"It would have to be a big treaty," he said — "something that would help regulate these weapons, because they're actually, they have a lot of issues.
"So that one I think is genuinely scary," reiterated Bezos.
On the other hand, Bezos is not concerned that some omnipotent artificial intelligence will conquer humans.
"The idea that there is going to be a general AI overlord that subjugates us or kills us all, I think, is not something to worry about. I think that is overhyped," said Bezos.
"First of all, we are nowhere close to knowing how to build a general AI — something that could set its own objectives," the Amazon CEO said.
Right now we have "narrow AI," explained Bezos, where machine intelligence assists in a specific task. Billionaire tech titan Elon Musk has made a similar distinction: Narrow, functional artificial intelligence is what is used in self-driving cars, according to Musk, while general AI "literally has a million times more compute power and an open-ended utility function."
Further, continued Bezos, "I think it is unlikely that such a thing's first instincts would be to exterminate us. That would seem surprising to me."
Instead, Bezos said it is "much more likely it will help us. ... We are perfectly capable of hurting ourselves. We could use some help. So I am optimistic about that one and certainly don't think we need to worry about it today."
In contrast, Tesla and SpaceX CEO Musk said at the South by Southwest festival in Austin, "We have to figure out some way to ensure that the advent of digital super intelligence is one which is symbiotic with humanity. I think that is the single biggest existential crisis that we face and the most pressing one."
And according to the late, legendary physicist Stephen Hawking, "Success in creating effective AI, could be the biggest event in the history of our civilization. Or the worst. We just don't know. So we cannot know if we will be infinitely helped by AI, or ignored by it and side-lined, or conceivably destroyed by it," he said in November. "Unless we learn how to prepare for, and avoid, the potential risks, AI could be the worst event in the history of our civilization. It brings dangers, like powerful autonomous weapons, or new ways for the few to oppress the many. It could bring great disruption to our economy."
Bezos is also optimistic about the potential impact robots will have on human workers.
"And then the jobless.... Is AI going to put everybody out of work? I am not worried about this," said Bezos.
However some recent reports give cause for concern. A report published in March by the Organization for Economic Co-operation and Development (OECD) predicts 14 percent of the jobs measured in 32 OECD countries, or about 66 million jobs, are "highly automatable," which the report defines as having a probability of being automated over 70 percent over the next 15 to 20 years. Another 32 percent have a risk of being automated at 50 to 70 percent, the report says.
But Bezos believes our fear of being put out of work by robots is a failure of imagination more than anything else. "I find that people, all of us, I include myself, we are so unimaginative about what future jobs are going to look like and what they are going to be," he explained.
For example, said Bezos, if a convention of farmers a century ago were to learn that "massage therapist" would be a job, they would have been incredulous. "And in fact, I was telling this story to a friend, and they said, 'Jeff, forget massage therapist, there are dog psychiatrists,'" he said. "I went and looked that up the Internet. Sure enough you can easily hire a psychiatrist for your dog."
In fact, Bezos said artificial intelligence will improve the quality of jobs.
"Humans like to do things and we like to be productive and we will figure out things to do and we will use these tools to make ourselves more powerful," he said. "What I predict is that jobs will get more engaging. Because you have to remember, a lot of the jobs today are quite routine."
There are those who have jobs, those who have careers and then a lucky few who have a "calling," according to Bezos, and the rise of automation will allow more individuals to have more fulfilling professional lives.
"I predict that, because of artificial intelligence and its ability to automate certain tasks that in the past were impossible to automate, not only will we have a much wealthier civilization, but the quality of work will go up very significantly and a higher fraction of people will have callings and careers relative to today," said Bezos.
Like this story? Like CNBC Make It on Facebook.