The chip may be required to help Apple power software in self-driving cars and augmented reality. It could also help improve Apple's Siri voice assistant on a local level.
Bloomberg said that Apple "plans to eventually integrate the chip into many of its devices, including the iPhone and iPad." Apple has already tested a version of an iPhone with the chip inside, but we might not see that in September when Apple launches its new iPhones.
Google already offers a dedicated AI processor known as the Tensor Processing Unit. Qualcomm has similar technology inside existing mobile processors.
Apple may discuss its AI plans during WWDC in June, Bloomberg said.
Apple declined to comment to CNBC.
CNBC's Josh Lipton contributed to this report.
Watch: Put your money in Apple or Facebook?