Here's a taste of how much smarter your iPhone will be in September

Share
×

Tech Guide

Here's a taste of how much smarter your iPhone will be in September

  • Apple's new machine learning tools allow developers to create incredible new apps
  • You can identify objects, like food or even rashes
  • The possibilities are endless, take a look at what's coming this fall

Apps will soon leverage new tools from Apple to identify everything from a bowl of almonds to a poison ivy rash.

CNBC met with Apple on Wednesday to take a look at some of the first applications that will leverage this technology. It's pretty incredible, and Apple is doing all of the processing on the device, which means nothing is sent out to the cloud.

Take a look at what's possible.

This app is called Lose It!,  and it's meant to help you lose weight by making wise eating choices. With iOS 11, you'll be able to point your phone like this bowl of spaghetti, and LoseIt will identify it.

CNBC Tech: Machine learning
Todd Haselton | CNBC

Or this banana.

CNBC Tech: Machine learning 2
Todd Haselton | CNBC

Or this bowl of almonds.

CNBC Tech: Machine learning 3
Todd Haselton | CNBC

To be clear, the app isn't new. But before, you had to scan barcodes or manually enter what you ate.

With iOS 11, Lose It! can identify what's exactly on the plate in front of you. This information isn't being sent to the cloud for processing, and developers didn't have to build the machine learning engine themselves. Apple did the work that allowed Lose It! senior data scientist Dr. Edward Lowe to develop the deep-learning framework you see at play, all without an internet connection.

There are other use cases, like in medicine.

This app is called Visual DX. It's already used by doctors and hospitals around the world. The new version is smarter.

CNBC Tech: Machine learning 4
Todd Haselton | CNBC

This is a rash that a patient might ask his or her doctor about. The doctor can take a picture and the app can identify it even without an internet connection.

CNBC Tech: Machine learning 5
Todd Haselton | CNBC

Here we see it recognizes a rash that's likely on a leg.

CNBC Tech: Machine learning 6
Todd Haselton | CNBC

After a couple of seconds it presents the results, with the most likely ailment listed at the top. Looks like this is a classic case of poison ivy.

CNBC Tech: Machine learning 7
Todd Haselton | CNBC

This function wasn't available at all in prior versions of the app. You had to search through symptoms and dig through multiple menus to find what the ailment might be.

These are just two examples of updated apps that are coming this fall. Apple showed us another app that's not quite finished that can detect a face so that, when you take pictures, the person in the photo automatically receives a copy of the image.

The possibilities seem pretty endless, and that's just a small taste of what machine learning can do.