Food in Visual Look Up ⇥ macstories.net
With Visual Look Up, you can identify and learn about popular landmarks, plants, pets, and more that appear in your photos and videos in the Photos app. Visual Look Up can also identify food in a photo and suggest related recipes.
Meal identification is new to iOS 17, and it is a feature I am not sure I understand. Let us assume for now that it is very accurate — it is not, but work with me here. The use cases for this seem fairly limited, since it only works on photos you have saved to your device.
Federico Viticci, in his review of iOS 17, suggests two ways someone might use this: finding more information about your own meal, or saving an image from the web of someone else’s. One more way is to identify a meal you took a picture of some time ago and may have forgotten what it was. But Visual Look Up produces recipes, not just dish identification, so that suggests to me that this is to be used to augment home cooking. Perhaps the best-case scenario for this feature is that you stumble across a photo of something you ate some time ago, get the urge to re-create it, and Siri presents you with a recipe. That is, of course, assuming it works well enough to identify the meal in the photo.
Viticci:
Except that, well, 🤌 I’m Italian 🤌. We have a rich tapestry of regional dishes, variations, and local cuisine that is hard to categorize for humans, let alone artificial intelligence. So as you can imagine, I was curious to try Visual Look Up’s support for recipes with my own pictures of food. The best way I can describe the results is that Photos works well enough for generic pictures of a meal that may resemble something the average American has seen on Epicurious, but the app has absolutely no idea what it is dealing with when it comes to anything I ate at a restaurant in Italy.
Siri struggles with my home cooking, too, often getting the general idea of the dish but missing the specifics. A photo of a sweet corn risotto yielded suggestions for different kinds of risotto and various corn dishes, but not corn risotto. Some beets were identified as different kinds of fruit skewers or some different Christmas dishes; the photo was taken in August.
In many places, getting the gist of a dish is simply not good enough. The details matter. Food is intensely binding — not just among a country, but at smaller regional levels, too. It is something many people take immense pride in. While it is not my place to say whether it is insulting that Siri identified many distinct curry preparations as interchangeable curries of any type, it does not feel helpful when I know the foods identified are nothing like what was actually in the photo.
Update: Kristoffer Yi Fredriksson emailed to point out how Apple could eventually use food identification in its health efforts; for example, for meal tracking. I could see that. If it comes to pass, the accuracy of this feature will be far more important.