Get Image Descriptions: Google uses AI to describe what's in images

 

Google has taken its new Get Image Descriptions system a step further when it comes to viewing and interpreting the images we include in our articles and using them in its image search.

 

While it's true that the ALT attribute helped algorithms understand what was hidden in the image , artificial intelligence continues to improve and is now able to interpret what is seen in the image without any ALT attribute, with Google itself automatically incorporating it into its search results.

 

Get Image Descriptions from Google

 

In a process of making it easier to reach all types of people, Google realized that screen readers for the blind, or those with severe vision problems, were unable to interpret images without an Alt attribute or " Alternative Text " since there are hundreds of millions of images on the Internet without this attribute, thus depriving users of this information.

 

To improve this experience, Google has created an automatic image description feature called Get Image Descriptions from Google, so your screen reader can automatically create one. Currently, this feature will only be available in the Chrome browser.

 

While far from perfect, Google itself notes on its blog that automatically generated descriptions aren't as good as those written by humans, even though they include additional context, but that if you understand the context in which they're presented, they can be accurate and useful. Undoubtedly, an image description can help a blind person read a restaurant menu or better understand what their friends are posting on social media .

 

The AI system operates on Google's servers. This means that the computer or device used to read the image will not be the one to perform the task of understanding what's in the image .

 

Get Image Descriptions helps prevent overloading our own hardware.

Official Google Blog

 

Google sends the image to its server and returns it, adding data. The workflow will be a learning process, and it is hoped that over time it will improve its "intuition" when displaying the resulting image.

 

The AI algorithm will first search for text in the image itself , including signs, labels, and handwritten words. If it can't find any, the next system will step in, searching for recognizable objects it has been programmed to identify, such as a pencil, a tree, a person in a business suit, or a helicopter. If these systems fail, the more sophisticated system will step in , reading and trying to understand the main idea of an image .

 

The description is evaluated for accuracy and valuable information: Does the annotation describe the image well? Is the description useful? Based on whether the annotation meets those criteria, the machine learning model determines what to show the person. The algorithm will only provide a description if it believes it to be reasonably accurate. If the algorithm is conflicted and doesn't know what to describe, it will do nothing and will not act immediately.

 

Get Image Descriptions : Examples of AI in Google Images

 

Here are a couple of examples of the actual descriptions generated by Chrome when used with a screen reader.

 

 

Pineapples, bananas and coconuts
Machine-generated description for this image: " It looks like fruits and vegetables at the market ."

 

 

Man playing guitar on a gray sofa
Machine-generated description for this image: " It looks like a person playing guitar on the couch ."

 

Outstanding Tests and Increasingly Effective AI

 

During recent testing, over 10 million descriptions have been created, with hundreds of thousands more being added every day.

 

For now, the feature is only available in English, but according to Google itself, more languages are planned to be added soon.

 

The search also confirms that image descriptions in Chrome are not intended to replace the creation of the website responsible for that image ; remember, Google always encourages developers and authors to follow SEO best practices and provide image descriptions on their sites.