Let's look at an example product dataset. There are two files for each image. The first file is the image itself, like the one below. How to use computer vision to automatically generate structured data You can view it in Google Colab using the following Python line. advertisement Continue reading below Import numpy as np Import cv2 img = cv2.imread ("img1063.jpg", cv2.IMREAD_COLOR) importcv2_imshow from google.colab.patches cv2_imshow (img) The second part is the corresponding DetectNet text file labeled. You can check it on the command line. !! cat imag1063.txt Main product imagesThe screenshot has a row for all the labeled objects. This line contains the label name and the coordinates of the bounding box.
Let's see what this particular example ghost mannequin effect looks like when loaded into AutoML. How to use computer vision to automatically generate structured data You can see that the structured data elements are carefully labeled in the bounding box. advertisement Continue reading below Product name, price, quantity, reviews, description, etc. Before uploading the example, I had to convert it to the format expected by AutoML. I also had to upload the image to a Google Cloud Storage bucket, which is easy. The format is: set, path, label, First, I read the files from the disc and created a list of dictionaries, one for each bounding box. The predictions are very good. The model just missed reviews and prices. advertisement Continue reading below
This means that this example has a high precision rate and a low recall rate. You can improve that by increasing the number of examples labeled with price and reviews. There are few reviews, but there are many price examples. But perhaps this price doesn't fit into the existing pattern. The next part is probably the most interesting. How can I convert the coordinate predictions of these bounding boxes into structural data that can be inserted into HTML? Convert bounding box predictions into structured data First, you need to take screenshots automatically from Python. You can use Pyppeteer for that.