This article, which will take you 5 minutes to read, was written by 2 graduate students in Applied Artificial Intelligence at the Erasmus University of Applied Sciences and Arts in Brussels. Together, we started our four-week internship on March 14th at Micropole Belux.
Summary
During our internship, we created a pool detection model. We built this model based on satellite images we found on the web. Since we were working in Micropole's Microsoft Azure team, we used the Microsoft Azure Cloud platform for all our tasks.
Step by step presentation of the creation of a pool detection model
Where and how did you get the data?
The first step in the process was to collect data or extract images from the zoom.earth website. Using a pre-existing dataset with all addresses in Belgium, we were already able to start extracting data, as it contained not only the address, but also the longitude and latitude of the location. The URL zoom.earth in your browser's search bar had these features (longitude = black rectangle and latitude = yellow rectangle), so we were able to use them to direct us to the correct URL.
We decided to explore the Azure Cloud Platform and started using the Azure Functions component. After doing some digging in JavaScript, we wrote local code that was able to navigate to the URL of the specified location and then take a screenshot of that web page. The next step was to removeunnecessary elements that were visible in the screenshot and resize the web page before taking the screenshot as seen in the satellite image below.
After running our local JavaScript code, we got the following result:
This is where we started to run into trouble, as we wanted to save the webpage screenshot to an Azure storage account (the file system used in Azure) while running it locally. We couldn't figure out how to get there with this approach, and since we only had 4 weeks for this assignment, we started exploring other options.
Our second option was Selenium, a testing tool for the web. By using this library in Python, we could do exactly the same things as in JavaScript. However, the main difference was in the image storage. Since we were no longer using Azure Function, but had switched to Azure Databricks, we were able to create our storage account in Notepad and save our screenshots to that directory. After adapting our JavaScript code, we got the same result as the image above.
How do we label the data to train and test our model?
After collecting all the data, we had to assign relevant information to this raw data (in this case, the images) by labeling them.
During the labeling process, we had to define 2 labels:
- pool (1) or
- no pool (0).
When there was a pool, we assigned label 1, and when there was not, label 0. We made sure that all images with the same label were grouped in the same subdirectory, so that they could be used later for data analysis in the next step.
How do you teach a computer what a pool is?
For data analysis, we used a neural network, and more specifically transfer learning. This type of neural network takes a pre-trained model that could be relevant in different situations, in our case image detection, and applies it to this new situation (pool detection). We used the Residual Network, or ResNet, as a pre-trained model for pool detection. These networks have a depth of 18 to 152 layers. The more layers, the more complex the pre-trained model.
After training the neural network on our dataset, we noticed that the results were not as good as we had hoped. Because of these disappointing results, we decided to take another approach.
This time, we did not use machine learning algorithms, but we filtered out the correct blue color from each image, as shown below. When there was enough blue in an image, we gave it the label 1 (pool). This model has an accuracy of 85% and can correctly predict the presence of a pool 80% of the time and the absence of a pool 90% of the time . The drawback is that it cannot predict indoor pools.
Conclusion
Satellite imagery can be collected in an Azure cloud environment by Selenium via Databricks. Azure ML can be used to label our dataset and eventually train a model to make predictions.
Trying out the different approaches was a great learning experience. We had a lot of fun doing this project and enjoyed the discussions with colleagues. We thank Egon, Kenny, Esa and Antoine in particular for helping us with this project and for kindly answering our questions.
Want to know more?
Let's talk about it!
