Franz-Waldner

Franz Waldner

 

 

[Image of a split circle appears with photos in each half of the circle flashing through of various CSIRO activities and the circle then morphs into the CSIRO logo]

 

[Image changes to show a new slide showing a split circle with two satellite maps in either side and Franz Waldner can be seen inset in the top right talking and text appears: Detect, consolidate, delineate: scalable, mapping of field boundaries using satellite images, Franz Waldner, Foivos Diakogiannis and the whole ePaddocks^TM mob]

 

Franz Waldner: Good day everyone, and thanks for having me today. My name is Franz Waldner and I work at Digiscape ECR. I’m sorry I cannot join you live today. The time difference with Italy where I am currently based makes it a bit difficult. Right, some of you might have heard about ePaddocks. For those who have not, ePaddocks is a national layer of field boundaries that was produced within Digiscape and in this presentation I will tell you everything about the science behind ePaddocks but before I start I would like to acknowledge Foivos Diakogiannis and everyone who contributed and supported ePaddocks.

 

[Image changes to show a new slide showing a red dot pattern on the left and Franz Waldner can be seen inset in the top right talking and text appears: Field boundary data serve a range of applications, Field-level digital products, Improved land-use maps and crop identification, Indicator of human development, mechanisation, species richness]

 

Why do we need field boundaries? Well, field boundary data serves a range of applications. Think of field-level digital products like services providing farmers with analytics to help them manage or monitor their crops. Think also about, you know, ways that field boundaries can be used to improve the land use maps or help with crop identification and also field boundaries can be used as an indicator for human development, agricultural mechanisation or species richness and I’m sure you can also think about many more applications. Digital services currently ask their users to manually draw field boundaries which is time-consuming and creates many disincentives. So, our job was to create a method to automatically generate field boundaries and for that we use satellite imagery and deep learning.

 

[Image changes to show a new slide showing symbols of three cogs, a drink, and a pinpointed map of Australia and Franz Waldner can be seen inset in the top right talking and text appears: An old problem tackled with new science, To design a method to delineate paddock boundaries with three features, Data driven – No local assumptions, Limited pre-processing – Applicable to any image, Scalable – Inference over large areas]

 

Extracting field boundaries from satellite imagery is not a new problem but we tackled it with new science.

 

[Image changes to show a new slide showing multiple boxes showing satellite imagery and data mapping and Franz Waldner can be seen inset in the top right talking and text appears: The DECODE approach to extract field boundaries, DECODE (Detection-COnsolidation-DElineation), automatically extracts accurate field boundary data from satellite imagery based on spatial, spectral and temporal cues, DECODE is a three-step method – 1. We Detect fields in single-date images at the pixel level using state-of-the-art-deep-learning methods, 2. We Consolidate model outputs by averaging all single-date model predictions, 3. We Delineate fields by applying a hierarchical watershed segmentation algorithm on the consolidated model outputs]

 

So we came up with DECODE. DECODE stands for DEtection, COnsolidation and DElineation. And DECODE automatically extracts field boundaries from satellite imagery based on spatial, spectral and temporal cues. It’s a three step method. In the first step we detect fields in single-date images at the pixel level using a state-of-the-art deep-learning method. In the second step we consolidate the model outwards by averaging them, and in the last step we delineate the fields by applying a hierarchical watershed segmentation algorithm on to consolidated model outputs.  

 

[Image changes to show a new slide showing a diagram showing the input image moving through the FracTAL ResUNet to the distance mask, boundary mask and extent mask and photos appear at the bottom of various masks and distances and text appears: Detection with FracTAL ResUNet, FracTAL ResUNet is a state-of-the-art deep convolutional neural network with the following characteristics, a UNet encoder-decoder architecture, A Pyramid Scene Parsing Pooling algorithm to extract multi-scale contextual features, Skip connections to maintain information flow to the deepest layers of the network, Conditioned multi-tasking to predict multiple outputs, Attention modules]

 

Let’s have a better look at the detection step with FracTAL ResUNet. So Fractal ResUNet uses as input satellite imagery and in our case we used satellite imagery with four bands, the blue band, the green band, the red band, and the near-infrared band. It does its magic and outputs three layers – the distance mask, the boundary mask, and the extent mask, and the rationale for outputting multiple predictions is two-fold.  The first one is that the model can learn better or faster when it is asked to predict more things, but also by predicting more outputs we have more outputs also at hand to digitise and delineate field boundaries. I’ve listed a set of features of the algorithm for the deep-learning enthusiast but I will go quickly on that and if you want to, to learn more I’ve listed also the papers at the end of this presentation.

 

[Image changes to show a new slide showing a line graph and various data maps and text appears: Delineation with hierarchical watershed segmentation, Watersheds are defined as groups of pixels from which a drop of water can flow down towards distinct minima, The dynamics of a path that links two pixels is the difference in altitude between the points of highest and lowest altitude on that path, In an image, the dynamics of two pixels is equal to the dynamics of the path with the lowest dynamics]

 

Moving on now to the delineation step. The reason we need a delineation step is that if you simply use the boundary mask to define your boundaries without any additional pre-processing there is a risk that the boundary would not be closed and that you’ll lose a lot of paddocks and information in the process. So we had, we need the step to make sure that all our paddocks have closed controls. So the idea of the watershed segmentation is basically to create a network between all the pixels in the image. And pairs of pixels are related by or are defined by a single value which is called the dynamics. And the dynamics is basically defined as the magnitude between or the altitude or the difference in values, if you want, between the highest and lowest point in the image. And you can apply that across your image and you’ll populate the values in your network with this. The last step then is to define where to cut these networks to define individual fields and that is a process that you can fully automate using some reference data.

 

[Image changes to show a new slide showing a map of Australia, a map of South Africa, and some number data points for each scenario and text appears: Comparing two scenarios, Cloud-free Sentinel-2 single-date images (10 m, B, G, R, NIR), Training data from South Africa or Australia]

 

Coming up now to our experimental framework, so we tested our approach in two ways. The first way is that we used trained data from Australia, which we call the target-to-target approach, and then we used data from South Africa where we had a lot more trained data.

 

[Image changes to show a new slide showing a map of Australia, and a Source to target data model, and a Target to target data model and Franz Waldner can be seen inset in the top right talking and text appears: Detection with Fractal ResUNet, Fractal ResUNet provides a reliable detection of fields in a range of landscapes, including in dry areas with poor crop establishment, Source-to-target and target-to-target models on par]

 

And if we look at the detection step, there are basically two main takeaways here. The first one is that we’ve got a pretty good detection of fields across the landscapes, but also the second thing is that the source-to-target and the target-to-target models are on par, which is quite interesting from an application point of view because it means that a model that has never seen an Australian field can perform equally well to a model that was exclusively trained on Australian fields.

 

[Image changes to show a new slide showing six line graphs and then a bar graph comparing source to target and target to target data and Franz Waldner can be seen inset in the top right talking and text appears: Source to target is more accurate than target to target]

 

Now looking on at the field level results, so we defined six metrics that characterise accuracy at a field level for different aspects, different ways, you know, matching between the boundaries, the second one is looking at the similarity in locations between the fields or similarities in shape. And we found that interestingly the source-to-target model was in many cases better than the target-to-target model, or at least was equally, or performed equally to the target-to-target model, and again that implies that the model has a really good generalisation capability and it can be applied across a range of regions.

 

[Image changes to show a new slide showing Sentiel-2 image and reference data, Extent mask, Boundary mask, Distance mask, Reference boundaries, Extracted boundaries, and Agreement maps for different states and a colour key can be seen at the bottom and Franz Waldner can be seen inset in the top right talking]

 

Here you’ve got a set of examples across the country. Most areas would be small areas located around the boundaries but the main thing to, to see here is that across landscapes you’ve got a pretty good delineation regardless of the complexity of those landscapes.

 

[Image changes to show a new slide and Franz Waldner can be seen inset in the top right talking and text appears: Summary, DECODE is a scalable approach to delineate field boundaries using satellite imagery, DECODE first detects fields on single-date images, Next, it builds consensus across time by averaging model predictions for different dates, Finally, it delineates fields from the consensus predictions and ensures all contours are closed, DECODE relies on a state of the art deep learning model, Its generalisation and transfer abilities can offset the need for training data, Uptake by several institutions and organisations]

 

So to summarise, DECODE is a scalable approach to delineate field boundaries using satellite imagery. It’s a three-step process. The first one is to detect fields and their boundaries at the pixel level before using single-date imagery, and then you consolidate those predictions by averaging across the temporal dimension, and then finally you delineate fields using the consolidated prediction. And the model has a really good generalisation and transfer abilities. That, in some cases might offset the need for trained data especially if you want to apply the model in new regions and it has already been used by several institutions or organisations.

 

[Image changes to show a new slide showing various journal articles and Franz can be seen inset talking in the top right and a text heading appears: Check out the related papers]

 

If you want to know more about our methods, you can have a look at the papers because I will not be able to answer your questions right now.

 

[Image changes to show a new slide showing a data map and text appears on the right: ePaddocks^TM

Australian Paddock Boundaries, The ePaddocks^TM Australian Paddock Boundaries dataset is a shapefile containing the boundaries of paddocks at national spatial extent]

 

Yep. That’s just the link to the Ag Climate Data Shop.

 

[Image changes to show a new slide showing a photo of Franz on the left and a blue pattern on the right and text appears: Thank you, Franz Waldner, Agriculture & Food, franz.waldner@ec.europa.eu]

 

And with that I complete my presentation. Thanks a lot for your attention.

 

[Image changes to show the CSIRO logo and text appears: CSIRO, Australia’s National Science Agency]