Javier-Navarro

Javier Navarro

 

 

[Image of a split circle appears with photos in each half of the circle flashing through of various CSIRO activities and the circle then morphs into the CSIRO logo]

 

[Image changes to show a new slide showing a photo of a crop under a cloudy sky and Javier Navarro can be seen inset talking in the top right and text appears: AgScore, A standardised and consistent benchmark for climate forecasts, Javier Navarro, Research Scientist, Patrick Mitchell, Senior Research Scientist, Agriculture & Food, www.csiro.au, 18^th March 2021]

 

Javier Navarro: Welcome everyone. My name is Javier Navarro. I’m a Research Scientist in Agriculture and Food in the Sustainability Programme and I’m here to talk to you about AgScore, which is a service/tool that we have designed and it provides us a standardised and consistent benchmark for climate forecasts. I’m only part of the project and Patrick Mitchell is the lead for the project. But there’s many other, many other names that couldn’t fit right here but hats off to them because everyone has done a fantastic job so far.

 

[Image changes to show a new slide showing text: Today, Introducing AgScore, Agriculture is a risky business – largely due to climate variability, Seasonal forecasts mean to be accurate, but accuracy and utility are different things, AgScore is a service that empowers scientists to assess the utility of seasonal forecasts, It assesses forecast “goodness” using state of the art verification metrics, It packs diagnostics into a dynamic dashboard, AgScore’s value, It enables the fine-tuning of models/downscaling to enhance value and utility, AgScore next steps]

 

So, if you only have time to listen to one thing in this presentation, this is it. This is a summary of what I’m going to be talking about today.

 

[Image changes to show a new slide showing an Australian map and a line graph showing wheat yields and Javier can be seen in the top right talking and text appears: Introducing AgScore, Agriculture is a risky business – largely due to climate variability, Year to year climate variability = major risk to growers, Management decisions required well in advance, Highly variable historical wheat yields, Worsening conditions since 1970, 16% decline Apr-Oct rainfall, 20% decline May-Jun rainfall, Projections of longer lasting droughts, Not just about total rainfall, Rainfall at right time, Effect of temp., radiation most acute at particular crop phases, Frost damage during flowering, Heat stress reduces grain number]

 

Yes, so agriculture is a risky business and this has a lot to do with climate variability. So, the year to year climate variability in Australia, and elsewhere in the world, has been identified as a major risk to growers. And why is this? This is because farmers need to make management decisions well in advance of the season. For example, the, the opportunity for fertilising is quite narrow. So, if you haven’t pre-purchased your fertilisers ahead of time you’re not going to be able to apply it, and similar, similar things happen with sowing. The sowing window is quite narrow, and harvesting. So, a lot of the operations the farmers do have to be bang on time, and the quicker they can do them the better.

 

The climate variability, you can see an example here where I’m showing you historical wheat yields, and I’m showing New South Wales and Victoria as illustrative examples. From 1990 to 2016, the average yield for those states has gone up and down quite a lot and this is partly to do with the fact that this has been, there’s been worsening conditions in terms of rainfall since 1970. For example, from 1970 to 2015 I think it is, there’s been a 16% decline in rainfall from April to October, which is the growing season for wheat, and a 20% decline in May to June rainfall. And the projection is that this is only going to get worse. But I don’t want you to walk away thinking that all we have to do is predict total rainfall for the season. It really isn’t just about that. Rainfall is needed at specific times of the growing phase of crops and other climate variables like temperature and radiation have most acute effects or are needed at particular crop phases. So, for example, frost damage is bad at any time but it’s worse during flowering of the crop.

 

[Image changes to show a new slide showing a world map showing a Probabilistic Multi-Model Ensemble Forecast, and Javier can be seen in the top right talking and text appears: Introducing AgScore, Seasonal forecasts mean to be accurate, but accuracy and utility are different things, Multiple global forecasts, Different models and downscaling approaches, Different productivity and quality metrics, Focus of modellers has been increasing accuracy (r²), But r² can be improved with no palpable change in yield accuracy, Feeding through crop models -> turn forecast into Ag. Outcomes]

 

So AgScore, the problem is that they’re seasonal forecasts, and they try to reduce uncertainty but in the effort to improve their accuracy they’re missing that utility and accuracy are different things. So there’s been, there are multiple global forecasts that are published every year. There’s the Chinese, there’s the French, there’s Australian, nearly every country has one. But, every country model is different. Some are dynamic, or some are statistical models and they use different downscaling approaches. And also the measures by which we look at how good they are, are different. And historically the focus on modellers has been in increasing the accuracy of the weather predictions, rainfall, radiation, and you know, we know this is the r² but there’s many ways in which the r² can be improved with no, no significant change in yield accuracy. This means that the utility is quite low. If we feed climate information through crop models like AgScene or GrassGro, we can turn the forecast into agricultural outcomes like yield and biomass, and that’s the first step towards improving utility.

 

[Image changes to show a new slide showing a flow chart on the right showing daily weather data moving through an Ag productivity model to the Ag. Production outcomes and text appears: Introducing AgScore, Agscore is a service that empowers scientists to assess the utility of seasonal forecasts, Users submit climate model hindcast, Hindcast converted to yield predictions, How close is yield to historical?, Is user hindcast better than climatology?, Progressively diagnose differences, Wheat, sorghum, sugar, pastures]

 

So in AgScore, what AgScore does is that it allows users to submit a climate model hindcast. So, that’s a historical forecast. And the model will tell you on Day 1 this is the total rainfall. It gives you all the information daily, and then AgScore, it can feed that information through AgScene or GrassGro or any other productivity model so that the hindcast is converted to yield predictions in order to answer the questions such as “How close is the yield predicted by the user model?”, “How close is that to historical yield?”, no, “How close is that to the yield using historical climate?”. And another type of question is, “Is the hindcast that the user providing better than climatology?”, climatology being anything can happen that has happened over the last 30 years. So, that’s kind of a common baseline that we’re using. And what AgScore does is that it progressively allows the user to diagnose and tease out the differences. And currently we’ve got versions of AgScore for wheat, and we’re working on versions for sorghum, sugar, and pastures.

 

[Image changes to show a new slide and Javier can be seen in the top right talking and text headings appear: Introducing AgScore, It assesses forecast “goodness” using state of the art verification metrics, Three properties of a good forecast are consistency, quality and value, AgScore measures forecast quality attributes such as, bias, accuracy, skill, reliability, Using metrics like CRPSS, PIT, Hit rates – how often forecast was right, wrong, close, or inconclusive, Over multiple locations across agro-ecological regions, By converting climate into Ag. Outcomes, AgScore helps modellers enhance value]

 

So, what makes a good forecast? There was a seminal paper by Murphy in 1993 and he explained that there are three properties of a good forecast: there’s consistency, which means how closely a model represents what a climate scientist knows; the quality; and value, value being ability. What AgScore does is that it allows us to measure forecast quality, attributes such as the bias, the accuracy, the skill, and reliability, and I’ve put some definitions there for you to look at. And the way we do this is that we use common state-of-the-art actually metrics like the Continuous Rank Probability Skill Score, which essentially says how much better is the user supplied model compared to climatology, and this is widely used.

 

There’s other scores like the PIT reliability score which tells you the prediction bias given wet or, wet or dry conditions and also other metrics like the heat rate which tells you how often the forecast was right versus wrong, close, or inconclusive. And AgScore does this providing this type of metrics over multiple locations and across agro-ecological regions. And by converting climate into agricultural outcomes, like yield and bio-mass, or plant-available water, AgScore helps modellers enhance value and utility.

 

[Image changes to show a new slide showing the AgScore dashboard showing a map of Australia and Javier can be seen in the top right talking and text appears on the page: Introducing AgScore, It packs diagnostics into a dynamic dashboard, HTML web interface, sent via email, Interactive charts]

 

Here’s an example of a couple of snapshots of the AgScore dashboard. So, AgScore when it finishes doing its calculations it sets up a HTML web interface which [] and it sends it to users via email and it has a collection of interactive charts for properties, productivity properties like yield and biomass, or climate variables like rain, temperature, maximum or minimum temperature, radiation, and plant-available water. And there’s plenty of ways the user can click around and inspect individual values when they will have to do a demo.

 

[Image changes to show a new slide showing a map of Australia and Javier can be seen in the right talking and text appears: AgScore’s value, AgScore enables fine-tuning of models/downscaling to enhance value and utility, AgScore workflow provides capability which is not easily accessible to climate modellers, Crop/pasture modelling taken care of, Multiple locations across regions, Prompt feedback, high level to detailed, Enables agile back-forth between user and AgScore, Enhance Value of models and downscaling, Will help growers make decisions under less uncertainty, Tech that supports higher long-term productivity growth]

 

So, the AgScore, what value does AgScore provide? AgScore enables modellers to fine tune their model or downscaling procedure to enhance value and utility. Essentially this is a capability, the crop or pasture modelling capability which climate scientists rarely have access to. In CSIRO it’s a different story but in many places you have to know how to run AgScene or GrassGro or similar models. So, AgScore can take care of all of this for the users and not only it allows the users to do this but it actually does the apps in runs over multiple locations such as what we’re seeing on the right. That’s an example of the AgScore locations for wheat.

 

And AgScore also provides prompt feedback with detail ranging from quite high level to very detailed. And this enables a very agile back and forth between the user, ie the modeller, and AgScore. And this agile back and forth is what enables them to enhance the value of their model or downscaling procedure, and this in the end will help growers make better decisions under less uncertainty because instead of focussing on increasing the r²  or the accuracy of rainfall or radiation, what we’re actually worrying about is how well are we predicting agricultural outcomes.

 

[Image changes to show a new slide showing text: AgScore Next Steps, GRDC 2020 call for projects, “Improve the way growers and others in the grain industry use seasonal climate forecasts”, GRDC-AgScore project started this FY, Continuation of DigiScape work, very intertwined, Focus on wheat, sorghum, sugar, and pastures, Potential for future uptake in other countries familiar with APSIM, e.g. India through ICRISAT, Potential for journal article, conference abstract, More to come…]

 

So, next steps. AgScore has already had a little bit of impact. The GRDC last year made a call for projects and I thought what they said they wanted ideas to improve the way growers and others in the grains industry use seasonal climate forecasts. So, unsurprisingly Pat and others applied for this opportunity and the GRDC AgScore project started this financial year and there’s a lot of overlaps between both projects and yes the focus is on wheat, sorghum, sugar, and pastures.

 

And there is a potential in the future for AgScore to be applied in other countries, especially the ones that are familiar with AgScene already, such as India through ICRISAT. And there’s potential, we’re looking at potentially writing a journal article, or a conference abstract and, or ideas that I’m sure will come up with time. And that’s it for me. Thank you very much.  

 

[Image changes to show a white screen and the CSIRO logo and text can be seen: CSIRO, Australia’s National Science Agency]