Defra's Model Intercomparison Exercise
The Model Intercomparison Exercise (MIE) was a useful data source for the Modelling Review. The exercise is being run in phases and will continue after the review is complete. It will give Defra access to model capability and innovation and will enable Defra to collect details on the range and limits of model performance. The exercise has created an ongoing forum for modellers to discuss and evaluate models within a technical community. The exercise is split into 3 groups in accordance with Defra's needs – Urban, Regional Transboundary (Ozone) and Deposition modelling.
Data analysis has been undertaken centrally using the Open Air/R software. All code and methods are publically available. This process is unique within the UK/Europe in terms of scope and methodology.
The exercise is an inclusive process, open to all. For more information please email: aqevidence@defra.gsi.gov.uk
Phase One
Phase one of the exercise focused on operational evaluation and aimed to use pre-existing data where possible from other model evaluations s such as CREMO or Defra's existing evidence base. The exercise was undertaken with limited harmonisation of inputs to reduce resource requirements. All data provided were processed centrally using an automated report template developed using the Open Air/R software. This ensured harmonisation of calculation process and transparency of the analysis. Evaluation was undertaken using statistical analysis techniques based on Defra's Model Intercomparison Protocol (PDF 1.29 MB).
Modellers also provided detailed questionnaires regarding model setup so any assumptions or calibrations were noted. Thirty participants took part in phase one of the exercise, and these included researchers, practioners and consultants.
Phase One of the exercise is now complete and the reports for each group can be viewed from the following links:
Phase Two
Phase two of the exercise built on the phase one work, extending the analyses to cover a wider range of issues, which were developed in consultation with the modellers taking part in the exercise. The analysis was again carried out using the Open Air R software, so that the reports themselves are dynamically generated and completely reproducible. For more information on this, please see appendix C of each of the reports.
Phase two of the exercise is now complete for the urban group and for the regional group. Five air quality models took part in the urban group and nine in the regional group. The models were compared with ground-based measurements using various different techniques. The reports can be viewed from the following links:
Work is ongoing for the deposition group, and will be published in the near future.