Quality Assurance Guidelines

Introduction

Quality Assurance (QA) is the system of measures and controls put in place to ensure that a consistent and acceptable level of quality is achieved for a product or service such as Australian Flood Study Database (AFSD). It is generally accepted that a quality product is one that is fit-for-purpose and free of errors.

Quality Assurance Guidelines (QA Guidelines) have been developed to ensure the accuracy and completeness of the flood study data (an entry) submitted to the Australian Flood Studies Database. The QA Guidelines describe the minimum requirements against which each entry are assessed before publication.

These guidelines support the National Flood Risk Information Project¿¿¿s objectives of improving the quality and accessibility of flood risk information, by ensuring that the Australian Flood Risk Information Portal delivers the highest possible reliability and usefulness to users.

The quality assurance assessment for a flood study entry has two parts - an accuracy check and a completeness check. The accuracy check involves verifying the correctness of the information entered into the database against the original flood study resource. The completeness check involves ensuring that, where appropriate information is available, there are no 'gaps' in the flood study resource's metadata entry.

This guide is intended for those individuals undertaking a quality review of a flood study submitted for inclusion in the AFSD. Nominated reviewers should have experience entering data into the AFSD, and be familiar with the AFSD data entry fields. The reviewer will also need some familiarity with flood studies and their general structure.

The QA Guidelines are not intended to be an introduction to the use of the AFSD. Instructions for entering information into the database are available in the Data Entry Guidelines. The Data Entry Guidelines describe in detail the requirements for each field in the database, and provide examples of information that is acceptable for each field.

Example Entries
Flood studies 2815, 2829 and 2866 are good examples of studies that meet the quality standards. These flood study entries reflect the information within the flood study document to the fullest degree using the data entry application.

Figure 1 describes the QA process for ensuring the accuracy and completeness of flood study entries published through the AFSD.

Due to the complexity of this image, no alternative description has been provided. Please email Geoscience Australia at clientservices@ga.gov.au for an alternative description

Figure 1. A chart showing the basic process an AFSD entry undergoes before being published. The numbers correspond to sections of this guide where
those processes and decisions are discussed in more detail.

Quality criteria

A set of accuracy and completeness criteria have been identified to inform the quality review of all studies entered into AFSD before publication online. If necessary the submitter will be contacted, and a QA Dialogue completed to bring an entry up to the required standard.

Accuracy criteria:

  • The entry does not contain any inaccurate, incorrect or false information.
  • The entry does not contain information that contradicts any other information in the flood study report.
  • All resources submitted for publication, including maps and attachments, are appropriately licensed for use and publication.
  • All attachments, and other related resources, are relevant and correctly attached or referenced.
  • All attached resources are legible, and can be successfully downloaded and viewed.

Completeness criteria:

  • All available information has been extracted from the flood study report and included in the entry.
  • The entry includes all of the report¿¿¿s modelled scenarios.
  • All recorded mapping entries have a corresponding modelled event.
  • All available attachments are provided, and are as complete as possible.

Once the flood study has been submitted it is the responsibility of the reviewer to perform both an accuracy check and a completeness check on the content.

This QA Guideline provides specific examples, relating to separate parts of the data entry process, to demonstrate how to check the 'quality' of flood study entries.

As shown in Figure 1. the process starts with a quality review.

Reviewer quality review

The reviewer role is generally provided by the commissioning agency.

Quality Assurance overview

The quality review of an AFSD entry is initiated by an email, in which the reviewer is advised of entries awaiting review. The reviewer can then view entries through the ASFD data entry and maintenance application.

To start a review:

  1. Login to the AFSD.
  2. Click on the Quality Assurance link (see Figure 2 below).
  3. View the list of entries open for review.
  4. Select an entry to review.
  5. From the Submission tab, download the attached flood study report, and any associated resources.
  Figure 2. Screenshot of Quality Assurance page in data entry and maintenance application showing a reviewer¿¿¿s list of entries awaiting review

Figure 2. Screenshot of Quality Assurance page in data entry and maintenance application showing a reviewer¿¿¿s list of entries awaiting review.

The reviewer should then undertake an accuracy check and a completeness check of all tabs completed for the entry.

Accuracy check overview

The purpose of the accuracy check is to identify any obvious errors or inaccuracies in the entry. As an initial step the reviewer should:

  • Review the introductory sections of the flood study report (Table of Contents, List of Figures, Introduction, Executive Summary etc.) to establish the size and overall content of the study.
  • Check that the attached flood study report is complete, has no missing appendices or sections, and that the file(s) display and download correctly.
  • Ensure that the preliminary information has been correctly transcribed, with no inconsistencies in content or format (check for consistent spelling and capitalisation). See Data Entry and Maintenance Guidelines.
  • Review the content of the General tab against this preliminary material to ensure the entry is accurate and complete.

The reviewer can then proceed to check the accuracy of the information recorded in all other tabs.

There is no restriction on, or suggested order, for completing a review of the remaining tabs. The reviewer will find that their preferred approach will depend on the structure of the flood study report.

However, regardless of how the review is approached the reviewer should ensure that they always:

  • Check for corresponding passages in the flood study report for all information recorded against the entry.
  • Ensure any information that was entered into a field is relevant to that field¿¿¿s description. Refer to the Data Entry Guidelines for a description of what information is expected in any particular field.

Record, for clarification through the QA Dialogue process, any information that cannot be cross-referenced to the report, is irrelevant or does not match the field descriptions. The accuracy check is complete when all issues identified through the QA Dialogue process have been addressed.

Completeness check overview

After the initial accuracy check of all data entry fields is complete, a more focused completeness check should be undertaken. The overall quality of the entry will be directly impacted by the reviewer¿¿¿s attention to any missing or incomplete information. Any shortcomings in the data entry process should be added to the Quality Assurance dialogue.

The purpose of the completeness check is to note any relevant information that has been missed or overlooked by the submitter when completing the data entry fields.

More details on completing a quality review of each tab, and their fields, are outlined in the following sections. These sections provide specific hints and tips for identifying missing information and data entry errors.

General tab

Most information for the General tab is derived from the 'Introduction' and preliminary pages. Accuracy checks on the General tab include verifying: the map location, creek and town name references; and confirming that the commissioning organisation, consultant names and contact details are correct and conform to the Data Entry Guidelines.

Below are a set of accuracy and completeness checks that are usually relevant to the General tab. This is not an exhaustive list, other checks may be required depending on the approach and nature of the specific flood study.

Accuracy check

Check that:

  • The watercourses and towns entered are relevant to the present study and are not, for example, mentioned in the report as a summary of previously undertaken studies.
  • The map location is close to the main town mentioned in the flood study.
  • The Lead Consultant is correctly identified, and is not actually a subordinate consultant or sub-contractor.
  • The Date Report Completed is the 'date of first draft' or similar, not the date of the latest published version.
  • Comments are accurate and clear.
  • The Study Name conforms to the field rules outlined in the Data Entry Guidelines (i.e. does not include subtitles).
  • Company names do not include unnecessary information (i.e. does not include "Pty Ltd"). See Data Entry and Maintenance Guidelines for more information.
  • Check for consistency of spelling and capitalisation.

Example Errors

  • Feature names incorrectly transcribed, ie. "Floody River" entered into the General tab under the Names of Watercourses field where in the report there is only ever mention of "Floody Creek".
  • Dates entered don¿¿¿t match the resource.

Completeness check

Check:

  • Using a key word search for "Creek" and "River" ensures all watercourses mentioned in the flood study are noted in the Names of Watercourses field.
  • Using a key word search such as "town of", "suburb of" etc. that all relevant information related to the study location is noted in the relevant fields.

Analysis tab

The Analysis tab usually records the most detailed information about the scenarios modelled. As this information is often spread cross numerous sections of the study, it may be time consuming to properly cross-check and verify. However, in the context of the AFSD this is the most critical tab. The more complete the AFSD¿¿¿s analysis data, the more useful the database will be for identifying areas where communities have the least resilience to flood.

Quality Assurance of the Analysis tab focusses on verifying the historical and design events. As information may not be particularly easy to find as each flood study report may be structured differently this part of the review may be the most difficult to achieve the goals of the commissioning organisation.

The completeness check of the Analysis tab will require careful reading of the flood study report. To ensure that the entry is complete, focus on the sections that specifically discusses modelling. Any information found in the study but not reflected in the entry, where it is possible to do so, should be noted down and form part of the Quality Assurance dialogue.

Below are a set of accuracy and completeness checks that are usually relevant to the Analysis tab. This is not an exhaustive list, other checks may be required depending on the approach and nature of the specific flood study.

Accuracy check

Check that:

  • The software recorded for both Hydrological Estimates and Hydraulic Modelling is consistent with the software used in the study to produce the design event.
  • The software mentioned in the entry relates to the present modelling and not to previous studies.
  • Each recorded design event has a correctly associated ARI.
  • Comments are accurate and clear.
  • Years and months recorded against a historical event are correct.
  • A historical event is correctly recorded as either a 'calibration' or 'validation' event.
  • Design hydrographs are correctly marked as not present, when the study does not include 'design' hydrographs. Verify against the report that the submitter did not mistake historical hydrographs, or other kinds of graphs for a 'design' hydrograph. Details of what constitutes a design hydrograph can be found in the Data Entry Guidelines.
  • Damage assessment details are correct, and that the relevant table data has been correctly transcribed. Check each of these numbers carefully with the information in the report.
  • The Average Annual Damages data is associated with the correct year. Data in this field may quote "dollars of a particular year". The year is usually assumed to be the same as the year the report was completed. Check that the amount quoted does not relate to different year. For example a study completed in 2000 having "Average annual damage was calculated as $250,000 (1997 dollars)". If the year for the dollars is quoted as different to the year of the flood study then this information should be recorded in the Comments field of the Damages Assessment section of the tab.
  • Check for consistency of spelling and capitalisation.

Completeness check

Check that:

  • Where available historical events have associated ARIs. Locate relevant ARI's using a key word search for the year or month of the historical event.
  • The modelling information is complete, and that all scenarios are included.
    For example, although the main focus of the study may have been the 10, 20, 50, 100 and PMF events, modelling may also have been undertaken for the 100 year ARI event under specific climate change or mitigation scenarios. These extra modelled scenarios can be missed if they are not listed in the introductory sections of the flood study report.
  • All Design Hydrographs are recorded in the entry, including those hydrographs that may have been presented either graphically or in tabular form.
  • A comment about the purpose of the ARI is recorded against each event if there are multiple events recorded for the same ARI. See DE and M Guidelines
  • A comment is recorded against the methods or modeling that enhances the value to the consumer. (optional).

All main content is recorded. Use key word stems; "hydrolog" and "hydraul" to check for any potentially missed content.

Survey tab

The quantity and quality of survey information found in flood studies varies greatly, so it is hard to define a minimum requirement. Consequently the range of what is acceptable for the Survey tab is quite broad.

To establish what survey information is included in the study undertake a key word search for "survey". Review the content of all search results. This will generally identify sufficient information for both the accuracy and completeness check. As with other tabs, missing or inaccurate information should be recorded for the Quality Assurance dialogue.

Below are a set of accuracy and completeness checks that are usually relevant to the Survey tab. This is not an exhaustive list, other checks may be required depending on the approach and nature of the specific flood study.

Accuracy check

Check that:

  • Survey dates are correctly recorded, particularly when the information is not clearly presented in the study.
  • Data custodian names do not include unnecessary information (i.e. does not include "Pty Ltd") or is not recorded as an acronym. The field rules are outlined in the Flood Study Data Entry Guidelines.
  • Check for inconsistencies between Survey Method and Format of Survey. For example, it is unlikely that a LIDAR survey would have a format of 'cross-sections'.
  • The numerical fields are accurate ('Vertical Accuracy' etc.).
  • Survey entries relate to modelling in the present study and are not, for example, part of a summary of previous work relating to other studies.
  • Check for consistency of spelling and capitalisation.

Completeness check

Check that:

  • All relevant survey information has been recorded. A key word search for "Survey" will identify potential information that may have been overlooked.
    • Often the mention of surveys undertaken for, or used in, the flood study can be very brief, for example the report may only name the surveyor. This scenario is acceptable where no more information is available.

Mapping tab

The Mapping tab records information about two types of maps often present in flood study reports¿¿flood inundation maps and flood hazard maps. An accuracy check of the Mapping tab data is generally straight forward. The reviewer need only sight each map that is identified in the database entry to satisfy the check.

The completeness check requires a little more investigation. It is not uncommon with older flood study reports for the associated maps not to have been digitised. So the reviewer will not be able to cite the map as an attached resource. However every map that was produced as part of the flood study, irrespective of whether that map is attached to the entry needs to be noted.

Usually the 'List of Figures' will include any maps included in the flood study report. Even, for example, if the 'Mapping' section of the report is missing.

Occasionally the flood study report states that mapping was produced for an event, but this information is not reflected in the report¿¿¿s lists of figures or maps. Such maps are often overlooked in data entry, but should be included. Note any missing maps for later reference in the QA Dialogue.

Below are a set of accuracy and completeness checks that are usually relevant to the Mapping tab. This is not an exhaustive list, other checks may be required depending on the approach and nature of the specific flood study.

Accuracy check

Check that:

  • Every map has an associated scenario.
  • The dates of each historical map recorded are accurate.
  • Comments are accurate and clear.
  • Hazard maps have the correct guidelines selected. Use a key words such as "guidelines" and "hazard" to identify relevant information.
  • For maps other than 'inundation' maps (such as afflux maps relating to different flood scenarios or velocity maps) that the type and purpose of the maps is noted in the Flood Hazard Mapping, Comments field.
  • Check for consistency of spelling and capitalisation.

Completeness check

Check that:

  • An ARI comment is recorded against each event if there are multiple events recorded for the same ARI.
  • All maps listed under the Mapping tab have a corresponding event entry under the Analysis tab.
  • All historical maps, where possible, are recorded in the ARI Comments field as either a 'calibration' or 'validation' event. If neither, specify the nature and purpose of the map briefly in the ARI Comments field.
  • All afflux or velocity maps record appropriate comments regarding the type of map in the Comments field.
  • The version of hazard guidelines used in producing the hazard maps is recorded. Use a key word search such as "guidelines" and "hazard" to locate the relevant information.

Mitigation tab

The information required to complete the Mitigation tab can be difficult to locate if the study¿¿¿s recommendations are not clearly summarized in the flood study report. However, the Mitigation tab records perhaps the most important output of a flood study process.

The relevant information for the Mitigation tab can usually be found most clearly articulated in the floodplain risk management plan derived from the flood study. Where available this document, and any other associated information, should be attached to the relevant flood study entry, see the Data Entry Guidelines for more details.

Any questions about the mitigation measures, or post flood study information, entered under the Mitigation tab should be noted in the Quality Assurance dialogue.

Below are a set of accuracy and completeness checks that are usually relevant to the Mitigation tab. This is not an exhaustive list, other checks may be required depending on the approach and nature of the specific flood study.

Accuracy check

Check that:

  • Only recommended mitigation measures are included in the entry. Mitigation measures that are desirable, but not actually recommended due to reasons such as costs, impacts etc., should not be recorded as 'recommended mitigation measures'.
  • Post study information is verified against other evidence and if this information is not a part of the original flood study report, should be attached.
  • Check for consistency of spelling and capitalisation.

Completeness check

Check that:

  • Any post-study recommendations are recorded against the database entry. Use a key word searches on "recommend" and "measures" etc. to locate any measures missed in the initial data entry.
  • Any post flood study information recorded in the entry (either noted in the original report or available as a separate document) is attached to the entry.

Resources tab

A review of the Resources tab involves ensuring that all attachments (whether they are associated documents or maps) are relevant to the flood study entry. This includes verifying that each associated file is named appropriately, and that the resource can be successfully downloaded and viewed (N.B. at this time maps cannot be viewed directly in the data entry and maintenance application).

In most cases the file description field should be completed according to the Data Entry Guidelines (unless there is a valid reason why an alternative description has been used). The description should include the study name, the status of the report, and the month and year the report was completed.

Example description for a flood study
Floody River ¿¿¿ Draft Report ¿¿¿ May 1987.

Below are a set of accuracy and completeness checks that are usually relevant to the Resources tab. This is not an exhaustive list, other checks may be required depending on the approach and nature of the specific flood study.

Accuracy checks

Check that:

  • The resource's file name is descriptive, meaningful, succinct and reflects the study name.
  • Attached files relate directly to the entered information.
  • Information recorded about each resource is correct.
  • All attached files directly relate to the flood study entry, or are otherwise relevant to the study or area of interest. See the Flood Study Data Entry Guidelines for more details on what should or should not be attached to an AFSD entry.
  • All attached documents can be downloaded or viewed easily by other users. Large files may be attached as multiple documents, but the resulting files should be logically structured and named. If the attached file is compressed non-standard compression software should be avoided.
  • All maps are in a format suitable for use by a third party (i.e. a shape file).
  • Check for consistency of spelling and capitalisation.

Completeness checks

Check that:

  • All relevant files are included with the entry and upload successfully.
  • All files are complete. Verify that sections of the study, such as appendices or figures are included. Any missing sections should be recorded and form part of the Quality Assurance dialogue.

Quality Assurance dialogue

The Flood Study Quality Assurance Details contains two fields. A Report status field with options to: reject, accept, pass or delete a flood study record. The second field Comments records the QA dialogue between author and reviewer in turn. Each person involved in the dialogue places their initials at the beginning of their portion of text. (The QA dialogue system will be modified and improved in the future).

When the dialogue regarding accuracy and completeness are resolved one of the options is selected by the reviewer and the QA is passed onto the national reviewer. Or the national reviewer determines the record is acceptable and the flood study is published. Either path is dependent on a different portion of the QA process.

Figure 3. A view of the current Quality Assurance dialogue shows Report status and Comments fields. Note the initials at the beginning of each statement indicating a different voice involved in the dialogue.

Figure 3. A view of the current Quality Assurance dialogue shows Report status and Comments fields. Note the initials at the beginning of each
statement indicating a different voice involved in the dialogue.

Quality requirements

Acceptable quality check

Before a report entry is published through the AFSD it must meet the following standards as certified by the assigned reviewer and has;

  • Undergone and passed an accuracy check.
  • Undergone a completeness check verifying, that appropriate information available in the flood study report/s has been included in the database entry.

Submit to national reviewer

Once an entry has been judged by the reviewer to have met the required accuracy and completeness checks the entry is submitted to Geoscience Australia for a final accuracy check before publication. The reviewer finalises the process by submitting the entry for review through the Geoscience Australia Data Entry and Maintenance application. The reviewer will receive an email acknowledging the submission. There may be follow up action requested by the national reviewer.

National reviewer accuracy review

National reviewer accuracy requirements

AFSD entries submitted through the Geoscience Australia Data Entry and Maintenance application will undergo an accuracy check by a National Reviewer.

This national reviewer accuracy review ensures Geoscience Australia meets its agency obligations when publishing directly to the Australian Flood Risk Information Portal. The aim of the national reviewer accuracy review is to identify and correct, or confirm the absence of, any information in the AFSD entry that is inaccurate, incorrect or false.

New or missing information will not be added during accuracy review. The completeness of the flood study entry is determined by the reviewer, in consultation with the author/submitter during the QA dialogue. Completeness is a matter to be discussed and decided on during the QA dialogue process between author/submitter and reviewer. If minor inaccuracies are identified these will be corrected by the national reviewer, however a QA Dialogue will be started between the national reviewer and the initial reviewer to address any significant issues.

Acceptable accuracy review

A national reviewer will deem a flood study entry as accurate if the following criteria are met:

  • All information in the entry is accurate and correct with no false information included.
  • All attached resources can be downloaded, and are legible.
  • All resources are appropriately licensed for use and/or publication.

Publish entry for public use

When all relevant quality criteria have been met and the Quality Assurance review process is completed the flood study entry will be made available to the public in the AFSD application found at Australian Flood Risk Information Portal. The original author and the reviewer will be notified by email that the entry has been published.