Conceptio › Archive › USGS Publications

USGS Publicationspublic full text

Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor

Jason S. Alexander, Jonathan P. O’Connell, Jeb E. Brown, D.J. Dean, D.J. Topping, D.D. Buscombe · U.S. Geological Survey

USGS Publications · Papers · License: Public Domain

Open Source ↗Direct PDF ↓

acoustic calibration filereport sediment sensor usgsopen

usgs, geological survey, united states, science, government publication, usgs open-file report, 2023, 1039, interim, guidance, calibration, checks, submersible, acoustic, backscatter, sediment, sensor

Prepared in cooperation with the Federal Interagency Sedimentation Project

Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor

Open-File Report 2023–1039 Version 1.1, November 2023

U.S. Department of the Interior U.S. Geological Survey

Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor By Jason S. Alexander, Jonathan P. O’Connell, and Jeb E. Brown

Prepared in cooperation with the Federal Interagency Sedimentation Project

Open-File Report 2023–1039 Version 1.1, November 2023

U.S. Department of the Interior U.S. Geological Survey

U.S. Geological Survey, Reston, Virginia: 2023 First release: 2023 Revised: November 2023 (ver. 1.1)

For more information on the USGS—the Federal source for science about the Earth, its natural and living resources, natural hazards, and the environment—visit https://www.usgs.gov or call 1–888–ASK–USGS. For an overview of USGS information products, including maps, imagery, and publications, visit https://store.usgs.gov/. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government. Although this information product, for the most part, is in the public domain, it also may contain copyrighted materials as noted in the text. Permission to reproduce copyrighted items must be secured from the copyright owner. Suggested citation: Alexander, J.S., O’Connell, J.P., and Brown, J.E., 2023, Interim guidance for calibration checks on a submersible acoustic backscatter sediment sensor (ver. 1.1, November 2023): U.S. Geological Survey Open-File Report 2023–1039, 23 p., https://doi.org/10.3133/ofr20231039. Associated data for this publication: O’Connell, J., Alexander, J.S., and Dutton, D.M., 2023, Data from lab experiments to support interim guidance for performing calibration checks on the Sequoia Scientific LISST-ABS acoustic backscatter sensor: U.S. Geological Survey data release, https://doi.org/10.5066/P9M41G3W. U.S. Geological Survey, 2022, USGS water data for the Nation: U.S. Geological Survey National Water Information System database, https://doi.org/10.5066/F7P55KJN. U.S. Geological Survey, 2023, USGS 09363500 Animas River near Cedar Hill, NM, in USGS water data for the Nation: U.S. Geological Survey National Water Information System database, https://doi.org/10.5066/F7P55KJN. [Site information directly accessible at https://waterdata.usgs.gov/nwis/inventory/?site_no=09363500&agency_cd= USGS&.] ISSN 2331-1258 (online)

iii

Contents Abstract��1 Introduction��1 Background��2 Purpose and Scope��3 Methods��4 ABS Sensors��4 Experiment Apparatus��4 Experiment Stir Bar Geometries��4 Experiment Standardized Particle Materials��5 Laboratory Experiments��6 Error Measurements��6 Experiment Repeatability��6 Field Application��6 Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks on a Submersible Acoustic Backscatter Sediment (ABS) Sensor��7 Mixing Materials and Methods��7 Experiments with Manufacturer Standardized Particles��8 Experiments with Raw Standardized Particles��8 Experiments with USGS Standardized Particles��13 Repeatability of Experiments��13 Field Application Results��16 Discussion��19 Interim Guidance for Calibration Checks on the Acoustic Backscatter Sediment (ABS) Sensor��20 Summary��20 References Cited��22

Figures 1. 2. 3.

Photographs showing the ABS submersible acoustic backscatter sediment sensor��2 Photograph showing benchtop apparatus used during the laboratory experiments��4 Photographs showing three types of magnetic stir bars tested for suspended-sediment concentration accuracy during the laboratory experiments��5 4. Photograph showing benchtop apparatus used during the field application at the U.S. Geological Survey streamgage on the Animas River near Cedar Hill, New Mexico��7 5. Graphs showing relations between standard and measured sediment concentrations, or standard sediment concentrations and mean percent differences in measured sediment concentrations using ABS manufacturer standardized particles��9 6. Graphs showing relations between standard concentrations and measured concentrations, or standard concentrations and mean percent differences of measured concentrations using raw standardized particles��12

Graphs showing relations between standard concentrations and measured concentrations, or standard concentrations and mean percent differences of measured concentrations using U.S. Geological Survey standardized particles and a cross-shaped stir bar��15 8. Graphs showing relations among concentrations and mean percent differences from a replicated experiment using raw standardized particles and a cross-shaped stir bar��17

Tables 1.

2. 3. 4. 5.

7. 8. 9.

10.

11.

Magnitude of calibration drift observed in four submersible acoustic backscatter sediment sensors deployed at U.S. Geological Survey streamgages in Montana and Wyoming, 2019 to 2020��3 Magnetic stir bars tested for suspended-sediment concentration accuracy during the laboratory experiments��5 Standardized particle materials used during the laboratory experiments��5 Summary of mean absolute percent differences relative to standard concentrations for the laboratory experiments��8 Summary statistics of suspended-sediment concentrations measured in the laboratory using the ABS manufacturer standardized particles and a polygon-shaped stir bar��10 Summary statistics of suspended sediment measurements made in the laboratory using the ABS manufacturer standardized particles and a cross-shaped stir bar��11 Summary statistics of suspended-sediment concentration measurements made in the laboratory using raw standardized particles and a polygon-shaped stir bar��13 Summary statistics of suspended-sediment concentration measurements made in the laboratory using raw standardized particles and a cross-shaped stir bar��14 Summary statistics of suspended-sediment measurements made in the laboratory using U.S. Geological Survey standardized particles and a cross-shaped stir bar��16 Statistics of suspended-sediment concentration measurements made during a replicate experiment in the laboratory using raw standardized particles and a cross-shaped stir bar��18 Summary of field calibration checks at the U.S. Geological Survey streamgage, Animas River near Cedar Hill, New Mexico��19

Conversion Factors U.S. customary units to International System of Units

Multiply

To obtain Length

inch (in.)

2.54

centimeter (cm)

inch (in.)

25.4

millimeter (mm)

Volume ounce, fluid (fl. oz)

0.02957

ounce, avoirdupois (oz)

28.35

liter (L)

Mass gram (g)

International System of Units to U.S. customary units

Multiply

To obtain

Length centimeter (cm)

0.3937

inch (in.)

millimeter (mm)

0.03937

inch (in.)

Volume liter (L)

33.81402

ounce, fluid (fl. oz)

Mass gram (g)

0.03527

ounce, avoirdupois (oz)

Temperature in degrees Celsius (°C) may be converted to degrees Fahrenheit (°F) as follows: °F = (1.8 × °C) + 32. Temperature in degrees Fahrenheit (°F) may be converted to degrees Celsius (°C) as follows: °C = (°F – 32) / 1.8.

Abbreviations GSD

grain-size distribution

MAPD

mean absolute percent difference

MPD

mean percent difference

RMSE

root mean square error

Sequoia

Sequoia Scientific, Inc.

SSC

suspended-sediment concentration

USGS

U.S. Geological Survey

Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor By Jason S. Alexander, Jonathan P. O’Connell, and Jeb E. Brown

Abstract

Introduction

Over the past two decades, the U.S. Geological Survey (USGS) and other agencies have pioneered the use of active acoustic sensors to monitor suspended-sediment concentrations and particle sizes in rivers and streams at the subdaily time scale. The LISST-ABS submersible acoustic backscatter sediment sensor (or “ABS sensor”) was developed by Sequoia Scientific, Inc., as an alternative to turbidity sensors for monitoring suspended-sediment concentrations in surface waters. The ABS sensor is different than traditional active acoustic instruments because it is small, lower in cost, lightweight, and requires less power; and the sampling volume is within the first 15 centimeters of the transducer face. Initial testing by the USGS indicated the ABS sensor had utility as a novel, cost-effective, off-the-shelf tool for monitoring suspendedsediment concentration in surface waters, and its use within the agency has increased in since its introduction around 2016. However, initial testing did not account for the potential of transducer calibration drift over longer deployments. As part of its mission to unify and standardize research and development activities of Federal agencies involved in fluvial sediment studies, the Federal Interagency Sedimentation Project partnered with the USGS Wyoming-Montana and New Mexico Water Science Centers to examine the potential for use of standard, low-tech laboratory equipment to perform calibration checks on ABS sensors on long-term deployments. The experiments were intended to provide USGS scientists and the public with interim guidance to assist in operating and maintaining the ABS sensor. Laboratory experiments and field applications of the experiments indicate that USGS personnel can accurately do calibration checks of the ABS sensor with standard laboratory equipment. Percent errors relative to the standard concentrations were lowest for raw Ballontini glass bead particles and USGS-sieved particles combined with cross- or polygonshaped stir bars. Percent errors relative to the standard were generally less than 10 percent for standard concentrations greater than or equal to 100 milligrams per liter and were near 10 percent for the 10.4 milligrams per liter standard concentration. Errors were generally greater than 10 percent at concentrations less than 10.4 milligrams per liter. Field applications of one of the experimental methods demonstrated accurate tracking of calibrations across a 5-month deployment period.

Sediment transport in rivers and streams is a fundamental physical process shaping the Earth’s surface. Quantifying sediment transport is of longstanding public interest because of its direct effects on economic activity, water quality, and ecosystem health. Predicting sediment transport in streams is notoriously difficult because the mass of sediment in transport at a point in a river is a function of complex fluid and sediment grain forces, and the sizes and mass of sediment available to the fluid for transport (Topping and others, 2000a, 2000b; Wright and others, 2010). When the sizes and mass of sediment available for transport are strongly affected by unpredictable or unmeasured tributary contributions, the mass of sediment in transport is only a weak function of water discharge, and accurate prediction using physically based sediment transport equations is intractable (Landers and Sturm, 2013; Topping and Wright, 2016; Dean and others, 2022). Over the past two decades, the U.S. Geological Survey (USGS) and other agencies have pioneered the use of active acoustic sensors to monitor suspended-sediment concentrations (SSCs) and particle sizes in rivers and streams at the subdaily time scale (Moore and others, 2013; Landers and others, 2016; Topping and Wright, 2016). Acoustic sensors emit sound at various frequencies into the water column, most commonly to measure water velocity via the Doppler effect (Topping and Wright, 2016); however, acoustic sensors also measure the amount of sound returning to the sensor (backscatter), and the magnitude of this backscatter strongly correlates to the SSC in the water column. When paired with physical samples of suspended sediment, acoustic backscatter can be used as a surrogate measure of SSC via statistical relations (Landers and others, 2016; Topping and Wright, 2016). Traditional deployments of acoustic instruments to monitor SSC in streams are commonly attached to a rigid frame that is secured to a stable body such as a bridge pier, abutment, or bedrock. Such rigid deployments are less successful when alluvial banks are the primary stable body because the banks can erode during high-flow events. For such settings, it is ideal to have flexible deployments whereby instruments can be suspended in the water column via cable, chain, or both, allowing for rapid retrieval and depth adjustments. Flexible deployments are more typical for water-quality instruments such as turbidity sensors, which use optical backscatter as a

2 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor measure of water clarity and can also be used to monitor SSC (Rasmussen and others, 2009); however, turbidity sensors are also prone to biofouling, and their optical signals are strongly sensitive to grain size, which can be problematic for rivers with complex suspended-sediment mixtures (Landers and Sturm, 2013; Agrawal and others, 2019). The LISST-ABS submersible acoustic backscatter sediment sensor (or “ABS sensor”) was developed as an alternative to turbidity sensors for monitoring SSCs in surface waters (Sequoia, 2016). The ABS sensor emits an 8-megahertz acoustic signal into the water column and measures the backscatter signal intensity returning to the instrument (Sequoia, 2016). The ABS sensor is different than traditional active acoustic instruments because it is small and lightweight, it requires less power, and the sample volume is within the first 15 centimeters (cm) of the transducer face (fig. 1). The acoustic backscatter intensity measured by the ABS sensor is translated directly to a measure of SSC via statistical relation of the scattering intensity, which is nearly constant beyond the Raleigh limit at the 8 megahertz frequency for grain sizes greater than about 60 micrometers (µm; Agrawal and others, 2019). Because of its smaller size, the ABS sensor can be deployed in the same fashion as a turbidity sensor (fig. 1) and can be less sensitive to biofouling (Sequoia, 2016). The ABS sensor can measure SSCs as much as 10,000 milligrams per liter (mg/L), and its lack of sensitivity to grain size has the potential to generate more stable statistical relations with SSC than turbidity sensors for complex suspended-sediment mixtures. The ABS signal can also be paired with the optical backscatter signal of a turbidity sensor to enhance sensitivity to finer grain sizes in suspension (Agrawal and others, 2019; Sequoia, 2020). To meet the quality standards typically required for the USGS Water Mission Area records available on the National Water Information System, field personnel are advised to perform regular checks on deployed instruments to ensure adequate performance (Wagner and others, 2006). In the case of turbidity sensors, fouling and calibration checks are done on most site visits to track the effect of biofilms and transducer drift on the transmitted signal (Rasmussen and others, 2009); calibration checks are done using commercially available standards. Despite the ABS sensor’s lower sensitivity to biofouling, uncertainty regarding the sensor’s drift from factory calibrations remains undocumented. This report summarizes laboratory experiments and field methods that tested the use of common materials and methods for checking calibration drift of the ABS sensor over the course of multimonth deployments and is intended to provide guidance for its use in surface waters.

intensity is a parameter associated with the internal calculations. The ABS sensor differs from these profilers because acoustic backscatter intensity is the primary product output, and a transducer is used to convert the strength of the backscatter signal to a standardized measure of SSC. The transducer in the ABS sensor is factory calibrated for SSCs ranging from 0 to 10,000 mg/L using a proprietary sediment suspension tank (Sequoia, 2016). Factory calibration uses Ballontini glass impact beads (glass beads) sieved to a range of 75 to 90 µm in calibration suspensions; thus, the standardized concentration is reported in units of milligrams per liter of 75- to 90-µm glass beads. Initial testing by the USGS indicates the ABS sensor has utility as a novel, cost-effective, off-the-shelf tool for monitoring SSC in surface waters (Snazelle, 2017; Manaster and others, 2020), and its use within the agency has increased since its introduction around 2016. However, initial testing did not account for the potential of transducer calibration drift over longer deployments. Comparison of factory calibration checks in ABS sensors recently deployed by the USGS Wyoming-Montana Water Science Center has indicated A

Transducer face

Cable suspending sensor housings

Cable spanning width of river channel

Tubular aluminum housing containing ABS sensor

Background Active acoustic sensors such as acoustic Doppler profilers and acoustic Doppler current profilers of varying frequencies have been used by the USGS over the past two decades to measure SSC in rivers and streams. These profilers are designed to measure water column velocity, and backscatter

Figure 1. The ABS submersible acoustic backscatter sediment sensor (Sequoia, 2016). A, closeup of the sensor. B, typical deployment of the sensor.

Introduction 3 Project partnered with USGS Wyoming-Montana and New Mexico Water Science Centers to examine if standard, lowtech laboratory equipment could be used to perform calibration checks on ABS sensors under long-term deployment.

average calibration drifts ranging from 5 to 61 percent per year; and maximum drifts from individual concentration bins have ranged from 10 to 92 percent (table 1). Of the four ABS sensors tested, only one stayed within 10 percent per year across all concentration bins (serial no. 6083). These magnitudes of calibration drift suggest a need to develop methods for checking calibration drift in ABS sensors deployed in the field. Currently (2023), the manufacturer recommends their lab perform calibrations, which requires ending the deployment and returning the sensor to the manufacturer (Sequoia, 2016). Returning the sensor thus requires the user to obtain a second sensor or accept data gaps of weeks to months, which is the approximate common range of time reported by USGS users for the manufacturer to calibrate and return the instrument. These issues could be avoided if the user could apply basic calibration checks in the field or remove the ABS sensor for short periods (less than 1 day) for checks in a laboratory. As part of its mission to unify and standardize research and development activities of Federal agencies involved in fluvial sediment studies, the Federal Interagency Sedimentation

Purpose and Scope The purpose of this report is to provide interim guidance for using the ABS sensor. This report summarizes the results of laboratory experiments that tested materials and methods to check calibration drift of the ABS sensor. Based on the criteria of Giesen (2015), the experiments tested the robustness, repeatability, and reliability of the materials and methods. The experiments did not test reproducibility of the materials and methods because the experiments were performed by the same operator, which is a common scenario for USGS continuous water-quality instrument deployments. An example application of one of the methods at a current deployment site is also presented. All data collected for this report are available in a USGS data release (O’Connell and others, 2023).

Table 1. Magnitude of calibration drift observed in four submersible acoustic backscatter (ABS) sediment sensors deployed at U.S. Geological Survey streamgages in Montana and Wyoming, 2019 to 2020. [Calibration data are provided by the manufacturer of the ABS sensor, Sequoia Scientific Inc.; streamgage information is from the National Water Information System database (USGS, 2022)]

ABS sensor serial no.

Calibration drift1, in percent per year

USGS streamgage identifier and name

Average

Maximum

Minimum

6083

−5%

−10%

06284010—Shoshone Road below Willwood Dam near Ralston, Wyoming.

6122

−19%

−40%

12324400—Clark Fork above Little Blackfoot River near Garrison, Montana.

6127

−21%

−34%

−3%

12324200—Clark Fork near Deer Lodge, Montana.

6149

−61%

−92%

−33%

06283995—Shoshone River above Willwood Dam near Ralston, Wyoming.

1These drifts are percent deviations calculated from six concentrations of 75- to 90-micrometer Ballontini glass impact beads (1.0, 3.0, 10.1, 100, 1,000, and 10,000 milligrams per liter).

4 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor

Methods

ABS Sensors

The apparatus components chosen for the laboratory experiments and field calibration checks consisted of a benchtop stir plate, 2-liter (L) glass beaker, stir bar, and benchtop stand (fig. 2). These components are widely available in most USGS laboratories. Three ABS sensors were used in each step of the laboratory experiments, and a fourth was used for the field calibration check. The laboratory experiments consisted of two components:

Four ABS sensors were used for the experiments described in this report. The manufacturer uses a unique fourdigit serial number to track its ABS sensors. Three sensors, serial numbers 6122, 6178, and 6211, were used for the laboratory experiments; sensor 6223 was used for the field calibration check. The sensors used in the laboratory experiments were all calibrated by the manufacturer in February 2022 and were not field deployed before the experiments. The sensor used for the field calibration check was calibrated by the manufacturer in September 2020 and then remained in the office until deployment in April 2022.

1. Different combinations of stir bar geometries were tested for efficacy and accuracy of suspending different standardized particles over a range of standard concentrations. 2. The repeatability of one of the experimental procedures was tested using one of the combinations of stir bar and standardized particle materials deemed most accurate from the first set of experiments. A fundamental assumption of the experiments is that, because the time frame between the experiments was hours to days, instrument calibrations did not drift before or during the experiments.

ABS sensor

Turbidity sensors

Experiment Apparatus Each ABS sensor was suspended vertically with a benchtop lab stand into a 2-L beaker with the sensor head submerged about 1 cm into the water column (fig. 2). The inside diameter of the beaker was 12.7 cm. Water agitation to suspend standardized particles in the beaker was achieved using a magnetic stir bar propelled by a Fisher Scientific 11–502–49SH magnetic stir plate. The sample volume of the ABS sensor is 5.5 cm from the transducer head, but the total sample distance can extend to 15 cm (Sequoia, 2016). To avoid acoustic interference with the stir bar or sidewalls of the beaker, a minimum distance of 15 cm was maintained from the transducer face to the top of the stir bar.

Experiment Stir Bar Geometries

Stand

2-liter glass beaker

Stir plate

Figure 2. Benchtop apparatus used during the laboratory experiments.

Magnetic stir bars are commonly used in laboratories to generate shear and turbulent forces to mix a liquid with dissolved or solid constituents. For dissolved constituents, stir bars are effective at maintaining consistent vertical and horizontal mixtures. However, for suspensions of solids, differences in settling velocity between grains can create vertical differences in sediment concentration, and localized vortices can allow for coarser grains to settle in patches. The standardized particles used by the manufacturer to calibrate the ABS sensor range from 75 to 90 µm and have settling velocities of about 0.1 to 1 centimeter per second (Dietrich, 1982). These settling velocities are fast enough to settle out of the mixture and cause a time-dependent trend in measured standard particle concentrations, thus requiring some investigation of the efficacy of different stir bar geometries. Three stir bar styles were chosen to compare relative effectiveness of suspending stable concentrations of standardized particles: polygon, cross, and wedge (fig. 3). These stir bars are available from various vendors, and the stir bars used in our experiments ranged from about 6 to 8 cm in length (table 2). Vendors have different descriptions of the stirring characteristics and uses of each shape of stir bar (table 2). The stir bar descriptions shown in table 2 are summaries of various vendor descriptions and were not verified by the USGS.

Methods 5 A. Polygon-shaped stir bar

B. Cross-shaped stir bar

C. Wedge-shaped stir bar

Figure 3. Three types of magnetic stir bars tested for suspended-sediment concentration accuracy during the laboratory experiments.

Table 2. Magnetic stir bars tested for suspended-sediment concentration accuracy during the laboratory experiments. Length by diameter, in centimeters

Common vendor description of use

Polygon

7.5 × 1.2

General usage; mixing dissolved constituents.

Cross

6.0 × 2.0

High speed turbulence and less splashing.

Wedge

8.0 × 1.8

Low speed turbulence; bottom scraping.

Type/geometry

Accuracy of SSCs produced by each stir bar was most important across the range of typical higher calibration concentrations (100, 1,000, and 10,000 mg/L) because these are the conditions when settling of grains was hypothesized to have the greatest effect on measured concentrations.

Experiment Standardized Particle Materials The sensitivity of the ABS sensor signal is nearly constant between particle sizes of about 60 to 100 μm in diameter (coarse silt to very fine sand; Agrawal and others, 2019). Because of the lack of sensitivity of the ABS sensor in this grain-size range, the manufacturer uses 40- to 90-μm glass beads, sieved to 75 to 90 μm, to calibrate the sensor across a range of concentrations. Typical calibration concentrations are 0, 1.1, 3.2, 10.4, 100, 1,000, and 10,000 mg/L. The term “standardized particles” is used in this report in reference to materials for which the distribution of nominal diameters of the b-axis are known with a reasonable level of precision. For

the purposes of this report, standardized particles were used to create suspensions of known concentration (called “standard concentrations” in this report). Three standardized particle sets were used to create standard concentrations with which to check ABS sensor measurements (table 3). The first set of standardized particles was the 75- to 90-μm glass beads sieved and furnished by the manufacturer (called “manufacturer standards” in this report). This set of particles was used to test the potential of the apparatus to accurately reproduce the manufacturer’s calibration concentrations. The second set of standardized particles was raw (direct from glass bead manufacturer), 40- to 90-μm glass beads, which were used to test the accuracy of off-the-shelf standards (called “raw standards” in this report). The third set of standardized particles were the raw standards sieved by the USGS New Mexico Water Science Center to 75- to 90-μm (called “USGS standards” in this report). The USGS standards were used to provide some estimate of error between particles sieved by two operators.

Table 3. Standardized particle materials used during the laboratory experiments. Origin

Particle size range, in micrometers

Description

ABS manufacturer

75–90

Sieved by Sequoia Scientific, Inc.

Grainger

40–90

Raw material from the glass bead manufacturer.

Grainger/U.S. Geological Survey

75–90

Sieved by the U.S. Geological Survey New Mexico Water Science Center.

6 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor

Laboratory Experiments Sediment standard concentration test intervals for particle standards were chosen to replicate the ABS manufacturer’s calibration intervals: 0, 1.1, 3.2, 10.4, 100, 1,000, and 10,000 mg/L. Standard concentrations were achieved by weighing particle mass using a Mettler AE160 digital analytical balance. Water volume was quantified by taring the balance with a dry beaker and incrementally adding room-temperature deionized water. After mass was determined, standards were added to the water column and allowed to mix for 2 minutes before submerging the sensor face. Live readings were made using the ABS sensor manufacturer’s software and were monitored for stability before collecting 20 sequential measurements (Sequoia, 2016). If a trend or instability was observed in the data, the cause was often bubble nucleation on the sensor face, or a beaker adjustment was needed to ensure the stir bar was rotating in the center of the beaker. If the stir bar was not precisely in the center of the beaker, sediment pockets often developed on the bottom of the beaker, and adjustments were needed to ensure suspension of all particles. Sediment pockets were most likely to develop at concentrations greater than or equal to 1,000 mg/L. Measurements at lower concentrations, even if inaccurate, did not seem to destabilize by minor misalignment of the stir bar.

Error Measurements Calibration checks are commonly reported in percent difference of the sensor measurement relative to a standard because the measurement range of water-quality sensors often spans several orders of magnitude. Thus, for the purposes of the experiments described in this report, the error of concentrations measured with the ABS sensor was quantified as the mean percent difference (MPD) relative to the standard (expected) concentration: _ x i − xs  100 n _ δ = _ n ∑ i=1 x  , s

(1)

where

_ δ is the MPD between a measured concentration and the expected concentration, n is the number of sample measurements, xi is the measured concentration, and xs is the expected concentration. The term “expected” is used here to indicate a standard concentration that can be repeated, even if inaccurate. This term is rooted in the notion that a check on instrument drift is still useful if it can be repeated because calibration drift should still be indicated by deviation from the expected value. Standard USGS guidance for continuous water-quality instrumentation considers errors less than or equal to 5 percent to be within calibration tolerance, and errors beyond about 10 percent to indicate a need for instrument recalibration

(Wagner and others, 2006); thus, the primary measure of accuracy used herein is the mean absolute percent difference (MAPD), which uses a modified version of equation 1, where the absolute value of the difference in the numerator on the right side of the equation is used instead of the simple difference. The root-mean-square-error (RMSE) was calculated as a second measure of accuracy in the units of measurement (milligrams per liter): ____________

√

∑ ni=1 (x i − xs  )  2 RMSE = ____________      , n

(2)

where

RMSE is the root mean square error. The RMSE was most useful for quantifying instrument error in the native units of the instrument.

Experiment Repeatability A crucial element of instrument calibration checks is that, under typical field or laboratory conditions, checks made by a user can be reliably repeated over time. After the first set of experiments, initial calculations were made to determine which combination of stir bar and standardized particles produced the most accurate measurements. This combination was then used in a second set of experiments in which two back-to-back sets of experiments were run to test for repeatability of experimental methods under optimal conditions. Although measurement accuracy relative to a standard is ideal, if a standard (expected) concentration can be reliably repeated, it can still act as a baseline from which to measure instrument calibration drift. Thus, the MPD was calculated using the measured concentrations from the second set of experiments as xi and expected concentrations from the first set of experiments as xs in equation 1.

Field Application One of the calibration checks developed using laboratory experiments was applied to an ABS sensor currently deployed in the field at the Animas River near Cedar Hill, New Mexico, streamgage (USGS streamgage 09363500 [not shown]; USGS, 2023). The deployment includes the ABS sensor alongside turbidity sensors, which are used with SSC measurements to build statistical regression models for predicting SSC for a separate project. Predicted values of SSC are then compared with samples analyzed for suspended metals to understand metal transport in the Animas River. The ABS sensor and turbidity data are foundational for estimates of SSC and metal loads for that project. It was thus ideal to use a method for verifying calibration of the ABS over the term of the deployment. A field calibration check station for the ABS sensor was set up inside the gage house of USGS streamgage 09363500, which is an about 8- by 10-foot steel shed with shelving and a countertop (fig. 4). The check station was set up on the

Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks 7

Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks on a Submersible Acoustic Backscatter Sediment (ABS) Sensor

Figure 4. Benchtop apparatus used during the field application at the U.S. Geological Survey streamgage on the Animas River near Cedar Hill, New Mexico (09363500; USGS, 2023).

countertop and included the following: battery-powered Fisher Scientific stir plate, 1-L Pyrex beaker, polygon stir bars, deionized water for rinsing and mixing calibration solutions, and a laboratory stand with flask clamps to hold the ABS sensor in the 1-L beaker in proper orientation to avoid interference with the sides and bottom of the beaker. Glass scintillation vials (20 milliliters) were used to hold pre-weighed glass beads to make standard concentrations of 10.4, 100, 1,000, and 10,000 mg/L. The standardized particles were weighed in the USGS New Mexico Water Science Center Sediment Laboratory and were added to a known volume of water in the field to arrive at the exact concentration. The 1.1 and 3.2 mg/L concentrations were omitted in field calibration checks because it was a range of instrument operation that was not critical to the project objectives, and preliminary results from laboratory experiments indicated substantial potential for error at those standard concentrations. During each monthly site visit, field personnel removed the ABS sensor from the housing at the river’s edge, cleaned the instrument to avoid introducing sediment to the calibration check process, and brought the instrument to the gage house. The materials for calibration checks included the 7.5-cm polygon stir bar and glass beads sieved by the USGS to nominal diameters of 75 to 90 µm; the primary measure of error was recorded as MPD between the expected concentration and 20 to 60 measured concentrations.

Laboratory experiments and field applications of the experiments indicate that accurate calibration checks of the ABS sensor can be made by USGS personnel with a standard laboratory apparatus of stand, beaker, and stir plate. MAPDs relative to the expected concentrations were lowest for raw glass beads and USGS-sieved glass beads combined with cross- or polygon-shaped stir bars (table 4). MAPDs relative to the standard were generally within the acceptable range of 10 percent for standardized concentrations greater than or equal to 100 mg/L and were near 10 percent for the 10.4 mg/L standard concentration. MAPDs were generally greater than 10 percent at concentrations less than 10.4 mg/L. Field applications of one of the experimental methods indicated accurate tracking of calibrations across a 5-month deployment period.

Mixing Materials and Methods The wedge stir bar had difficulty maintaining speed in the mixing chamber, creating erratic readings, and experimentation with that stir bar was abandoned. The polygon and cross-shaped bars combined with the raw standardized particles provided the most consistent results; MAPDs for higher concentrations were generally less than 10 percent. The cross-shape bar had slightly smaller errors across the range of standard calibration concentrations, but these errors were not judged to be significant. Speed of the stir bar played a critical role in suspending the standardized particles. Low speed resulted in particles settling out; too high of a speed resulted in the formation of a vortex that would interfere with the ABS sensor backscatter signal. A moderately high speed of 8 on the stir plate produced consistent mixing results. Optimal mixing speeds may vary by stir plate mode and manufacturer and should be adjusted as necessary.

8 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor Table 4. Summary of mean absolute percent differences relative to standard concentrations for the laboratory experiments (O’Connell and others, 2023). [Data are summarized from O’Connell and others (2023). Manufacturer, standard Ballontini glass beads sieved to 75- to 90-micrometer diameters by Sequoia Scientific, Inc., and provided to the U.S. Geological Survey; Raw Ballontini, standard Ballontini glass beads of 40- to 90-micrometer diameters available from an outside vendor; USGS-sieved Ballontini, standard Ballontini glass beads sieved to 75- to 90-micrometer diameters by the U.S. Geological Survey, New Mexico Water Science Center; Polygon, polygon-shaped stir bar; Cross, cross-shaped stir bar; --, data not measured]

Range of concentrations, in milligrams per liter

Mean absolute percent difference1 Manufacturer Polygon

Raw Ballontini

Cross

USGS-sieved Ballontini

Polygon

Cross

Polygon

Cross

ABS sensor 6122 1.1, 3.2, 10.4, 100, 1,000, and 10,000

100, 1,000, and 10,000

ABS sensor 6178 1.1, 3.2, 10.4, 100, 1,000, and 10,000

100, 1,000, and 10,000

1.1, 3.2, 10.4, 100, 1,000, and 10,000

100, 1,000, and 10,000

ABS sensor 6211

1Average of absolute values of mean percent difference relative to standard across range of concentrations shown.

Experiments with Manufacturer Standardized Particles MPD relative to the standard concentration using ABS manufacturer standardized particles varied by instrument and standard concentration, but a general pattern of the magnitude of MPD being inversely proportional to standard concentration was consistent across instruments (fig. 5). The MPDs of measured concentrations were all positive, indicating the instrument was consistently reading higher than the standard concentration. At the lowest standard concentrations of 1.1 and 3.2 mg/L, MPDs varied from 58 to 219 percent (table 5) but decreased to between 6.4 and 29 percent at the highest standard concentrations of 1,000 and 10,000 mg/L. Behavior of RMSE was not consistent across instruments; instruments 6178 and 6211 had smaller relative RMSE values at lower concentrations relative to instrument 6122, and RMSE values generally aligned at higher standard concentrations. Experiments using the ABS manufacturer standardized particles and the cross-shaped stir bar showed accuracy and behaviors similar to those with the polygon-shaped stir bar in that the MPDs were consistently higher at lower concentrations and improved with increasing standard concentration (fig. 5). Magnitudes of MPD were like those produced using the polygon-shaped stir bar except for measurements at the 100-mg/L standard concentration, which were consistently lower than the expected concentration (table 6). Root-mean square errors were consistently close between instruments for concentrations as much as 1,000 mg/L but differed by as much as a factor of four between instruments at the 10,000-mg/L standard concentration (table 6).

Experiments with Raw Standardized Particles MPD relative to the standard using raw standardized particles exhibited similar patterns as those for sensor manufacturer standardized particles whereby the largest percent differences were observed at low concentrations and the smallest at higher concentrations (fig. 6), but maximum percent differences were mostly negative at low concentrations. The magnitude of percent differences relative to the standard also declined substantially relative to those observed for Sequoia standardized particles; maximum MAPD was 20 percent across the range of concentrations and 10 percent across concentrations greater than 10.4 mg/L (table 4). MPD did not change substantively among stir bar types over the range of standard concentrations and for high concentrations (greater than 10.4 mg/L). Maximum MPDs observed using the raw standardized particles and the polygon-shaped stir bar were largest for low concentrations and diminished for concentrations greater than or equal to 10 mg/L (fig. 6). MPDs for each standard concentration ranged from −80 to 16 percent at low standard concentrations of 1.1 and 3.2 mg/L but decreased to a range of −2.5 to 14 percent for concentrations of 100 mg/L or greater (table 7). Instrument 6211 read consistently lower than the other instruments when the polygon-shaped stir bar was used, but no consistent bias was observed over the range of concentrations in the other instruments (fig. 6). Root-mean square errors were generally consistent across instruments in that values fell within a factor of two across instruments (table 7). MPDs observed using the combination of raw standardized particles and the cross-shaped stir bar exhibited the same behavior as those for polygon-shaped stir bar, whereby the

Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks 9 A. Measured versus standard concentration, polygon-shaped stir bar

B. MPD versus standard concentration, polygon-shaped stir bar 250

1,000

1:1 10 percent error region

100

Mean percent difference in measured sediment concentration

Measured sediment concentration, in milligrams per liter

10,000

200 150 100 50

5 percent error region

0 10 percent error region

−50 −100

C. Measured versus standard concentration, cross-shaped stir bar

D. MPD versus standard concentration, cross-shaped stir bar 250

Mean percent difference in measured sediment concentration

Measured sediment concentration, in milligrams per liter

10,000

1,000

100 1:1 10 percent error region

200 150 100 50

5 percent error region

0 10 percent error region

−50 −100

100

1,000

10,000

Standard sediment concentration, in milligrams per liter

100

1,000

Standard sediment concentration, in milligrams per liter EXPLANATION Error bars—A and C represent the root-meansquare error; B and D represent the range of mean percent differences (MPDs) Instrument 6122 Instrument 6178 Instrument 6211

Figure 5. Relations between standard and measured sediment concentrations, or standard sediment concentrations and mean percent differences in measured sediment concentrations using ABS manufacturer standardized particles (O’Connell and others, 2023).

10,000

10 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor Table 5. Summary statistics of suspended-sediment concentrations measured in the laboratory using the ABS manufacturer standardized particles and a polygon-shaped stir bar. [Data are summarized from O’Connell and others (2023); mg/L, milligram per liter; MPD, mean percent difference; RMSE, root mean square error]

Summary statistic

Standard concentration, in milligrams per liter 1.1

3.2

10.4

100

1,000

10,000

ABS sensor 6122 Mean measured concentration, mg/L

3.51

6.6

115

1,119

11,698

Median measured concentration, mg/L

3.50

6.6

115

1,121

11,650

15.2

11.9

4.7

15.5

119

1,706

MPD of measured concentration relative to standard concentration

219

113

RMSE of measured concentration relative to standard concentration, mg/L

2.41

Mean measured concentration, mg/L

3.06

6.3

126

1,064

10,179

Median measured concentration, mg/L

3.04

6.4

13.1

125.2

1,063

10,154

32.0

25.6

6.4

3.24

25.7

223

3.51 ABS sensor 6178

MPD of measured concentration relative to standard concentration

178

105

1.8

RMSE of measured concentration relative to standard concentration, mg/L

1.97

Mean measured concentration, mg/L

2.90

4.9

106

1,113

12,902

Median measured concentration, mg/L

2.88

4.9

106

1,110

12,890

28.7

5.8

11.3

1.82

2.89

6.0

113

2,909

3.25 ABS sensor 6211

MPD of measured concentration relative to standard concentration RMSE of measured concentration relative to standard concentration, mg/L

164 1.81

Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks 11 Table 6. Summary statistics of suspended sediment measurements made in the laboratory using the ABS manufacturer standardized particles and a cross-shaped stir bar. [Data are summarized from O’Connell and others (2023); mg/L, milligram per liter; MPD, mean percent difference; RMSE, root mean square error]

Summary statistic

Standard concentration, in milligrams per liter 1.1

3.2

10.4

100

1,000

10,000

ABS sensor 6122 Mean Meanmeasured measuredconcentration, concentration,mg/L mg/L

3.29

6.7

1,059

11,264

Median Medianmeasured measuredconcentration, concentration,mg/L mg/L

3.31

6.8

1,062

11,280

47 47

−5.7 −5.7

5.9 5.9

13 13

4.8 4.8

6.7 6.7

62 62

1,278 1,278

6.0

1,062

10,554

1,059

10,540

MPD MPDofofmeasured measuredconcentration concentrationrelative relative totostandard standardconcentration concentration

199 199

RMSE RMSEofofmeasured measuredconcentration concentrationrelative relative totostandard standardconcentration, concentration,mg/L mg/L

2.20 2.20

Mean Meanmeasured measuredconcentration, concentration,mg/L mg/L

3.42

117 117 3.63 3.63

ABS sensor 6178 Median Medianmeasured measuredconcentration, concentration,mg/L mg/L MPD MPDofofmeasured measuredconcentration concentrationrelative relative totostandard standardconcentration concentration

3.42 211 211

6.0

15.3

95.36

94 94

52.9 52.9

−5.3 −5.3

6.2 6.2

2.92 2.92

5.30 5.30

6.5 6.5

64 64

592 592

5.5 5.5

RMSE RMSEofofmeasured measuredconcentration concentrationrelative relative totostandard standardconcentration, concentration,mg/L mg/L

2.33 2.33

Mean Meanmeasured measuredconcentration, concentration,mg/L mg/L

2.23

6.5

1,049

12,632

Median Medianmeasured measuredconcentration, concentration,mg/L mg/L

2.29

6.6

1,045

12,610

39.9 39.9

−16.6 −16.6

4.9 4.9

26 26

4.02 4.02

16.6 16.6

54 54

2,646 2,646

ABS sensor 6211

MPD MPDofofmeasured measuredconcentration concentrationrelative relative totostandard standardconcentration concentration RMSE RMSEofofmeasured measuredconcentration concentrationrelative relativeto standard concentration, mg/L to standard concentration, mg/L

103 103 1.15 1.15

110 110 3.42 3.42

12 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor A. Measured versus standard concentration, polygon-shaped stir bar

B. MPD versus standard concentration, polygon-shaped stir bar 250

1,000

1:1 10 percent error region

100

Mean percent difference in measured sediment concentration

Measured sediment concentration, in milligrams per liter

10,000

200 150 100 5 percent error region

50 0

10 percent error region

−50 −100

C. Measured versus standard concentration, cross-shaped stir bar

D. MPD versus standard concentration, cross-shaped stir bar 250

Mean percent difference in measured sediment concentration

Measured sediment concentration, in milligrams per liter

10,000

1,000

100 1:1 10 percent error region

100

1,000

200 150 100

0 10 percent error region

−50 −100

10,000

5 percent error region

Standard sediment concentration, in milligrams per liter

100

1,000

10,000

Figure 6. Relations between standard concentrations and measured concentrations, or standard concentrations and mean percent differences of measured concentrations using raw standardized particles (O’Connell and others, 2023).

Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks 13 Table 7. Summary statistics of suspended-sediment concentration measurements made in the laboratory using raw standardized particles and a polygon-shaped stir bar. [Data are summarized from O’Connell and others (2023); mg/L, milligram per liter; MPD, mean percent difference; RMSE, root mean square error]

Summary statistic

Standard concentration, in milligrams per liter 1.1

3.2

10.4

100

1,000

10,000

ABS sensor 6122 Mean Meanmeasured measuredconcentration, concentration,mg/L mg/L

0.57

3.6

106

1,077

11,431

Median Medianmeasured measuredconcentration, concentration,mg/L mg/L

0.57

3.6

107

1,076

11,430

16 16

17 17

6.5 6.5

7.7 7.7

14 14

0.56 0.56

1.8 1.8

7.0 7.0

77 77

1,434 1,434

MPD MPDofofmeasured measuredconcentration concentrationrelative relative totostandard standardconcentration concentration

−48 −48

RMSE RMSEofofmeasured measuredconcentration concentrationrelative relative totostandard standardconcentration, concentration,mg/L mg/L

0.53 0.53

Mean Meanmeasured measuredconcentration, concentration,mg/L mg/L

0.95

0.96 −14 −14

3.4

107

1,069

10,865

107

1,071

10,860

3.4

10.0

11 11

1.1 1.1

7.0 7.0

6.9 6.9

0.37 0.37

0.42 0.42

7.2 7.2

70 70

870 870

8.7 8.7

RMSE RMSEofofmeasured measuredconcentration concentrationrelative relativeto standard concentration, mg/L concentration, mg/L to standard

0.18 0.18

Mean Meanmeasured measuredconcentration, concentration,mg/L mg/L

0.22

2.7

1,035

11,310

Median Medianmeasured measuredconcentration, concentration,mg/L mg/L

0.21

2.7

1,035

11,280

5.8 5.8

−2.5 −2.5

3.5 3.5

13 13

0.64 0.64

2.7 2.7

38 38

1,326 1,326

ABS sensor 6211

MPD MPDofofmeasured measuredconcentration concentrationrelative relativeto standard concentration concentration to standard RMSE RMSEofofmeasured measuredconcentration concentrationrelative relativeto standard concentration, mg/L concentration, mg/L to standard

−80 −80

−12 −12

0.89 0.89

0.42 0.42

largest errors were observed at low concentrations and diminished with increasing concentration (fig. 6). MPDs for each standard concentration ranged from −66 to 28 percent at low standard concentrations of 1.1 and 3.2 mg/L but decreased to a range of −5.9 to 23 percent for concentrations of 100 mg/L or greater (table 8). Instrument 6211 read consistently lower than the other instruments for concentrations of 10.4 mg/L or less, but no consistent bias was observed among other instruments or standard concentrations (fig. 6). Magnitude of RMSE was generally consistent across instruments and concentrations up to 1,000 mg/L but differed by as much as a factor of five for standard concentrations of 10,000 mg/L (table 8).

Experiments with USGS Standardized Particles Experiments using USGS standardized particles were only done with the cross-shaped stir bar. The magnitude and pattern of MPDs relative to the standard concentrations were like those observed for the experiments with the raw standardized particles because errors where largest and negative for low concentrations and lowest (positive or negative) at higher concentrations (fig. 7). Maximum MAPD values for USGS standardized particles were 21 percent across the range concentrations and 8 percent across concentrations greater than

10.4 mg/L (table 4). MPDs for each standard concentration ranged from −70 to 17 percent for standard concentrations less than 10.4 mg/L and −10.1 to 17 percent for standard concentrations greater than 100 mg/L (table 9). Instrument 6211 had the largest errors on average, but MAPDs for that instrument were less than 10 percent for concentrations greater than 10.4 mg/L.

Repeatability of Experiments To test the precision of the methods described above, two sequential experiments were run within the same day. Raw standardized particles and a cross-shaped stir bar were chosen to create the conditions for quantifying repeatability of experiments. This combination was chosen because ideal standardized particles would be those that have the least cost and best accuracy. Raw standardized particles meet the criteria of low cost because they are readily available from commercial vendors and do not require additional processing. Likewise, the results of the experiments described above indicate that the raw standardized materials and cross-shaped stir bar produced results equal to or better than those observed using other combinations of stir bar and standardized particles.

14 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor Table 8. Summary statistics of suspended-sediment concentration measurements made in the laboratory using raw standardized particles and a cross-shaped stir bar. [Data are summarized from O’Connell and others (2023);. mg/L, milligram per liter; MPD, mean percent difference; RMSE, root mean square error]

Summary statistic

Standard concentration, in milligrams per liter 1.1

3.2

10.4

100

1,000

10,000

ABS sensor 6122 Mean measured concentration, mg/L

0.84

3.9

107

980

11,194

Median measured concentration, mg/L

0.86

3.9

108

976

11,200

7.5

−2

0.77

2.5

8.0

1,201

MPD of measured concentration relative to standard concentration

−24

RMSE of measured concentration relative to standard concentration, mg/L

0.29

Mean measured concentration, mg/L

1.07

3.5

1,085

10,409

1,084

10,400

ABS sensor 6178 Median measured concentration, mg/L

1.05

3.5

12.8

93.88

MPD of measured concentration relative to standard concentration

−3

−5.9

8.5

RMSE of measured concentration relative to standard concentration, mg/L

0.11

0.48

2.83

6.2

425

Mean measured concentration, mg/L

0.37

2.8

101

1,052

12,263

Median measured concentration, mg/L

0.35

2.9

101

1,052

12,200

−9

9.3

1.1

5.2

0.37

1.01

1.8

2,290

4.1

ABS sensor 6211

MPD of measured concentration relative to standard concentration RMSE of measured concentration relative to standard concentration, mg/L

−66 0.73

Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks 15 A. Measured versus standard concentration, cross-shaped stir bar

B. MPD versus standard concentration, cross-shaped stir bar 250

1,000

Mean percent difference in measured sediment concentration

Measured sediment concentration, in milligrams per liter

10,000

1:1 10 percent error region

100

1,000

200 150 100

0 10 percent error region

−50 −100

10,000

5 percent error region

Standard sediment concentration, in milligrams per liter

100

1,000

10,000

Standard sediment concentration, in milligrams per liter EXPLANATION Error bars—A represents the root-meansquare error; B represents the range of mean percent differences (MPDs) Instrument 6122 Instrument 6178 Instrument 6211

Figure 7. Relations between standard concentrations and measured concentrations, or standard concentrations and mean percent differences of measured concentrations using U.S. Geological Survey standardized particles and a cross-shaped stir bar (O’Connell and others, 2023).

16 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor Table 9. Summary statistics of suspended-sediment measurements made in the laboratory using U.S. Geological Survey standardized particles and a cross-shaped stir bar. [Data are summarized from O’Connell and others (2023);. mg/L, milligram per liter; MPD, mean percent difference; RMSE, root mean square error]

Summary statistic

Concentration of standard, in milligrams per liter 1.1

3.2

10.4

100

1,000

10,000

ABS sensor 6122 Mean measured concentration, mg/L

0.45

3.3

100

998

11,744

Median measured concentration, mg/L

0.46

3.3

100

1,006

11,770

−0.2

0.24

0.4

2.7

1,752

MPD of measured concentration relative to standard concentration

−59

RMSE of measured concentration relative to standard concentration, mg/L

0.65

Mean measured concentration, mg/L

0.55

3.3

1,020

11,013

Median measured concentration, mg/L

0.53

3.3

9.5

90.47

1,023

10,970

−5.3

−10.1

0.26

0.62

10.3

1,022

ABS sensor 6178

MPD of measured concentration relative to standard concentration

−50

10.1

RMSE of measured concentration relative to standard concentration, mg/L

0.55

Mean measured concentration, mg/L

0.33

2.6

918

9,357

Median measured concentration, mg/L

0.33

2.6

919

9,350

−15.4

−8.0

−8.2

−6

1.59

8.1

666

ABS sensor 6211

MPD of measured concentration relative to standard concentration RMSE of measured concentration relative to standard concentration, mg/L

−70 0.77

−17 0.57

For the purposes of testing repeatability of creating standardized conditions, standardized conditions were reproduced for concentrations of 1.1, 10.4, 100, 1,000, and 10,000 mg/L. Only 2 of the 15 MAPD values between first and second experiments exceeded 10 percent, and both were for concentrations of 10.4 mg/L or less, where previous experiments demonstrated large errors (fig. 8). For standardized concentrations of 100 mg/L or greater, MAPD values were all less than 10 percent, and eight out of nine measurements were less than 7.5 percent (table 10), indicating conditions were reproducible within the maximum limits generally accepted by the USGS for requiring re-calibration. Despite the relatively small errors between the repeated experiments, unpaired, nonparametric statistical hypothesis tests indicate that the differences between experiments were significant at the 95-percent confidence level for 13 of 15 experiments (table 10). These data suggest that the experimental conditions produced by the experiments described here were not stable enough, or within-instrument precision was

not high enough, to produce conditions that were statistically indistinguishable within the 10-percent error bounds accepted by the USGS.

Field Application Results Monthly field calibration checks were completed at USGS streamgage 09363500 (USGS, 2023) from July through November of 2022 (table 11). All concentrations measured during the calibration checks were higher than the standard concentration; however, all MAPDs except one were within 10 percent, and 12 of the 20 checks were 5 percent or less. As expected, errors were highest for the smallest concentration of 10.1 mg/L, but only one of the checks exceeded the 10 percent threshold at that concentration. Across the 5 months, none of the calibration check values indicated a consistent increasing or decreasing trend, suggesting that the sensor calibration was not drifting.

Experiments and Field Applications to Support Interim Guidance for Doing Calibration Checks 17 B. MPD versus standard concentration, cross-shaped stir bar

10,000

250

1,000

Mean percentage difference between first sample and replicated sample

Measured concentration in replicate sample, in milligrams per liter

A. Measured versus standard concentration, cross-shaped stir bar

1:1 10 percent error region

100

1,000

200 150 100

0 10 percent error region

−50 −100

10,000

5 percent error region

Measured concentration in first sample, in milligrams per liter

100

1,000

Standard concentration in first sample, in milligrams per liter EXPLANATION

Error bars A represents the root-mean-square error relative to the standard concentration B represents the range of mean percent differences (MPDs) between first and second experiments at each concentration Instrument 6122 Instrument 6178 Instrument 6211

Figure 8. Relations among concentrations and mean percent differences from a replicated experiment using raw standardized particles and a cross-shaped stir bar (O’Connell and others, 2023).

10,000

18 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor Table 10. Statistics of suspended-sediment concentration measurements made during a replicate experiment in the laboratory using raw standardized particles and a cross-shaped stir bar. [Data are summarized from O’Connell and others (2023); mg/L, milligram per liter; MPD, mean percent difference; RMSE, root mean square error; p-value, probability of declaring there is a signal in the data when one does not exist. In this case, the signal is a true difference between the mean concentrations from the first and replicate experiments]

Standard concentration, in milligrams per liter

Summary statistic

1.1

10.4

100

1,000

10,000

ABS sensor 6122 Mean measured concentration— First experiment, mg/L

0.90

119

1,176

11,581

Mean measured concentration— Replicate experiment, mg/L

0.94

121

1,203

12,116

MPD of replicate experiment relative to first experiment

4.5

−13.1

Wilcoxen Rank-Sum Test statistic p-value

140

400

1.0E−01

6.7E−08

1.7 100 6.7E−03

2.2

4.6

4.7E−06

5.9E−07

ABS sensor 6178 Mean measured concentration— First experiment, mg/L

0.90

100

1,188

10,927

Mean measured concentration— Replicate experiment, mg/L

0.86

110

1,099

10,989

MPD of replicate experiment relative to first experiment

−4.0

−7.2

9.3

−7.5

Wilcoxen Rank-Sum Test statistic

279

383

400

6.8E−08

6.6E−08

p-value

3.4E−02

8.4E−07

0.6 163 3.2E−01

ABS sensor 6211 Mean measured concentration— First experiment, mg/L

0.41

104

1,154

13,386

Mean measured concentration— Replicate experiment, mg/L

0.28

110

1,070

14,177

MPD of replicate experiment relative to first experiment

−32

−5.2

6.2

−7.2

5.9

Wilcoxen Rank-Sum Test statistic

375

341

400

1.4E−07

6.6E−08

2.8E−06

p-value

2.3E−06

1.4E−04

Discussion 19 Table 11. Summary of field calibration checks at the U.S. Geological Survey streamgage, Animas River near Cedar Hill, New Mexico (09363500). [Data are summarized from O’Connell and others (2023); mg/L, milligram per liter; MPD, mean percent difference]

Summary statistic

Standard concentration, in milligrams per liter 10.1

100

1,000

10,000

109

1,050

10,200

July 6, 2022 Mean measured concentration, mg/L

11.3

MPD of measured concentration relative to standard concentration

11.9

9.0

5.0

2.0

August 16, 2022 Mean measured concentration, mg/L

10.7

MPD of measured concentration relative to standard concentration

5.9

102 2.0

1,050 5.0

10,400 4.0

September 14, 2022 Mean measured concentration, mg/L

10.8

MPD of measured concentration relative to standard concentration

6.9

103 3.0

1,050 5.0

10,400 4.0

October 20, 2022 Mean measured concentration, mg/L

10.8

MPD of measured concentration relative to standard concentration

6.9

103 3.0

1,060 6.0

10,500 5.0

November 9, 2022 Mean measured concentration, mg/L

10.8

MPD of measured concentration relative to standard concentration

6.9

Discussion The laboratory experiments and field application described in this report demonstrate that standard laboratory equipment can be used to perform accurate calibration checks on the ABS sensor for long-term deployments. That said, standard laboratory equipment, such as stir plates, can substantially differ between makes and models, and any user should expect to spend some time establishing equilibrium standard conditions for their specific combinations of equipment. Of particular note, we found substantial differences in the stability of standard conditions depending on the speed of the stir bar; low speeds caused a time-dependent trend in concentration for higher standard concentrations, and high speeds created instabilities in standard concentrations. After stable standard conditions were established, the largest source of variability in standard concentrations observed in our experiments was choice of standardized particles. It is unclear why the standardized particles provided by the sensor manufacturer produced larger errors than those of the raw and USGS standardized particles; it is also unclear why the sensor manufacturer standardized particles produced positive errors for low concentrations, whereas those of the raw and USGS standardized particles were negative at low

105 5.0

1,070 7.0

10,400 4.0

concentrations. Positive errors at low concentrations might be expected for mixtures with finer tails in the grain-size distribution (GSD), but the details of the GSD were not investigated for any of the standard particle mixtures beyond knowing the range of nominal diameters present. It is possible that, despite being sieved to between 75 and 90 µm, the sensor manufacturer and USGS standardized particles differed in GSD because of differences in the mixtures used during sieving. For example, a more-even GSD might be expected if one of the mixtures was created from a large, well-mixed batch, whereas a smaller, poorly mixed batch might create uneven GSD during sieving. Additional sources of variability were between instruments and differences between the stability of the standard concentrations in field application relative to those observed in the laboratory experiments. Slight differences between instruments are expected, and errors from within-instrument accuracy and precision can be accounted for by repeat experiments before deployment. The apparent greater stability of the standard concentrations in the field application may be a consequence of slight differences in the benchtop setup. For the laboratory experiments, three turbidity sensors were submerged in the Pyrex beaker next to the ABS sensor (fig. 1). This setup was used to run two experiments simultaneously, of which only one is reported herein. The presence of the three

20 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor turbidity sensors could have introduced additional turbulence to the mixture and caused the apparent lower stability of standard concentrations relative to those observed in the field application where the ABS was the only instrument submerged in the beaker. Additionally, the field application used a 1-L beaker, whereas the laboratory experiments used a 2-L beaker.

Interim Guidance for Calibration Checks on the Acoustic Backscatter Sediment (ABS) Sensor The results of the laboratory experiments and field application described herein can be summarized into the following interim guidance for calibration checks of the ABS sensor over longer term deployments: 1. Choose standard particles. a. Raw glass beads had acceptable ranges of error and are available direct from the bead manufacturer, whereas sieving particles requires additional steps. b. No substantive difference in error was observed between the two standard particles, so either is sufficient if sieving equipment is unavailable. c. If choosing to sieve raw glass beads to create standard particles with nominal diameters between 75 and 90 µm, do so in large batches to ensure the GSD of the mixture represents the intended frequency of nominal particle diameters. 2. Before deployment, establish accuracy of stable conditions for standard concentrations using laboratory equipment available including benchtop stand, stir plate, stir bar, and standard particles. a. Stable conditions can be defined as those wherein measured concentrations are unchanging within a 10-percent tolerance over periods of tens of seconds to minutes. b. Establishing stable conditions with a particular stir bar and stir speed will be beneficial for replicating those conditions on future checks. Here, the crossand polygon-shaped stir bars performed adequately. It is recommended to record the speed setting for each stir plate if multiple stir plates are used. 3. Before deployment, establish precision of stable conditions for your instrument and apparatus using a replicate experiment. a. Choose a range of standard concentrations appropriate for the project. If the range is expected to span 0 to 100 mg/L, for example, standard concentrations of 0, 10, and 100 mg/L may be adequate.

b. Measurements at standard concentrations (even if inaccurate), should be within 10 percent of those from the first experiment. 4. If more than one operator is planned for the calibration checks, steps 2 and 3 should be repeated for each operator to establish between-operator precision. 5. If more than one instrument is planned for deployment over the course of the study, steps 2 and 3 should be repeated for both instruments to establish betweeninstrument precision. 6. If instrument drift is observed, a replicate check is recommended to ensure the drift is within precision of conditions of the visit. Ideally, steps 2 and 3 would be performed after a factory calibration by the manufacturer such that the accuracy of the instrument calibration is ensured to be optimal. However, stable conditions can still be established even if the sensor readings are inaccurate. For example, a stable reading of 15 mg/L for standard concentrations of 10 mg/L can still provide information on instrument drift on subsequent calibration checks allowing for multipoint adjustments to be made in the time series.

Summary Over the past two decades, the U.S. Geological Survey (USGS) and other agencies have pioneered the use of active acoustic sensors to monitor suspended-sediment concentrations (SSCs) and particle sizes in rivers and streams at the subdaily time scale. Traditional deployments of acoustic instrumentation to monitor SSC in streams are commonly attached to a rigid frame, which is secured to a stable body such as a bridge pier, abutment, or bedrock. Such rigid deployments are less successful when alluvial banks are the primary stable body because the banks can erode during high-flow events. The Sequoia Scientific, Inc. (Sequoia), LISST-ABS submersible acoustic backscatter sediment sensor (or “ABS sensor”) was developed as an alternative to turbidity sensors for monitoring SSCs in surface waters. The ABS sensor is different than traditional acoustic instruments because it is small, lightweight, and requires less power; and the sampling volume is within the first 15 centimeters of the transducer face. Initial testing by the USGS indicates the ABS sensor has utility as a novel, cost-effective, off-the-shelf tool for monitoring SSC in surface waters, and its use within the agency has increased in recent years. However, initial testing did not account for the potential of transducer calibration drift over longer deployments. As part of its mission to unify and standardize research and development activities of Federal agencies involved in fluvial sediment studies, the Federal Interagency Sedimentation Project partnered with the USGS Wyoming-Montana and New Mexico Water Science Centers

Summary 21 to examine the potential for use of standard, low-tech laboratory equipment to perform calibration checks on ABS sensors under long-term deployment. The experiments were intended to provide interim guidance to help operate and maintain the ABS sensor. This report summarizes the results of laboratory experiments that tested materials and methods to check calibration drift of the ABS sensor, and a field application of the calibration check approach described herein. The apparatus components chosen for the experiments and field calibration checks consisted of a benchtop stir plate, glass beaker (2 liters for the lab and 1 liter for the field), stir bar, and benchtop stand. These components are widely available in most USGS laboratories. Three Sequoia ABS sensors were used in each step of the laboratory experiments, and a fourth was used for the field deployment. Each ABS sensor was suspended vertically with a benchtop lab stand into the beaker with the sensor head submerged about 1 centimeter into the water column. Water agitation to suspend standardized particles in the beaker was achieved using a magnetic stir bar propelled by a magnetic stir plate. Three styles of stir bar were chosen to compare relative effectiveness of suspending stable concentrations of standardized particles: polygon, cross, and wedge. Three standardized particle sets were used to create standard concentrations for which to check ABS sensor measurements. The first set of standardized particles was the 75- to 90-μm Ballontini glass beads sieved and furnished by the sensor manufacturer (or “sensor manufacturer standards”). The second set of standardized particles was raw, 40- to 90-micrometer Ballontini glass beads, which were used to test the accuracy of off-the-shelf standards (or “raw standards”). The third set of standardized particles was 75- to 90-μm Ballontini glass beads sieved by the USGS New Mexico Water Science Center (or “USGS standards”). The experiments consisted of three steps. First, different combinations of stir-bar geometries were tested for efficacy and accuracy of suspending different standardized particles over a range of standard concentrations. Next, the repeatability of one of the experimental procedures was tested using one of the combinations of stir bar and standardized particle materials deemed most accurate from the first set of experiments. Finally, a method was applied to a USGS field site to test for practical efficacy. Sediment standard concentration test intervals for particle standards were chosen to replicate the sensor manufacturer’s calibration intervals: 0, 1.1, 3.2, 10.4, 100, 1,000, 10,000 milligrams per liter (mg/L). Errors were computed as the mean percent difference of measurements relative to the standard concentration and the root-mean square error (RMSE) relative to the standard concentration, in milligrams per liter. Laboratory experiments and field application of the experiments indicate that accurate calibration checks of the ABS sensor can be made by USGS personnel with a standard laboratory apparatus of stand, beaker, and stir plate. Mean absolute percent differences relative to the standard concentrations were lowest for raw standardized particles and USGS standardized particles combined with cross- or polygon-shaped stir bars. Mean absolute percent differences

relative to the standard concentrations were generally within the acceptable range of 10 percent for standard concentrations greater than or equal to 100 mg/L and were near 10 percent for the 10.4 mg/L standard concentration. Mean absolute percent differences were generally greater than 10 percent at concentrations less than 10.4 mg/L. Field applications of one of the experimental methods indicated accurate tracking of calibrations across a 5-month deployment period. The laboratory experiments and field application described in this report demonstrate that standard laboratory equipment can be used to do accurate calibration checks on the ABS sensor for long-term deployments. Substantial variability was observed in the stability of standard conditions depending on the speed of the stir bar; low speeds caused a time-dependent trend in concentration for higher standard concentrations, and high speeds created instabilities in standard concentrations. Any user should expect to spend some time establishing equilibrium standard conditions for their specific combinations of equipment. After stable standard conditions were established, the largest source of variability in standard concentrations observed in our experiments was choice of standardized particles. Additional sources of variability were between instruments and differences between the stability of the standard concentrations in field application relative to those observed in the laboratory experiments. The results of the laboratory experiments and field application described herein can be summarized into the following interim guidance for calibration checks of the ABS sensor over longer term deployments: 1. Choose standard particles. Raw and USGS standardized particles performed best under the experimental conditions described in this report. 2. Before deployment, establish accuracy of stable conditions for standard concentrations using available laboratory equipment including benchtop stand, stir plate, stir bar, and standard particles. 3. Before deployment, establish precision of stable conditions for your instrument and apparatus using a replicate experiment. 4. If more than one operator is planned for the calibration checks, steps 2 and 3 should be repeated for each operator to establish between-operator precision. 5. If more than one instrument is planned for deployment over the course of the study, steps 2 and 3 should be repeated for both instruments to establish betweeninstrument precision. 6. If instrument drift is observed, a replicate check is recommended to ensure the drift is within precision of conditions of the visit. Ideally steps 2 and 3 would be performed after a factory calibration by the sensor manufacturer such that the accuracy of the instrument calibration is ensured to be optimal.

22 Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor

References Cited Agrawal, Y.C., Pottsmith, H.C., Dana, D., and Mikkelsen, O.A., 2019, Super-turbidity meter—LISST-AOBS combines optical turbidity with acoustics, in Proceedings of the 38th IAHR World Congress, Panama City, Panama, September 1–6, 2019: Bellevue, Wash., Sequoia Scientific, Inc., 6 p., accessed August 2022 at https://www. sequoiasci.com/wp-content/uploads/2019/06/Agrawal_ Panama_AOBS_full_Paper.pdf. Dean, D.J., Topping, D.J., Buscombe, D.D., Groten, J.T., Ziegeweid, J.R., Fitzpatrick, F.A., Lund, J.W., and Coenen, E.N., 2022, The use of continuous sediment-transport measurements to improve sand-load estimates in a large sand-bedded river—The lower Chippewa River, Wisconsin: Earth Surface Processes and Landforms, v. 47, no. 8, p. 2006–2023. [Also available at http://doi.org/10.1002/ esp.5360.] Dietrich, W.E., 1982, Settling velocity of natural particles: Water Resources Research, v. 18, no. 6, p. 1615–1626. [Also available at https://doi.org/10.1029/ WR018i006p01615.] Giesen, E., 2015, Quality management for robust and reliable research: International Journal of Metrology and Quality Engineering, v. 6, no. 4, art. 407, 10 p., accessed October 2022 at https://doi.org/10.1051/ijmqe/2015029. Landers, M.N., Straub, T.D., Wood, M.S., and Domanski, M.M., 2016, Sediment acoustic index method for computing continuous suspended-sediment concentrations: U.S. Geological Survey Techniques and Methods, book 3, chap. C5, 63 p. [Also available at https://doi.org/ 10.3133/tm3C5.] Landers, M.N., and Sturm, T.W., 2013, Hysteresis in suspended sediment to turbidity relations due to changing particle size distributions: Water Resources Research, v. 49, no. 9, p. 5487–5500. [Also available at https://doi.org/10.1002/ wrcr.20394.] Manaster, A.E., Straub, T.D., Wood, M.S., Bell, J.M., Dombroski, D.E., and Curran, C.A., 2020, Field evaluation of the Sequoia Scientific LISST-ABS acoustic backscatter sediment sensor: U.S. Geological Survey OpenFile Report2020–1096, 26 p., accessed August 2022 at https://doi.org/10.3133/ofr20201096.

Moore, S.A., Le Coz, J., Hurther, D., and Paquier, A., 2013, Using multi-frequency acoustic attenuation to monitor grain size and concentration of suspended sediment in rivers: The Journal of the Acoustical Society of America, v. 133, no. 4, p. 1959–1970. [Also available at https://doi.org/10.1121/ 1.4792645.] O’Connell, J., Alexander, J.S., and Dutton, D.M., 2023, Data from lab experiments to support interim guidance for performing calibration checks on the Sequoia Scientific LISSTABS acoustic backscatter sensor: U.S. Geological Survey data release, https://doi.org/10.5066/P9M41G3W. Rasmussen, P.P., Gray, J.R., Glysson, G.D., and Ziegler, A.C., 2009, Guidelines and procedures for computing time-series suspended-sediment concentrations and loads from in-stream turbidity-sensor and streamflow data: U.S. Geological Survey Techniques and Methods, book 3, chap. C4, 53 p., accessed August 2022 at https://doi.org/ 10.3133/tm3C4. Sequoia Scientific, Inc. [Sequoia], 2016, LISST-ABS acoustic backscatter sensor user’s manual (ver. 1.2): Sequoia Scientific, Inc., 35 p. [Also available at https://www. sequoiasci.com/wp-content/uploads/2015/04/LISST-ABS- Users-Manual_v1_2.pdf.] Sequoia Scientific, Inc. [Sequoia], 2020, LISST-AOBS superturbidity sensor user’s manual (ver. 1.3): Sequoia Scientific, Inc., 13 p. [Also available at https://www.sequoiasci.com/ wp-content/uploads/2019/06/LISST-AOBS-Users-Manual- 1.3.pdf.] Snazelle, T.T., 2017, Laboratory evaluation of the Sequoia Scientific LISST-ABS acoustic backscatter sediment sensor: U.S. Geological Survey Open-File Report2017–1154, 21 p., accessed August 2022 at https://doi.org/10.3133/ ofr20171154. Topping, D.J., Rubin, D.M., Nelson, J.M., Kinzel, P.J., III, and Corson, I.C., 2000a, Colorado River sediment transport—2. Systematic bed-elevation and grain-size effects of sand supply limitation: Water Resources Research, v. 36, no. 2, p. 543–570. [Also available at https://doi.org/10.1029/ 1999WR900286.] Topping, D.J., Rubin, D.M., and Vierra, L.E., Jr., 2000b, Colorado River sediment transport—1. Natural sediment supply limitation and the influence of Glen Canyon Dam: Water Resources Research, v. 36, no. 2, p. 515–542. [Also available at https://doi.org/10.1029/1999WR900285.]

References Cited 23 Topping, D.J., and Wright, S.A., 2016, Long-term continuous acoustical suspended-sediment measurements in rivers— Theory, application, bias, and error: U.S. Geological Survey Professional Paper1823, 98 p. [Also available at https://doi.org/10.3133/pp1823.] U.S. Geological Survey [USGS], 2022, USGS water data for the Nation: U.S. Geological Survey National Water Information System database, accessed January 2022 at https://doi.org/10.5066/F7P55KJN. U.S. Geological Survey [USGS], 2023, USGS 09363500 Animas River near Cear Hill, NM, in USGS water data for the Nation: U.S. Geological Survey National Water Information System database, accessed November 2023 at https://doi.org/10.5066/ F7P55KJN. [Site information directly accessible at https:// waterdata.usgs.gov/nwis/inventory/?site_no=09363500& agency_cd=USGS&.]

Wagner, R.J., Boulger, R.W., Jr., Oblinger, C.J., and Smith, B.A., 2006, Guidelines and standard procedures for continuous water-quality monitors—Station operation, record computation, and data reporting: U.S. Geological Survey Techniques and Methods, book 1, chap. D3, 51 p. plus 8 attachments, accessed April 10, 2006, at https://doi.org/ 10.3133/tm1D3. Wright, S.A., Topping, D.J., Rubin, D.M., and Melis, T.S., 2010, An approach for modeling sediment budgets in supply-limited rivers: Water Resources Research, v. 46, no. 10, 18 p., accessed August 2022 at https://doi.org/10.1029/ 2009WR008600.

For more information about this publication, contact: Director, USGS Wyoming-Montana Water Science Center 3162 Bozeman Avenue Helena, MT 59601 406–457–5900 For additional information, visit: https://www.usgs.gov/centers/wy- mt-water/ Publishing support provided by the Rolla Publishing Service Center

Alexander and others—Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor—OFR 2023–1039, version 1.1

ISSN 2331-1258 (online) https://doi.org/10.3133/ofr20231039

Interim Guidance for Calibration Checks on a Submersible Acoustic Backscatter Sediment Sensor

Related documents