Echotools

Introduction

This document contains information about the data processing software provided by the echotools library (echotools-libs) python modules. The library uses the dbdreader python module to decode slocum glider files. The slocum binaries, for decoding glider files, do not provide sufficient precision when extracting information from the DBD files. The dbdreader provides full precision float values to extract the embedded echograms stored in the least significant digits of the echometrics data.

The echotools library will be able to decode three types of echometrics data. The echometrics modes are:

  • metrics: Echometrics; no embedded echogram.

  • combo: Echometrics and a low resolution embedded echogram.

  • egram: The echometrics data is replaced by a medium resolution echogram.

Installation

General python package requirements:

  • dask

  • dbdreader (* see note below)

  • future

  • matplotlib

  • mockito

  • netcdf4

  • numpy

  • pandas

  • pybuilder (* see note below)

  • pyproj

  • scipy

  • shapely

  • xarray

Glider:

  • pyserial

Other advanced and interactive features:

  • bokeh

  • cartopy

  • jupyterlab

  • nodejs

  • plotly

  • pyzmq

  • tornado

Documentation and testing:

  • pytest

  • sphinx

Future dependencies:

  • pyarrow

Warning

This is not an exhaustive list. Major and/or obscure packages are shown.

Note

(*) dbdreader and pybuilder should be installed using pip after installing packages from conda. dbdreader will also require a compiler (gcc) to be installed.

$ python -m pip install pybuilder

$ python -m pip install dbdreader

If pip install of dbdreader fails due to a compiler error, the package will need to be installed from source. See GUTILS for package link.

An older compiler will need -std=c99 supplied as a build argument gcc via the setup.py file.

Update ext_modules, add extra_compile_args = ["-std=c99"] after include_dirs. Arguments are separated by a comma.

ECHOTOOLS

The echotools library has a two step installation process. The first step is using pybuilder to build the source tree. After the source tree is built, the echotools library can be installed using pip.

NOTE: Be aware that the version number may be different than what is indicated below.

$ git clone git@github.com:phishdoc/Glider-echo-tools.git
$ cd Glider-echo-tools
$ cd echotools-libs
$ pyb
$ cd target/dest/echotools-0.3.0
$ python -m pip install .

This can be combined with GUTILS in a conda environment. Review and use the gutils/rebuildGUTILS.sh script found in the echotools-libs directory. This script builds a complete environment based on python 3.9.

GUTILS also has the following package dependencies:
  • cf-units

  • compliance-checker

  • owslib

  • pocean-core

NOTE: This is not an exhaustive list.

GUTILS

For use within the GUTILS python module for processing glider data into the IOOS Glider DAC, three items are required:

  • dbdreader python module

  • processEchogram.py python script

  • teledyne.py python class utilized by processEchogram.py

The dbdreader package is available from github.com/smerckel/dbdreader.

The GUTILS package is available from github.com/SECOORA/GUTILS.

The echotools package is kept in a private repository that resides at github.com/phishdoc/Glider-echo-tools. Access can be requested from John Horne (jhorne@uw.edu).

At some point, echotools will be released to the public allowing for easier integration of echotools processing code as a standalone module.

Documentation

Indices and tables