This page contains the year's release notes and new features, primarily for Oceans 3.0 / 2.0 / DMAS, as well as any applications created/maintained by the ONC software development group, including mobile Apps and desktop/PC applications. In progress project status may be presented as well. The information below is written in plain language as much as possible. "Major" releases include all completed items at that time (monthly), while "minor" releases are updates to the previous major release with bug fixes and specific features. Detailed release notes are available on internal confluence pages (internal users only).
[DMAS-74837] - Adjust eew selected site tab on earthquake detail page to render slightly to the right
Task
[DMAS-74750] - Add various features to the earthquake detail page
[DMAS-74821] - Add logging around the qaqc archiver to see if it has a memory leak
Implementation
[DMAS-74772] - Time series scalar JSON product with min/max resampling has extra fields (sampleTime, value, and qaqcFlags)
December 7, 2022
Major release
Maintenance
Improvements or new features:
Rebuild Data Preview Management page with react
Support multiple extension ids in a topology link
When downloading PNGs on Plotting Utility is now done on the client side. This was done to fix various download errors like out of order plots and failure to download plots with a lot of data
Color management in PU (half way)
CSV/JSON data products to support min/max and min/max+avg options
Message on Task Management to explain "select all" checkbox only select values currently displayed
Make stationcode in taskschedule on Task Management page editable
Several java file-based post-process scheduled jobs will prioritize new files and process more files on each run
Display all sensors in sensor tab on Device Detail page without paging
Sensor calibration tab is ordered by dateFrom
Many selenium tests updated
Verify and update resource permission list
Bug fixes
Internal projects
fixed potential web server crash in FileManagement
Oceans 3.0 runs on Java 17 in development environment
SeaTube
Continue on WoRMS update.
Added basic widget communication framework in the new generic dashboard app.
Continue on the integration of the Deck Log app.
Continue on Map Layer app.
Finished the back-end implementation of Cordell Bank annotation ingestion.
Continue on the front-end design/implementation of Cordell Bank annotation ingestion.
Few usability improvements
Few bug fixes.
CIOOS
(ongoing) DB/Backend changes to support saving ISO 19115 history for Community Fishers casts
(ongoing) Annotation reviews are now saved using new schema. Note: as of 2022-12-05 it's still possible to save reviews but those reviews won't show on an annotation, a minor release will correct this ASAP.
Dashboards
All widgets now provide a clear message when no data is found for the widget configuration
Add Clone widget functionality to dashboards - now can create a copy of a widget which displays on the same dashboard
ExperiMINTED
Can now add tasks to batch while the batch is in progress
Annotations
Fix bug where the Low Priority status for the Data Preview message wasn't working
EEW - Earthquake Early Warning
developed python client template that subscribes to the new MQ topics for end users
initial version of the openAPI page with all webservices is now available athttps://qa.oceannetworks.ca/OpenApi (and in actual Oceans 3.0, but this page is still in development)
Fixed corrupted Data Search view full screen button (it was blue when it shouldn't be)
Implement Cast Site Device Subset Service
Fix no data case for community fishers ISO XML dataproduct
Reduced repeated calls to GeospatialAreaService and SiteDeviceSubsetService from map
Drop siteDeviceId column from im_iso19115datasetrecord
Implement aligned and depth binned on demand as a new CSV format (equivalent to the existing .cor files)
Add maintenance notes to the ISO 19115 metadata
Increased text size on CF data visualizations
AID Viewer
Timeline dates are calculated correctly when data viewer widget is expanded
Data Products
Major feature: min/max and min/max+avg options added for time series scalar data products (CSV/JSON and MAT/ODV, done in partially in maintenance sprint)
Added 5 minute resample period to echosounder averaging, added 1 meter to range averaging for BioSonics echosounders
Fixed instrument clock drift in binary ASL products (.01a files)
Greatly reduce page size and file size of some PDF products
demean added to seismometer plots
bug fixes
In-progress: update to MATLAB R2022b in order to be compatible with an update to Java 17 and make better use of GPU processing. First step is a new logging system in both the matlab-as-a-service java bundle and in the matlab code itself.
[DMAS-74720] - Plotting Utility Download plot buttons not working on release branch in QA
[DMAS-74731] - Several bundles don't set compiler compliance correctly
[DMAS-74745] - InteractivePlotWidget public plots initially loading as private when selected from config
[DMAS-74746] - Cannot download dataproducts from second card on geospatial map
[DMAS-74747] - QA search failure for Community Fishers ISO XML
[DMAS-74749] - DeviceConsole and TopologyView diagram tabs show all extensions as "undefined" in qa
[DMAS-74763] - Add frontend check to ensure that video clip recording dates are correct before adding to a playlist
[DMAS-74779] - unable to delete a topology with extensions
User Story
[DMAS-28608] - As a data steward, I would like to enter multiple extension ids in a topology link.
[DMAS-53037] - As a software developer, I would like to run Oceans 3.0 on Java 17 in development so that I can use the latest versions of tooling
[DMAS-53250] - As a community user of cast data I would like the 'Aligned and Depth-Binned Profile Data (On-Demand)' in .csv format
[DMAS-66840] - As a Data Search user, I would like estimates of processing time and download size for searches before running them
[DMAS-72924] - Show consistent feedback on all widgets when data is missing
[DMAS-72940] - As a scalar data user, I would like CSV/JSON data products to support min/max and min/max+avg options
[DMAS-74176] - As a dev, I would like to re-write Data Preview Management page with React
[DMAS-74482] - As a software developer, I want CI builds to be explicit about what version of Java they need so that I can decouple them from the version used to run Jenkins
Task
[DMAS-56923] - verify if resource permissions list is still up to date and update as needed
[DMAS-57446] - As a data steward, I would like to be able to add additional tasks to a batch after tasks have been started to run.
[DMAS-67615] - Investigate why Jan 14th earthquake notification occured 188 seconds after the earthquake
[DMAS-72945] - Add search automation test cases for scalar DP file min/max, min/max+avg options
[DMAS-73205] - clarify confusion on datacite doi date created with 2 different occurrences on landing page
[DMAS-73400] - Add Warning Message to Tell Users that Task Management Search "select all" Checkbox Only Selects Values Currently Displayed
[DMAS-73649] - Load map layers only when they are checked, and provide better error handling if a map layer fails to load
[DMAS-73731] - Improve Device Console turn on/turn off button to clearly indicate state/action
[DMAS-73778] - As a user, I would like the IP address listing under device listing pages to be sorted by most recent datefrom by default
[DMAS-73833] - Add downloading lassoed CF annotations on the geospatial map
[DMAS-74011] - As a data steward, I would like to be able to edit taskschedule stationcode in TaskManagement UI
[DMAS-74014] - Investigate ginormous appended PDF file for Kongsberg SMB search automation test case in QA
[DMAS-74222] - Update data product options for echosounders to better support 1 meter + 5 minute averaging
[DMAS-74233] - Improvement to search size warning dialogue
[DMAS-74247] - Use Google checkstyle rules by default
[DMAS-74279] - Allow system properties to be set in init scripts without being set in a properties file
[DMAS-74365] - Update Earthquake Detail Page MMI contours to have distinct colours
[DMAS-74374] - Create new python version of the EEW client supporting MQ and all message formats
[DMAS-74439] - Update postrefresh script to disable EEW download jobs in QA
[DMAS-74444] - update audio downsampling to explicitly specify a good anti-alias filter regime
[DMAS-74468] - Add a comment to Renovate PRs about what ticket to use for time tracking
[DMAS-74492] - Implement Cast Site Device Subset Service Call
[DMAS-74512] - Add a table loading indicator for the react tables
[DMAS-74520] - Resolve auto merge conflict between 22.10.000 and master
[DMAS-74524] - If a Cassandra query times out, log the query
[DMAS-74550] - Renovate builds should fail and notify developers if they have errors
[DMAS-74585] - Add Loading indicator to the sensor tab in device details
[DMAS-74607] - Don't log every single call to ServiceUtil
[DMAS-74657] - Add resolution information to SeaTube Video information feature
[DMAS-74669] - Improve the kongsberg SMB job to prioritize new files and process more files on each run
[DMAS-74678] - As a data specialist, I would like sensor calibration tab be ordered by datefrom
[DMAS-74686] - some nrcan events arent showing up on the earthquake catalog page
Design
[DMAS-73640] - Mockup display of missing data messages and/or indicators
Implementation
[DMAS-15215] - Update CSV files for down-sampled MinMax data to include both associated QAQC flags
[DMAS-49318] - Add support for widget clone to dashboard edit frontend
[DMAS-74210] - Make Organizations page visible only to internal users
[DMAS-74452] - If EEW MQ notification isn't targeted at a specific user it doesn't work
[DMAS-74466] - Instrument Report page is not on menu but can be accessed by url for external users
November 2, 2022
Major release
Maintenance
New features or improvements
Rebuild sensor tab on device detail page with React
Add clean and raw data button on PU
Lots of new automation UI Selenium tests
Bug fixes
Organizations page
Fix issue download mp4 files for a particular device on Search Hydrophone Data page
Auto refresh labels for sensor attribute or device attribute
Improve estimate file size info on Data Search
Data Products
New data product option on spectral data MAT files added to supply users with the raw hydrophone spectral data or the data behind the spectrogram plots. Existing data product is a one-minute averaged spectra. This supports automated, machine-learning based, quality assessment of hydrophone data by the Data Analytics and Quality team.
New option on hydrophone spectrograms to limit the frequency / Y-axis range on the plot, allows users to "zoom-in" on the lower frequencies.
Compass heading added to ADCP netCDF formats
Seismometer plot filter override logic improved
EEW
Finished testing of sending notifications over ActiveMQ
Supporting MMI values in the earthquake detail map
Running 111 earthquake simulations through the system
OpenAPI
Added new response field to the scalar data service. The SensorData object now has an "outputFormat" field specifying if the data is in array or object form. This was needed so openAPI generated clients can differentiate between the two responses.
A few usability updates such as displaying video info
A few bug fixes
Internal projects
Improvements to front end CI/deployment process
Cassandra client driver update
Improved support for Java 17 in development
OSGi improvements in File Management
Cache loading performance improvements (quite noticeble for system operators/developers)
CIOOS / Dashboards / New Features
Annotations: One last change, Annotations link from spectrograms now goes to the new Annotations page
ERDDAP: Fix ERDDAP issues. Old JDBC jar being used by tomcat. Deprecated issue by upgrading ERDDAP v2.18 → v2.21 which also required Tomcat v8 → v10, Java v1.7 → v17. Fixes to address compatibility issues. Future refactors/updates should be easier.
CIOOS/OBIS: Correct WKT footprint for Dive in OBIS export
Dashboards: Fix issue with how dates are displayed in Latest Readings Widget
Community Fishers
Added user warning tositeDeviceSubset maintenance when deleting subset
Implemented ISO XML report as a separate data product for CF devices (users can now see this as a checkbox in Data Search)
Updated UI for lasso downloading, now one cast per line with associated products
Corrected UI styles on map icons
Changes main menu and map title to Community Fishers Map
Fixed broken cast deletion
Implemented station searching on Geospatial Map
Fixed missing first zoomed in station on first load
AID Viewer
Updated ADCP Matlab code to allow for selecting image size
Fixed failing request errors with missing data product options on ADCP data
Fixed null pointer in Annotation Service
Fixed scrolling and timeline issues on Spectrogram widget
Instruments
PVCS View page updated - No driver change was required, This is a change to the PVCS (ferry pump and control system) view screen only.
MaximumFrequencyLogger – more enhancements to memory usage (this is a part of the device driver data acquisition infrastructure). Partially fixed in September, remainder of fix done this month.
Multi queue fixes and enhancements: fixed configuration issue related to Community Fishers use in multi queue. Added Activity column to multi queue screen so users can see activity on each queue.
Bug [DMAS-72984] - Attribute Labels restricted to previously used labels [DMAS-73896] - I cannot download the mp4 files ( test data) from SubC Imaging 1Cam Mk5 SN 15548 (23941) [DMAS-73897] - Unexpected AdException caused raw file generation to fail in dev [DMAS-73899] - Date fields on the new File Management page are too narrow [DMAS-73915] - The archiver reloads the "Layout" cache a lot [DMAS-73955] - Correct dataviewer widget scrolling & initialization to work for varying screen resolutions [DMAS-73963] - Cannot delete casts that were created without a cast definition [DMAS-73997] - Search result service will not package all files and will return corrupted file if name collision occurs [DMAS-74006] - Fixes to Dashboard Latest Readings display [DMAS-74075] - New Oceans Next branches fail to build with peer dependency conflicts [DMAS-74078] - Correct the timeline calculation for the dataviewer widget [DMAS-74125] - Data Availability Plot is Broken in QA [DMAS-74131] - Dataviewer widget timeline jumps when fetching new images [DMAS-74140] - Data Search status field clips info [DMAS-74146] - qadivelogger2 fails to start Data Product Preview Search [DMAS-74156] - Geospatial polygon associated with a community fishers station does not render on first load [DMAS-74158] - Site cache loads very slowly on shorestations [DMAS-74174] - Get number of Matlab instances to run on task machine from TaskServiceManger taskTypeByRole as well as taskTypeByStation [DMAS-74185] - Cassandra read exception in QA: "Query timed out after PT2S" [DMAS-74187] - Renovate fails to automerge updates when configured to do so [DMAS-74189] - Errors starting correlator jvms [DMAS-74194] - refresh DeviceListing page after label is saved [DMAS-74206] - Cassandra can't connect to cql-hosts [DMAS-74208] - Device ID link in Device Workflow page is not linking to the workflow tab [DMAS-74216] - Reprocessed ScalarData Not being Ingested into Cassandra [DMAS-74219] - Annotation List Widget not displaying information [DMAS-74220] - Unable to add site device [DMAS-74227] - Build failure in Oceans 3 master [DMAS-74229] - Fix null pointer in Annotation Service [DMAS-74231] - Another build failure in Oceans 3 master [DMAS-74234] - Dataviewer produces failed request errors with missing dpos on ADCP data [DMAS-74249] - RDI postprocess batchable task completes but does not regenerate any .rdi files [DMAS-74258] - Sensor Tab in does not display sensors in QA [DMAS-74269] - SiteDeviceSubsetMaintenance Page not Showing overlapping subsets on Save [DMAS-74277] - File cleanup task not functioning correctly [DMAS-74280] - LastReadingJSON service is returning time of first reading on last day of sampling [DMAS-74292] - Fix typos in new page titles/menu entries User Story [DMAS-33555] - We want to move filemanagement to an OSGi module [DMAS-69101] - As a deployment engineer, I would like Oceans Next deployment builds to be independent of its CI builds so that deployments can't be blocked by CI performance issues and to improve security [DMAS-69680] - As a software developer, I want dependencies of Oceans Next to be kept up to date systematically, so that breaking changes aren't discovered at inconvenient times [DMAS-73703] - As an user of the Geospatial Map, I would like to be able to search by station name [DMAS-74114] - As a System, I would like to remove VPSView and BPSView pages Task [DMAS-65392] - Sync bathymetries to divelogger machine [DMAS-65416] - Support to DAQ Team hydrophone machine learning project [DMAS-66844] - Restore ONC video info (file name, deviceId ) in the JWPlayer about the video [DMAS-72995] - Upgrade Matlab spectrogram code to produce requested image size [DMAS-73536] - Improve taxonomy-related tables' constraints [DMAS-73978] - Modify Search and Lasso Icons' onHover text to match other icons on the Geospatial Map [DMAS-74003] - Improper use of MaximumFrequencyLogger [DMAS-74016] - Create map_layer table [DMAS-74069] - NetCDF Data Products for RDI ADCP are missing the compassHeading variable [DMAS-74090] - Log a warning if we try to load a cache that has a dependency that's disabled on the current role [DMAS-74124] - Make it easier to install Oceans Next in Oceans 3 as an OSGi bundle in dev [DMAS-74126] - Upgrade dotenv-expand to v9 [DMAS-74135] - Change Geo Map tile name on O3 Landing page [DMAS-74137] - Change dropdown menu from Geospatial Map to Community Fishers Map [DMAS-74145] - Create MapLayerState and backend for may_layer table [DMAS-74148] - Multiqueue Fix for Community Fishers [DMAS-74157] - Add serializability testing for Bathymetry objects [DMAS-74159] - Update @types/node less often [DMAS-74181] - Remove getTaskTypesByRole and getTaskTypesByRoleAndStation methods from TaskServiceManager [DMAS-74204] - Add MMI Contours to the earthquake catalog page [DMAS-74205] - Deleting a cast from site device subset maintenance page should ask for confirmation Requirement [DMAS-73918] - Write requirements for searching on geospatial map Design [DMAS-69103] - Update Oceans Next deployment design [DMAS-73840] - Design how to search for Stations on Geospatial Map Implementation [DMAS-67129] - Split up the Oceans Next Jenkinsfile into separate test & deployment files [DMAS-69684] - Set the reviewers of Renovate PRs to random developers (department-wide) [DMAS-72904] - Deprecate all old annotations search page references [DMAS-73311] - Upgrade Cassandra driver to 4.14 [DMAS-73678] - Separate API and implementation of already-extracted parts of filemanagement [DMAS-73852] - Database changes for WoRMS taxon update application [DMAS-73886] - Implementation for New PVCSView Page with React [DMAS-73895] - Implement one cast per line with associated products in lasso download dialog [DMAS-73920] - Implement Searching on Geospatial Map [DMAS-73991] - Add Date Filter to that Table filters [DMAS-74000] - Improve the performance of loading the DeviceServiceSearchTreeNode cache [DMAS-74004] - add clean and raw data buttons to plots [DMAS-74020] - Add a option to limit or set the y-axis/frequency axis upper bounds in hydrophone spectrograms [DMAS-74076] - Add taxonomyAttributeLineId to the return of seatubeV3/annotations service [DMAS-74083] - Replace misc javax modules with their jakarta equivalents [DMAS-74091] - Implement ISO XML report as dataproduct - java [DMAS-74094] - Create new dataproduct for XML report - SQL [DMAS-74105] - Back end changes to accommodate database changes [DMAS-74106] - Add remaining discovery services to open api definition [DMAS-74113] - SQL to add image size option to ADCP plots [DMAS-74123] - Add the scalar data api to the open api definition [DMAS-74141] - Matlab code to produce smaller ADCP current plots when requested [DMAS-74171] - Update the device listing sensor tab to react [DMAS-74191] - Update Organizations Table Headers [DMAS-74192] - Remove Default Citation from organizations database and related dmas code [DMAS-74221] - Update WoRMS node types (taxonomic ranks) [DMAS-74225] - Activate XML data product on geospatial map [DMAS-74237] - Shift EEW MQ grid message 3 degrees east Test [DMAS-74120] - Add search automation test cases for new spectrogram frequency limit option [DMAS-74186] - Create search automation tests for Community Fishers ISO XML dataproduct Bug (ST) [DMAS-73281] - Jetty task crashes when attempting to execute Matlab command [DMAS-73301] - Reflection error using jackson for XML with java.time [DMAS-74059] - Should the space at the beginning of the text be considered when sorting? [DMAS-74129] - Jenkins fails to deploy Oceans Next to dashboard.onc.uvic.ca [DMAS-74160] - Add HP filter override for unfiltered seismometer data in data preview [DMAS-74173] - Mockito-based tests fail with InaccessibleObjectExceptions [DMAS-74195] - New spectral resolution comment in plot is wrong [DMAS-74198] - Correct point order in dive footprint [DMAS-74246] - Spectrograms with low frequency limits look too chunky Documentation [DMAS-74033] - update the documentation for this radio buttons features of plotting utility page [DMAS-74121] - Update external documentation for new spectrogram frequency limit option [DMAS-74142] - Document new dataproduct option for image size on ADCP current plots [DMAS-74166] - Document new dataproduct in public wiki
October 12, 2022
Minor release: fixes for system console, BioSonics data products, CSV memory size issue, Organization UI, Data Search file size estimates.
code complete for search by station name ongeospatialmap, tagged for 22.10.00 (next major release)
can nowdownload ISO XML reports from geospatial map
fixed geospatial map and Fix Search Tree Maintenance tests
now support turbidity piggyback device connected to AML6 in pipeline
correctedISO 19115 XML reports for siteDeviceSubsets with restricted data
change Geospatial Map Layer Names
added downloading CF annotations on the geospatial map
fixed double clicking save comment on Sitedevicesubset maintenance page
refactored CastPopup of geospatial map to pull generateListComponent into its own component
changed Geospatial Map page name to 'Community Fisher's Geospatial Map
fixed search result service so no name collision occurs
CF app pop-up notifications now colour coded
Acoustic Instruments Data Viewer
upgraded AID viewer to have resizable widget images
modified the data viewer widget to calculate and align images with startDate
Maintenance and Automated Testing
Improvements or new features:
rewrite Network Console > Organisations with REACT
new REACT page for Device Category metadata
label for sensor attributes
junction box port state
deviceid on hydrophone data page link to device detail page
data search warning message for large or long running searches
deep link for each tab on device detail page
update qaqc_priority_trigger
Fix or answer several prod dev tickets
update test cases and selenium test scripts
Data Policy
support first version of Licence Management
support mapping of a Licence to a Data Agreement
Sea Tube
continue on WoRMS update
continue on Deck Log implementation
continue on Map Layer improvements
continue on UI tests implementation
improve the annotation list highlight behaviour
add attribute group info for annotation query to support advance attribute search
fix the bug for not switch video resolution immediately
fix the bug for converting attribute button to taxon button in the Quick Button Set Config app
other minor bug fixes
Data Products
Bug fixes: sound metrics indexing, State of Ocean/Env/Climate plots not finding some deployments.
Minor changes: add warning when averaging mobile/spatial data, hydrophone plot title replace "wav" with "audio" for source type, plot comment limit by line number.
Instruments
fix memory leak on logger, fix telnet console issue with RBR BPRs
[DMAS-74021] - There is no annotation showing in any dive in QA after DB refresh
[DMAS-74042] - Since 09 Sept 2022, hydrophone log files are being archived without dataproductformatid
User Story
[DMAS-31260] - As a user of the SearchHydrophone tool, I would like to have a link to Device Details page from Search Hydrophone Data page to the current device being displayed
[DMAS-73340] - Allow download of iso xml report for SiteDeviceSubset from geospatial map
[DMAS-73459] - As a Dev, I would like to Update the Network Console's Organizations to be its own React Page
[DMAS-73682] - As a data steward, I would like to see a label feature for sensor attributes like for device attributes
[DMAS-73792] - As a dev, I would like to see some improvements or bug fixes on Device Category page
Task
[DMAS-72996] - Upgrade AIDviewer to have resizable images in a widget
[DMAS-73067] - Clean up calculateExpectedSamples code
[DMAS-73529] - Test Community Fisher post processing pipeline with additional turbidity device
[DMAS-73586] - Implement keyboard entry into TabbedButtonSet
[DMAS-73614] - Add warnings about averaging mobile data to time series scalar MATLAB data products
[DMAS-73667] - Modify the data viewer widget to calculate and align images to the startDate
[DMAS-73764] - Investigate the 2022-09-05 missed detection
[DMAS-73779] - Add downloading CF annotations on the geospatial map
[DMAS-73804] - Links to Device Tabs not working as intended
[DMAS-73816] - Refactor CastPopup of geospatial map to pull generateListComponent into its own component
[DMAS-73841] - Add group information to the attribute fields of the seatubeV3/annotations service call
[DMAS-73857] - Adjust spacing on SPD plot comment for new line-limiting scheme
[DMAS-73853] - FIle postprocessors fail if the source directory contains unexpected files
Task
[DMAS-72998] - Add database check constraints to taxonbuttonsetline table
[DMAS-73774] - Add category for new turbidity piggyback device to list of possible community fishers device category's
[DMAS-73775] - Consolidate definitions for possible community fishers device category's to one location
[DMAS-73798] - As a dev, I would like to see some improvements about pop-up warning when data would be better accessed through the API
[DMAS-73808] - Don't pass empty strings as comments to TaxonButtonSetService
Implementation
[DMAS-73746] - Update service call to return DMASUsername and Id instead of just Id
[DMAS-73796] - UI display for dmasuser name and id
[DMAS-73832] - Turbidity data from piggyback device is written into data file for CTD
September 8, 2022
Major release
CIOOS
Upgraded CKAN production version and updated documentation
Dashboards
Widgets now display a descriptive message when required configuration details are missing (part 1 of error handling, part 2 deals with missing data)
Data Policy
Ongoing, work has progressed on page for creating licences and licence management
SeaTube
continue on WoRMS update
continue on Deck Log implementation
continue on Map Layer improvements
continue on UI tests implementation
bring the search field values to advance search
don't allow anonymous users to create playlist
click edit button will not set video seek time (fix)
add highlight for annotation editing
improve live/historical mode for SeaTube
add SeaTube deep link for SeaTube exports
other bug fixes
Internal Projects
Bugfix: production crashes from reading junction box data with /api/scalardata
Bugfix: possible production crashes from aborting requests to /api/dataProductDelivery
Improvements to web service logging
ActiveMQ client upgraded to 5.16.5
Cleanup of code dealing with ArchiveFile filename parsing
Cleanup of Cassandra scalardata/rawdata archiving code
Maintenance Sprint
Data Search
Internal email notification for when a data search is cancelled or killed
Pop-up warning to suggest user to use API
Remind user that he/she does not have to have browser open for searches to complete
User receive an email even if a search fails
New feature or improvement
New Device Category page written with React
Filter to display list of active events on event detection page
Show maximum number of characters and the count in a text field if there exists a maximum
Re-organized fields in user profile
Add confirmation dialog to geospatial map if close selected while still drawing
Bug fixes
wiki link for Digital Fishers on the landing page
Dataset Landing page error when QueryPIDs have no DOIs
JunctionBoxalarmSetting page if one JB have port 11 or 12
Instruments with a hyphen in the name are not added correctly in user management page
Click "search data product" buttons causes screen to go black in search map when fullscreen
EEW
In-progress: simulation work, new message formats
API
OpenAPI project initiation
Data Products
BioSonics Echosounder data products: in addition to the existing DT4 manufacturer's binary format and hourly raw MAT files, added calibrated and resampled MAT files, plotting (PNG and PDF formats) on daily and multi-day bases, daily plots for Data Preview, a CSV format for EchoView import, plus documentation
Seismometer data product improvements: use the "All" option in Data Preview so all channels with be plotted, plot title improvement, improve error handling
Minor updates for documentation, links, Sound Metrics speed improvement, final search result update, one bug fix, add turbidity device to CF products
Community Fishers
Restrict date range to 1 hour when creating a manual cast
Download XML reports from the map
Added the ability to show CF cast history to search tree maintenance
Removed Cruise Information popup on Community Fisher App
Improved response time for getAll siteDeviceSubset with expanded option
Stopped App from sending cast data (3 sets of files) multiple times on poor connections
Instruments
Updated Didson postprocessor to produce DDF files with more accurate file/frame headers, including device attribute values
Driver update for ONC lights and other driver maintenance
[DMAS-73527] - Fix Popup for Deleting Subsets when Editing Dates
Task
[DMAS-69513] - update job 177-seismometer-SOH-batch-ftp-job to take in new seismometer file names produced by John D's driver
Implementation
[DMAS-73396] - Copy existing parserdefinition rows without current transaction records into parserdefinition_trx
[DMAS-73484] - script for a job to move DMAS files to Archive folder
August 10, 2022
Major release
Instruments
Didson driver fixes, improvements:
Set parameters command has been fixed and will now set all parameters (went out in earlier minor release).
A new acquisition mode 'burst mode' has been added (capable for higher frame rates) and some changes to driver behaviour were implemented.
Driver now sets the device's time on driver start. No other commands are sent on driver start.
When under driver control (not using seascript), the driver will send the following commands before polling: Set Time, Set Parameters using attribute settings
New command burstModeCapture. This command is the same as startCapture except retrieves data using burst mode.
When polling under seascript control, the driver will send the set time command whenever executing the startCapture or burstModeCapture command.
During testing the focus setting would sometimes drift during the acquisition cycle. The driver now checks the focus setting and will adjust if necessary once every 5 seconds.
Rayfin and camera control fixes/improvements (went out in earlier minor release).
Community Fishers
Added MATLAB memory usage logging
Added check to warn of a possible cast deletion when creating a new new site device subset
Fixed missing aligned and depth binned cast data for T'Sou-ke on June 20 2022
Created separate directories for the scheduled and manual pipeline jobs
Upgrade creating overlapping subsets alert to not allow the creation
Fixed Data Search to download Tablet Annotations
Added ability to view annotations for a station's most recent cast on the GeospatialMap
Handle overlapping empty castdata files in ExternallyDerivedSensorDataIngester
Updated toolkit and protocols into Android app
Upgraded sitedevicesubset maintenance page with better workflow, can no longer enter a siteid for a sitedeviceid
Created and tested App version CF.3.2.7
Added 'MANUAL' OR 'MEASURED' GPS flag to App raw data upload
Added APP feature to view the latitude and longitude data from the tablet after data has been collected with the CTD
Changed App Aanderaa oxygen units from µm/L to µmol/L
Corrected validation for 3D WKT on SiteDeviceSubsetMaintenace page
Modified the MethodToCreateCast to use deviceId and Date Range to get siteDeviceId
Removed im_sitedevicesubsetcastdefinitionid_fkey on im_SiteDeviceSubsetCastDefinitionTransaction table so SiteDeviceSubsetCastDefinitions can be deleted
Added check to see siteDeviceSubsetId has siteDeviceSubsetCastDefinition in MethodToDeleteCast
Set default site device subset type to "downcast" instead of upcast on SDS maintenance page
Prod-error: deleted SiteDeviceSubsets with 10 year date range
Data Products
Update for seismometer data products:
Improve option sets and code to support accelerometer channels, add an all velocity option
Improved handling of filter options and overrides, improved plot title and comments in that case
Fix display of Maris accelerometer data in Data Preview
Fix scaling issues, add a new scaling parameter for Maris "C" channel
Improve feedback to user when selected channel option is not available
Documentation updates for new Data Search time/size estimates, new annotation UI, all data product options now have a link to their documentation
Updated time/date formats for internal services
Prepare for BioSonics data products: add to Data Preview, configure option sets and documentation
Bug fixes for: bad RDI data, ferry and profile product edge cases, old hydrophone data
Maintenance Sprint
File Management page refactored to REACT standard with a new ingestion ready feature
Remove Budget module code
Improvements: autocomplete deviceid for Device Sync page; not allow future date for datefrom in deviceattributeservice
Fix bugs:
PU: downsampling issue, download button, no datarating situation,
data search: timeout stuck and response queue for cancelled search, help icon for send email, data availability for all available option,
Device Details: data error in sensor additional attribute tab
API: RawData service return null when should return error message
Digital Fishers: "about us" link
System log: flexjson.JSONException: Error trying to deepSerialize
Camera control page
Date agreement tab on Network Console
Selenium tests
EEW
Fix auto refresh issues for eew dashboard
Fixed convergence check in Kalman filter
Full earthquake simulations working and mostly automated
[DMAS-73037] - QA web server failing to find missing resource "ResourceManager : unable to find resource 'ca/uvic/dmas/core/system/serviceframework/externalapi/proxy/ApiProxy.vm' in any resource loader."
[DMAS-73094] - Rawdata service returning null when it should be returning an error message
[DMAS-73097] - Oceans Next CI builds fail with "@types/lodash not accessible from @storybook/mdx1-csf"
[DMAS-73135] - RawData service not properly validating date parameters
[DMAS-73157] - Data preview not available for Maris
[DMAS-73222] - As a annotation creator a negative sign is now appearing at time-stamp
[DMAS-73244] - Search automation user can't rerun searches
[DMAS-73255] - ERDDAP Currents Records don't seem to resolve when added to erddap table
[DMAS-73266] - SiteDeviceSubsetMaintenace Page does not validate 3D WKT correctly
[DMAS-73275] - Some searches aren't being queued after 22.06.002
[DMAS-69106] - Rayfin driver bugs related to takePicture
[DMAS-72975] - Prevent bad RDI ADCP SoG East data from getting into the RDI files
[DMAS-72990] - Data Search shows data availability when there is none
[DMAS-73014] - QA stops responding at the same time every night
[DMAS-73040] - Missing aligned and depth binned cast data for T'Sou-ke on June 20 2022
[DMAS-73137] - Fix mmi:MD_MaintenanceInformation not expected in ISO 19115 XML
[DMAS-73150] - The data availability bar is not showing the data availability correct for both DDL Sonars- search by location ( see ticket body for details)
[DMAS-73152] - Camera Control Page Commands failing
[DMAS-73171] - New data products not appearing under Attributions in Data Agreements in Network Console
[DMAS-73233] - Default attributes not populating when taxon buttons used
July 12, 2022
Minor release: fix excess logging and ISO 19115 extra field (needed to restart web server for a start up issue anyway)
Infrastructure Management: performance improvement for data agreement on Network Console; fix spelling typo on device detail page; improvement on device attribute upload feature; AdFile return message for no file case; Remove Budget module (half way, still need keep some features); re-write File management page (half way)
Plotting Utility: fix plot title, Support Sensor level data ratings (also applied for all web services); save/download all plots in one file; show/hide plot legends for all plots at once; data gap when no exist raw data;
UI Test Automation: fix failing tests and add new test cases
CF
Geospatial Map now places multiple downloads lasso selection into one search results folder
Added a manual reprocessed flag (M) to Site Device Subset Maintenance and to Search Tree Maintenance
Built tests for CF date.From and Date.To Task Job parameters
Removed "live" reviewed check boxes from Search Tree Maintenance, replaced with flag (R)
No longer render Geospatial Area Field in Form for casts in Site Device Subset Maintenance (data model update)
Updated stale Contact Us link on Android App
SeaTube
In-progress: Bounding Box with animation
In-progress: WORMS taxonomy update
In-progress: configurable annotation table demo for deck log
In-progress: add ONC internal deck log page to display the annotations with filters (supports both AND/OR logic)
UI testing improvements
Bug fixes and small usability improvements
Internal projects
Java 17 upgrade (in progress)
Simplify date parsing/formatting code
Automatic dependency upgrades (in progress)
Annotation UI Updates
Annotation V2 fully integrated, this includes using Annotations V2 in the following:
More menu
Hydrophone Search
Plotting Utility
Data Search
Device Listing
and more
Tweaked Annotation Entry display so it scrolls if content doesn't fit vertically
CIOOS
National updated so deleted CKAN sets will be removed after deleted in RAs
Translation upload detects character encoding preventing incorrect characters from being saved
ISO 19115 now include RORs for organizations
CKAN changes
Correct ISO 19115 XML format
Temporarilyremove spaces from EOV keywords
Upload job now saves ISO 19115 files instead of using the CKAN API
Replace use of datasetIdString with siteDeviceId in job
Use siteDeviceId instead of datasetIdString in im_iso19115datasetrecord
ERDDAP
Fix bug where ERDDAP titles had the incorrect format
OBIS - in progress
meta.xml now supported
EEW
Removed tau algorithm from magnitude calculation
Support sensor level data ratings (now the mmi sensor will show up as clean data with a sensor level data rating of 10 seconds)
[DMAS-69886] - Search Tree Maintenance Cast Reviewed check boxes should not be live
[DMAS-69899] - Master build is failing due to compile error in TaxonomyAttributeServiceTest.java
[DMAS-69905] - DMAS Search Failed Notification: 08-Jun-2022 15:10:06 UTC, Strait of Georgia East-Strait of Georgia East VENUS Instrument Platform, RDI ADCP Time Series, SearchHdrId 12082529, UserId 4377
[DMAS-69906] - Plotting Utility shows all plots as being downsampled for any time range
[DMAS-69937] - Please investigate "Data Error" in Sensor Additional Attributes tab after adding a new attribute to table in QAQC Auto Test Detail Page
[DMAS-69960] - Widget resize changes on Seatube Dashboard
[DMAS-69961] - Unable to vertically shrink video widget in videocentric layout
[DMAS-69963] - Camera not setting in ExpeditionManagment edit dive
[DMAS-69968] - CameraControl SeaScripts are failing
[DMAS-70000] - Plotting Utility Plots have minor display issues
[DMAS-70009] - Earthquake Catalog page associated two usgs events to the same earthquake
[DMAS-70015] - Creating Attribute Button without Attribute causes error
[DMAS-72903] - A few bugs related Taxon Button Set
[DMAS-72909] - Min/Max dates aren't set correctly in the Dive Form's date pickers when a date hasn't been selected yet
[DMAS-72919] - pgd and pd sensors always fail to derive in prod
[DMAS-72923] - Running datacentre in Dev fails with BundleException: missing package ca.uvic.dmas.core.infrastructure.dwc.text.domain.manager
[DMAS-72982] - Generating a Plotting Utility image with legend hidden causes overlap
[DMAS-72991] - Deleting an empty attribute deletes the button
[DMAS-73002] - Data Search Data Availability is Never Available in QA
[DMAS-73006] - Date Time Picker Field Validation broken in Form
[DMAS-73008] - Unable to save annotations in SeaTubeV3
[DMAS-73010] - DMAS Search Failed Notification: 29-Jun-2022 22:50:29 UTC, AML CTD Plus X 50328, Aligned and Depth-Binned Profile Data (On-Demand), SearchHdrId 12293071, UserId 112720
[DMAS-73015] - Unable to clear taxonomy field in seatube dashboard filter
[DMAS-73017] - Deleting a taxon button set button doesn't fully remove it from the UI
[DMAS-73028] - Dataset titles are showing up incorrectly on ERDDAP
[DMAS-73069] - Fix concatenatedTitle included in maintenance diff
[DMAS-73073] - CSV/json data products have bad date strings in QA
[DMAS-73083] - Exempt user list is still adding to the response queue
[DMAS-73091] - Plotting Utility download button does not work in QA
[DMAS-73100] - Cancelled tasks create messages in response queue which are never read
[DMAS-73108] - Taxon Button Set list is overlapping its search bar
[DMAS-73134] - Messages can be Improperly Sent to Response Queue on Startup
User Story
[DMAS-57948] - create back-end infrastructure to support ability to track changes in the ISO 19115 metadata record
[DMAS-65501] - As a sys operator, limit the number of searches running per user
[DMAS-66934] - Export dive and cruise information to event.csv
[DMAS-67070] - Add taxon information associated with dive WoRMS annotations to occurrence.csv
[DMAS-67181] - Only export annotations with enough up-votes to ensure only valid annotations are submitted to OBIS
[DMAS-67249] - As a data steward, I'd like to upload translation files without needing to check if the file is saved with UTF-8 or ISO 8859 or another encoding, so replacement characters (eg �) don't appear in translations
[DMAS-69034] - As a dev, I would like to remove deprecated columns in dmasuser and dmasuser_hist tables
[DMAS-69447] - As a Data Steward, I would like to see improvement on Network Console Performance - Adding/Editing Data Agreements takes years
[DMAS-69678] - As a data specialist, it would be great to have the UI display if a cast had its ref lat and long manually set
[DMAS-69835] - As a hydrophone data steward, I need improvements to the sensitivity vector device attributes upload feature
[DMAS-69864] - As a Plotting Utility user it would be a time-saver to have a button that saves/downloads all current plots into one file
[DMAS-73012] - DFO would like to change 'Dive Chief' to 'Dive Lead'
Task
[DMAS-53943] - Build tests for CF date.From and Date.To Task Job parameters
[DMAS-57371] - Add/update formsections to support Maris accelerometer / seismometers
[DMAS-57387] - Improve seismometer data products to support channels that are 1/2/3 instead of Z/E/N
[DMAS-69327] - Have all Annotation Entry options enabled by default (Quick Entry, Capture Time, & Attributes)
[DMAS-69747] - Implement units tag on AllowAny data product options
[DMAS-69776] - Implement using ckanext-spatial (cioos/siooc fork) and continuing to use API to upload CKAN datasets
[DMAS-69834] - Support Sensor level data ratings when getting scalar data
[DMAS-69840] - Dont use the Tau algorithm for determining earthquake magnitude
[DMAS-69860] - Restore Annotation V2 changes that were reverted
[DMAS-69870] - Delete orphaned classes DataSearchJSON and JSONHelper
[DMAS-69885] - Make parserdefinitionname column consistent
[DMAS-69893] - As a community fishers data user, I would prefer that downloaded COR/PNG files from geospatial map result in one folder for all files instead of individual folders for each file.
[DMAS-69950] - Fix formatting issues in ISO 19115 XML
[DMAS-69959] - Add skipSubfolders parameter to SearchResultsService in Geospatial Map laasso tool
[DMAS-69969] - Provide infrastructure to restrict allow any dataproduct options to integer values if desired
[DMAS-69992] - Do not render Geospatial Area Field in Form for casts in SiteDeviceSubset Maintenance
[DMAS-70014] - Drop support for deploying to /usr/local/gemini_driver
[DMAS-72897] - Don't save null attributes to taxon button set Attribute buttons
[DMAS-72905] - Add primary sensors for Cambridge bay community page
[DMAS-72910] - Dataset Manager and related code often uses ERDDAP dataset type, but should only be dependent on the site device id
[DMAS-72922] - As a Data Steward, I would like a deviceattribute_trx table to capture changes to device attributes
[DMAS-72929] - update acceleration download task to write files to sub directories for each seachtreenode code and date
[DMAS-72935] - Don't save an empty button into database right away after clicking "ADD Button"
[DMAS-72946] - Add MTC test site topology setup to qa post refresh script
[DMAS-72951] - Create scheduled job for continually downloading acceleration data from some EEW land sites
[DMAS-72954] - Minor improvement to handle bad hydrophone calibrations
[DMAS-72959] - As a Dev, I would like to add a warning to AdFile Service when a file does not exist
[DMAS-72994] - Set documentation link on seismometer channel options
[DMAS-73019] - Add confirmation dialogs for deleting buttons and attributes on Taxon Button Set Config page
Bug [DMAS-42666] - The "link" link on the start page of PU is broken [DMAS-52110] - Plotting Utility not displaying correct plot title [DMAS-64433] - Required permissions for adding and saving dives are different [DMAS-67410] - Call to ScalarDataAPIService linked to a production crash [DMAS-67738] - editing a batch task to remove the parameter value doesnt remove the key [DMAS-69125] - Oceans Next builds are timing out in multiple branches [DMAS-69174] - Editing an Attribute's Data Type Breaks Annotation List Edit Functionality and Crashes STv3 Page [DMAS-69495] - updating playlist description results in 'playlist already with name "<name>" error. [DMAS-69566] - Frontend changes for preventing attribute datatype changes if attribute is in use [DMAS-69616] - Banner message is not shown up on landingpage [DMAS-69641] - As a system operator, I want to prevent calls to DataSearchScalarDataService from crashing the web server [DMAS-69666] - Missing space in file archiver warning message [DMAS-69672] - Casts correctly being attributed to station in COR files, but the Sitedevicesubset name still has the generic "CAST" prefix [DMAS-69696] - Fix oceans next loading on search tree maintenance [DMAS-69709] - QAQC Create page has incorrect fields [DMAS-69713] - Seatube - taxonomy required to add attribute - behavior not consistent [DMAS-69732] - In dive detail widget the Active flag is not checked even it is in the edit mode [DMAS-69751] - Save and Cancel button showing in Taxon Button Set Config for RO users [DMAS-69768] - DMAS Search Failed Notification: 23-May-2022 20:44:23 UTC, Clayoquot Slope-ODP 1364A, Time Series Scalar Plot, SearchHdrId 12036575, UserId 19343 [DMAS-69771] - All CSV data products showing $DateUtil.dateRangeToScienceFormat($citation.startDate $citation.endDate) instead of a date in header in QA [DMAS-69772] - Earthquake Catalog page shows some detections as unconfirmed when they should be confirmed [DMAS-69773] - Fix bug in metadata information causing object loading to be difficult when there are null values [DMAS-69791] - I can't update some field in Dive Log using the popup in Expedition Management page [DMAS-69793] - DMAS Search Failed Notification: 26-May-2022 04:14:07 UTC, ROV Data-Odysseus, Time Series Scalar Data, SearchHdrId 12056870, UserId 48660 [DMAS-69813] - Device ID does not automatically fill in when editing on Driver Schedule Page [DMAS-69823] - scheduled job 378-DOI-Registration-job-deviceTypes has failed the last 3 days [DMAS-69829] - Fix issue where bndrun files were generated incorrectly [DMAS-69843] - Can't save multiple attribute in a button during one operation User Story [DMAS-54177] - As a passive acoustic scientist interested in low frequency signals, I would like downsampled 2 kHz versions of high sampling rate data to be available [DMAS-67032] - As a data specialist, I would like changes to the Sensor QAQC Attribute UI that allow us to modify historical values and datefroms [DMAS-67182] - As a biological expert reviewer I want to vote on biological annotations to give my opinion [DMAS-67999] - As a Didson SONAR user, I would like value-added data products to visualize and analyze the data [DMAS-69243] - As a Scientist, I'd like a pH property filter in DataSearch [DMAS-69367] - Modify the TelnetConsole page to not require device permissions [DMAS-69670] - As a Device Console user I would like to see a field that displays the latest driver start time-stamp [DMAS-69781] - Add searching ability for pages using Device ID links [DMAS-69784] - As a dev, I would like to see traceability for the configproperty table Task [DMAS-58198] - Document services used in existing Annotation Search [DMAS-58203] - Define high level component structure [DMAS-67641] - Update external documentation for Oceans 3 - data products [DMAS-68007] - Enable parsing of zoom values for Axis cameras, devicetype 158 [DMAS-69331] - It's impossible to clear the Taxonomy field in Annotation Entry without clearing all fields [DMAS-69485] - Don't embed unnecessary data in the DeviceListing velocity template [DMAS-69510] - Investigate 2022-04-23 earthquake [DMAS-69535] - Investigate new PDF append function for MATLAB data products [DMAS-69586] - Fix new checkstyle warnings from updating to 9.3.0 [DMAS-69594] - Update external documentation (DP and HydroCal) to fix links broken by the new ONC website [DMAS-69601] - As a scientist using SeaTubePro, I would like to see a closed "Cruise"-tree [DMAS-69615] - Remove the old TaxonButtonSetConfig servlet [DMAS-69642] - Support deploying storybook for branches with new dependencies [DMAS-69668] - Update data policy URLs [DMAS-69669] - Update configproperty data policy url reference [DMAS-69673] - Update siteDeviceSubset reference fields when re-post-processing a cast with scalar lat long [DMAS-69694] - Remove character limit from expedition table fields [DMAS-69728] - clarify 'dateTo' or empty sensorLoad black functionality in Watchman declarative parser to prevent data getting assigned to wrong devices parsing of [DMAS-69730] - Don't require unit test coverage on typescript files in the storybook [DMAS-69734] - Improve test coverage of TestHistoryTable.jsx [DMAS-69737] - Improve test coverage in TaxonButtonConfig [DMAS-69743] - Increase Test Coverage of AvailabilityChart.jsx [DMAS-69846] - Revert changes to AnnotationsV2 as not complete (was: Clicking "Query Annotations" in Device Details opens AnnotationsV2 in *two* new tabs) [DMAS-69862] - As a Data Steward, I would like a nameplate_trx table to capture changes to nameplate metadata Requirement [DMAS-69549] - Document requirements for the video preprocessor job Design [DMAS-69550] - Document video preprocessor job design Implementation [DMAS-67402] - Create Simulator for Valeport Current Meter [DMAS-69177] - Don't allow unit tests to make API calls [DMAS-69226] - Implement react code for Attribute table [DMAS-69245] - Add "(how to do this)" link to instructions on Plotting Utility splash screen [DMAS-69267] - Use a file preprocess job to set exact start/end timestamps in video files [DMAS-69403] - Create a matlab reader for the Didson sonar .ddf files [DMAS-69409] - Add jaxb generated EML classes [DMAS-69416] - Add raw icon for DIDSON DDF files [DMAS-69458] - Update AnnotationsManager to support V2 querying [DMAS-69476] - Implement Java service to update/delete attribute values [DMAS-69491] - Update PlottingUtilityPlot.js to support querying annotations [DMAS-69499] - Create packages to generate meta.xml file [DMAS-69508] - Switch out Default Lasso Tool Icon for a better looking one [DMAS-69525] - Implement new Sound Metrics (Didson) sonar data product formats [DMAS-69537] - Change modifyBy field to name instead of ID [DMAS-69584] - Update data search annotations link to use new V2 URL [DMAS-69587] - Fix new checkstyle warnings in dmas [DMAS-69607] - Revise instructions block on Plotting Utility opening page [DMAS-69656] - Allow any user with RO privleges on the telnet console to see device data streams [DMAS-69679] - Create new options set for audio data products [DMAS-69681] - Set up Renovate in Jenkins [DMAS-69689] - Implement audio file downsampling using ffmpeg [DMAS-69692] - Get Annotations Menu Items working [DMAS-69701] - Remove and confirm removal of Budget Item and related modules from DMAS code [DMAS-69703] - Remove and confirm removal of Budget Junction Box and related modules from DMAS code [DMAS-69708] - Modify ARIS postprocess job and add device attributes movement detection configuration [DMAS-69721] - Add a field that displays the latest driver start time [DMAS-69723] - Update Sound Metrics postprocessor email recipients [DMAS-69740] - Add tests for QaqcAttributesTable.jsx and QaqcAttributesConfig.jsx [DMAS-69741] - Modify payload when click save qaqc [DMAS-69742] - Modify code when update a qaqc [DMAS-69748] - Write script to give Taxonomy RW group write permission on Attribute Management page [DMAS-69801] - Create DB script (to be run after each DB refresh) to support test cases made in DMAS-69712 [DMAS-69804] - DMAS-69804 Create a transaction table for the configproperty table [DMAS-69805] - Add better logging to help diagnose annotation form errors [DMAS-69807] - Update DMAS code to trace configproperty table changes Test [DMAS-69406] - Add test cases for Didson value-added data products [DMAS-69691] - Create automated test cases for audio downsampling data product options Bug (ST) [DMAS-69636] - AIS Source - filter out command responses [DMAS-69752] - Driver Scheduler doesn't search on Device ID [DMAS-69783] - QaqcAutotestsFinder doesn't search on Description, Device Id, or Sensor Id [DMAS-69785] - Device Listing Details IP Tab doesn't search on IP [DMAS-69787] - Device Actions does not search on Device Action ID [DMAS-69788] - Device Listing Page contains multiple tabs with inability to search link text [DMAS-69821] - Search failure in QA for new downsample audio data option [DMAS-69822] - Search failure in QA automation: out of memory in new append_pdfs function [DMAS-69830] - Fix add attribute crashing QAQC attributes page [DMAS-69872] - Didson test case generating duplicate new video where there's pre-generated video Investigation [DMAS-67559] - Investigate differences and changes needed to support MATLAB R2021b (or the version we upgrade to) [DMAS-69179] - Profile some of the slowest unit tests Documentation [DMAS-67997] - External data product documentation for Sound Metrics [DMAS-69404] - Internal documentation and requirements for Didson value-added data products [DMAS-69690] - Document new audio downsampling options in the DP wiki
May 26, 2022
Minor release: driver/seascript update for Didson sonar (no web downtime)
Support associating taxons from Search Tree Maintenance (pending review)
Annotation UI Updates
In Progress - not in this release: open new Annotation V2 search and entry pages from Device/Sensor, etc links
Instruments
SeapHOx (PH sensor) was completed last month
Modifications were made to some of the acoustic drivers to allow configuration files to be uploaded/deployed
Improvements to make use of existing infrastructure to more easily take photos from devices
Current project for AIS device is finishing up
EEW
Improvements made to the earthquake catalog page
Old earthquakes from 2018 onwards have been reprocessed, the data is now available
Leaflet map on earthquake page has a new tool to give the lengths in kilometers for drawn lines
Support for EEW simulations
Data Products
Added a new option for deployments with mobile position sensors to default to only integrating depth, lat, lon. Leaving out heading, pitch, roll makes the searches much faster (users can still opt in)
Speed improvement for mobile position sensor integration for some searches
[DMAS-67664] - As a creator of published plots on plotting utility, I would like to see my published plots listed separately from everyone else's and see names of who published other plots
[DMAS-69143] - As a user, I would like the Sensor tab in Oceans 3.0 to be sortable
[DMAS-69339] - As a dev, I would like to remove "Status Console" from list of resource name on User Management page
[DMAS-69364] - As a community fisher user, I would like to be able to lasso unassigned casts on the geospatial map
[DMAS-69470] - As a data specialist, I would like the Cast Re-postprocess Management UI to have the same filter options as Cast Review UI
Task
[DMAS-53807] - when the timestamp is empty and a user clicks a button (without autosave) the timestamp should populate
[DMAS-55748] - Investigate and improve performance for mobile position sensor scalar data searches (MATLAB)
[DMAS-56572] - Export button should be disabled once it is clicked. Otherwise users will click twice.
[DMAS-66754] - As a data steward, I want to see the Dive ID in SeaTube V3
[DMAS-66907] - Investigate the 2021-10-12 missed earthquake detection
[DMAS-67706] - Investigate adding a unique constraint on the sitedevicesubsetId column in searchtreenodesitedevicesubset
[DMAS-67825] - Add the mail.from configuration for emails
[DMAS-69274] - Filename change in ADCP data products (non-averaged) that cross midnight
User Story
[DMAS-42635] - As a developer and data steward, I would like to remove archivefiletype database table and remove relations of other database table with it
[DMAS-57234] - As a Data Specialist, I would like to be able to search on the /QaqcAutotestsFinder page for tests by device
[DMAS-64295] - As a data specialist, I would like the ability to flag a binned sensor as '4' and for the data products to be populated with NaNs for that bin
[DMAS-65487] - As a hydrophone data product user, include post-calibration txt files with hydrophone data products
[DMAS-66619] - As a NOAA ST admin, I would like to ingest archived chat log messages into ST
[DMAS-67068] - Export WoRMS annotations to event.csv
[DMAS-67910] - As a data specialist, I would like the Cast Review UI to have the functionality to add multiple sitedevicesubsets to review through filtering options
[DMAS-67912] - As a data specialist, I would like a to see a cast review status checkmark for each SDS in searchtreemaintenance
[DMAS-67929] - As a data specialist, I would like data products that are pre generated for the geospatial map to stay up to date with edits to casts
[DMAS-67990] - As a community fishers end-user, I would like an option to download COR files for individual casts instead of one large file with many casts
[DMAS-68969] - Select STv3 dive loggers should be able to delete annotations
[DMAS-69136] - As a data steward/operator, I would like a device attribute entry when there is only a default value
Task
[DMAS-64706] - Data search does not populate if archive file datefrom precedes sitedevice datefrom
[DMAS-67028] - Please investigate the response of seatub/map service
[DMAS-67230] - driver change for RBR BPRZero to have valve commands as device readings
[DMAS-67809] - For non-averaged ADCP MAT/netCDF in searches of 24 hours or less, concatenate the files over midnight
Fixed issues with rounding in the sensitivity and frequency values, enables narrow sweeps
Improve the drive amplitude user interface, add quick scaling parameters, improve and clarify use of drive amplitude max and bias values
Improve handling of files and file-naming, remove the save button
Improve handling of time stamps - in the metadata UI, in file-naming and in keeping track of acquisition and post-process times
Workflow and data file I/O improvements, improve switching between files / switching configurations, loading (particularly for old files) and saving, fix bugs in this area
Output data increasing order for better integration with Oceans 3.0
Minor improvements such as spelling mistakes, user manual updates, plot labels, improved diagnostic logging, add more wait/busy indicators, bug fixes
Improvements for main user interface
March 16, 2022
Minor release: database optimizations, bug fixes for Data Search, Dataset Landing pages, SeaTubeV3 chat log, Didson driver update.
[DMAS-69021] - Check for and fix missing deployments in SOO/E searches
Implementation
[DMAS-67893] - Add front end support for deleting chat log operation
[DMAS-67973] - Update UI to match with the requested mockups
[DMAS-68014] - Don't extend data rating datefrom/to in MATLAB metadata struct
March 1, 2022
Major release
Maintenance
Data Search data availability plot zoom in/zoom out/interaction now populates the datefrom/dateto
delete published save plots
improve PU axis auto-scale configuration
GIFs controls and links in Data Preview easier to click on
Dataset landing page landing time fix (woohoo)
Device Synchronization UI improvements
Various bugs and small improvements
Instruments
Nortek profiler driver modified to improve timestamp jitter. Change was made for device 64167, may apply to other deployed Nortek profiler instruments (device type 59 and 130)
ProOceanus CO2 Pro driver – fixed bug related to using the device controlled zero interval. This does not impact the driver controlled zero interval feature
Axis Camera driver – setFocusWindow command added for seascript and front end
Earthquake Early Warning
Update the EEW Report Job to expand how it relates organization detections
Fixed caching issue specific to correlator shore station
Support for historical/reprocessing earthquakes on the earthquake detail page
Added EEW performance metrics through jmx beans
improve speed/ease of deploying code to remote stations (in-progress)
SeaTube
Add back end support for Chat Log deletion
Support ST admin / dive chiefs synchronization
ST layout support for displaying Chat Log
User controlled index number display option for Quick Button Set
Some bug fixes
Data Products
sensor level data rating integration
bug fixes
Added a ASL echoview csv file-mode modifier
device-level data product mappings for nodes and SIIMs
CIOOS
Improved EOV keyword allocation for ISO19115 metadata
Created a front end prototype for biological expert review
Add checks to EOV and Marine Gazetter vocabularies to ensure incorrect attributes can not be added
Community Fishers
New page to review multiple casts at once
Bug fixes
Search page for site device subsets
Selectable sensors plotted on cast scalar multi profile plot
Name of plots on plotting utility no longer named with element id
Changes to plotting utility to fit long sensor unit names
Handle errors during creation of site device subsets
Ability to tell manual GPS apart of tablet GSP in tablet log file
Added community fishers site device subset management pages to DMAS menu
Contact information and support form added to the geospatial map
[DMAS-67777] - Manually created Site device subset id 6359 is missing reference information and name
[DMAS-67804] - Many NullPointExceptions in production from api calls
[DMAS-67811] - SiteDeviceSubsetTable Sorting on ID is broken
[DMAS-67824] - EEW correlator machine (eewshore1.dc.onc) - Cache loading errors due to NullPointerExceptions
[DMAS-67827] - ProOceanus Driver not setting idle timeout correctly
[DMAS-67859] - Creating new child taxon inherits sibling as child in taxonomy management
[DMAS-67861] - CODARContiguousConfigJob fails if a deviceid has no matching archivefiles
[DMAS-67870] - Oceans Next builds fail in Jenkins with "TS2307: Cannot find module 'assets/images/API-Oceans3.jpg' or its corresponding type declarations"
[DMAS-67915] - Test sometimes fails on Oceans 3.0's master branch
[DMAS-67921] - Fix test failing in DOIDatasetMinterImplTest in master branch
[DMAS-67922] - Video player not loading in SeaTubeV3
[DMAS-67924] - Any predefined layout other than Public is being compressed on refresh
[DMAS-37860] - As a user, I would like to utilize sensor-level data ratings in data products generated by matlab code
[DMAS-56796] - As a community user of the Geospatial map and CF app, we would like the ability to provide user feedback from these interfaces
[DMAS-64667] - As a data specialist, I would like to be able to search for a siteDeviceSubset ID
[DMAS-64684] - As a system admin, I would like the ST logger admins can be synced.
[DMAS-64772] - As a Community Fishers Data specialist, I want to be able to adjust parameters plotted in on-the-fly plots i.e. turbidity, CDOM etc. or even expand to 4+ plots, so that I can easily customize graphs (ONC data products) to include in reports.
[DMAS-67195] - As a data specialist, I would like a web page for displaying earthquake information
[DMAS-67577] - As a dev, I would like to see small UI improvement for Device Synchronization page
Task
[DMAS-10675] - Usability report suggests allowing click-drag and single click on a shaded area in Data Availability to populate the Date From and Date To
[DMAS-44797] - please add unique constraint for devicecode field in database table device
[DMAS-54120] - Please disable the delete button icon on User Management page for RO user
[DMAS-19662] - As an operator, I would like to be able to stop and start drivers on a schedule
[DMAS-25072] - As a creator of published plots on plotting utility, I would like the ability to be able to delete the published plots if I created them
[DMAS-50989] - As a Community Fishers data user, I would like configurable cast data products from Data Search
[DMAS-56294] - As a data steward, I would like a better date/time select tool (especially time)
[DMAS-56514] - As an Iqaluit Community Fishers Data Partner, I want to have the crude oil and refined fuels sensor added into suite (CTD, Oceans 2.0 data base and Pipeline data processing jobs)
[DMAS-57132] - As a user from NOAA, I would like to display chat logs into Ocean's 2
[DMAS-57835] - As a data specialist, I want cast data products to include annotations that they have been verified after being reviewed
[DMAS-64661] - As a data stewardship member, I would like to automatically update CODAR config file datetos when new files are archived
[DMAS-66713] - As a software developer, I want to extract session-handling boilerplate from DAOs to reduce the surface for bugs and reduce the number of tests needed
[DMAS-66862] - Import CSV with vocabularies and vocabulary terms to Taxonomies
[DMAS-66947] - As a Data Specialist, I want to re-post-process multiple Site Device Subset Ids at the same time.
Task
[DMAS-54910] - please remove two unused sequences (tm_taskgroup_seq and tm_taskdetail_seq)
[DMAS-56792] - jobs without names should not be created through the TaskManagement page
[DMAS-57135] - fix dmasuserid for quarantine operations
[DMAS-57851] - Add Monitoring to send a Notification/e-mail if a CF Pipeline job fails with an error
[DMAS-57933] - Add unassigned casts layer to Geospatial map
[DMAS-64392] - In PU only show CF nodes that have data
[DMAS-65593] - Can we include dmasuserid in dmas.log to tell who start or stop machine in reprocess?
[DMAS-66468] - Remove extra UUID column from the database
[DMAS-66701] - Write a script to update some incorrect tasktypeid in prod database
[DMAS-66730] - As a Data specialist, I would like extend the search field option to include deviceCode
[DMAS-66848] - Check on magnetic declination calculation
[DMAS-66995] - the state of devicetype DOI minting job 378 indicates completed when it is actually still running, causing problems when running job again before it is actually finished
[DMAS-67006] - document job 370 for DOI Tombstone Job
[DMAS-67177] - please create hist or trx table for groupprivilege database table
[DMAS-67225] - Allow Resource type to be changed when opening Annotation Entry from Device and Sensor
[DMAS-67264] - Add dateto to hydrophone sensitivity device attributes when adding new attributes through the device attribute upload
[DMAS-67282] - Return unassigned casts from GeospatialAreaService
[DMAS-67317] - as a Community Fishers data user, I would like Data Search to automatically apply the CF Sampling station filter when using a deep link to a CF Node
[DMAS-67326] - Update CF preview job to generate plots for unassigned casts
[DMAS-67386] - Investigate December 9th earthquake detection
[DMAS-67452] - Update all "Oceans 2.0" strings to "Oceans 3.0" in java code
[DMAS-67457] - Change links to DeviceListing in Oceans Next code to use OpenInNewLink
[DMAS-67472] - Update matlab code to use "Oceans 3.0" instead of "Oceans 2.0"
[DMAS-67493] - Apply changes to community fishers cast detection to VPS and BPS as well
[DMAS-67496] - EEW Event expression - move to JSON format only
[DMAS-67497] - Add job.failed.email.recipients job parameter for CF pipeline jobs
[DMAS-67498] - Add device-level MATLAB data products for JBs
[DMAS-67499] - move "migrated" earthquake files to Bitbucket
[DMAS-67532] - Investigate why we have different codecs for one ROV camera
[DMAS-67537] - new JEP formula for Rinko ARO-FT Oxygen Concentration Corrected
[DMAS-67547] - Do not send corrected cast emails from Cast Data (3) Correction Matlab Postprocess Job if no casts processed
[DMAS-67597] - New driver attributes for P/Swave_forgetting_factor and amplitude_step_size device attributes for EEW Accelerometer (350), Nanometrics Titan EA (156) and RBR Tilt Meter (430)
[DMAS-67601] - Replace references to qapgsql2 with qapgsql1 in Oceans2 code
[DMAS-67603] - Replace references to qapgsql2 with qapgsql1 in MATLAB code
[DMAS-67607] - Migrate Drupal Earthquake Dashboard code for new IG website