Updated schug library to patched version 1.8

[3.0.1]

Fixed

• Updated schug library to v1.8 (contains bug fixes)

Database schema change and other fixes

[3.0]

Changed

• Updated several libraries including schug (now v1.7)
• Update project's Python version to 3.9
• BREAKING CHANGE: modified the structure of the database table genes, converting the ensembl_id string field to ensembl_ids: an array of strings. This change addresses recent changes in the MySQL: https://bugs.mysql.com/bug.php?id=114838

Fixed

• The MariaDB healthcheck step in docker-compose-mysql.yml, preventing the demo app to start

Library updates, show complete intervals info and other

[2.1]

Added

• Refseq transcripts names on coverage overview page

Changed

• Replaced custom badges style with Bootstrap 5 badges
• Return complete gene, transcripts and exons info in intervals endpoints

Fixed

• Addressed the Starlette Denial of service (DoS) via multipart/form-data by updating starlette library, among others

Major release 2.0

[2.0]

Added

• Improve report explanation to better interpret average coverage and coverage completeness stats shown on the coverage report
• Check that provided d4 files when running queries using /coverage/d4/genes/summary endpoint are valid, with test
• General report with coverage over the entire genome when no genes or genes panels are provided
• A MANE coverage report, showing coverage and coverage completeness only on MANE transcripts for the provided list of genes
• Link out from MANE overview to gene overview
• Save ensembl_transcript_id and exon rank info on exons database records
• Display MANE badges on gene overview report
• Create PDF button on MANE overview and gene overview pages
• Documentation on how to create MANE overview reports

Changed

• Do not use stored cases/samples any more and run stats exclusively on d4 files paths provided by the user in real time
• How parameters are passed to starlette.templating since it was raising a deprecation warning.
• Replaced deprecated Pydantic parse_obj method with model_validate
• Report and genes overview endpoints accept only POST requests with form data now (application/x-www-form-urlencoded) - no json
• Sort alphabetically the list genes that are incompletely covered on report page
• d4_genes_condensed_summary coverage endpoint will not convert nan or inf coverage values to None, but to str(value)
• Updated the Dockerfile base image so it contains the latest d4tools (master branch)
• Updated tests workflow to cargo install the latest d4tools from git (master branch)
• Computing coverage completeness stats using d4tools perc_cov stat function (much quicker reports)
• Moved functions computing the coverage stats to a separate meta/handle_coverage_stats.py module
• Refactored code collecting stats shown on gene overview report
• Gene report to contain both transcripts and exons stats

Fixed

• Updated dependencies including certifi to address dependabot alert
• Update pytest to v.7.4.4 to address a ReDoS vulnerability
• Colored logs
• Link for switching between coverage thresholds on overview report
• Gene links in genes overview page open into new tabs

Update d4tools to v0.3.10 and new condensed summary endpoint

[1.9]

Added

• Condensed /coverage/d4/genes/summary for condensed stats over a gene list
• Documentation for new coverage summary endpoint

Changed

• GitHub tests action to use d4tools 0.3.10

Fixed

• Updated dependencies to address dependabot's security alerts
• Use a base image containing d4tools 0.3.10 in Dockerfile

Updated actions and templates, added cryptography dependency

[1.8]

Added

• cryptography lib dependency

Changed

• Updated PR template
• Generalised issue templates to make them more user-friendly for people outside our organisation
• Moved logging setup out of app lifespan and db initialisation logic

Fixed

• Updated version of external images used in GitHub actions

Removed pyd4 lib and fixed coverage report template

[1.7]

Added

• An environment.yml with the minimum supported python version (3.8) and the installed libs

Changed

• pyd4 library no longer available in chanjo2 Docker image

Fixed

• Position of Show genes checkbox on report page
• Updating gene panel name using the web form on report page

Report endpoints accepting form data and other improvements

[1.6]

Added

• Coverage report and genes coverage overview endpoints now accept also requests with application/x-www-form-urlencoded data
• Allow system admin to customise coverage levels to be used in reports' metrics by editing the REPORT_COVERAGE_LEVELS in .env file
• Documentation on how to change app's default coverage level values to be used when creating the reports

Changed

• Templates form submit data as application/x-www-form-urlencoded without having to transform it into json
• Customize form on report page now accepts genes as Ensembl IDs or HGNC symbols

Fixed

• Faster genes overview report loading
• Broken GitHub action due to d4tools failing to install using cargo
• Broken Codecov upload step in GitHub action failing due to missing token
• Completeness cutoff select not updating after submitting customize form on report page

Fix "Server has gone away" and other things

[1.5.1]

Fixed

• Avoid MySQLdb.OperationalError Server has gone away by modifying by setting pool_pre_ping=True when creating the engine
• Coverage report screenshot displayed on README page and on the documenattion to reflect true statistics from the demo samples
• Coverage report/overview page crashing when transcripts or exons intervals are required only genes are loaded
• Coverage overview over a gene should return transcript statistics if D4 file contains WGS data

Switch from pyd4 library to d4tools calls and fixes

[1.5]

Added

• coverage.d4_intervals_coverage responses contain also interval name as provided in bed file
• coverage.d4_interval_coverage responses now returns also the genomic region used to compute the stats on
• Test for modified function collecting coverage report data

Changed

• Speed up response by coverage.d4_intervals_coverage by replacing pyd4 lib with direct calls d4tools and multiprocessing
• Removed 2 redundant functions in meta.handle.bed.py
• coverage.d4_interval_coverage is using direct calls to d4tools to retrieve stats over an entire chromosome or a genomic interval
• Reformat report sample' sex rows and coverage.get_samples_predicted_sex endpoint to use d4tools and not pyd4 for evaluating sample sex
• Refactored code to create coverage report and genes overview report to be faster by using d4tools calls and multiprocessing
• Renamed handle_tasks.py to handle_completeness_tasks.py
• Refactored coverage endpoints samples_genes_coverage, samples_transcripts_coverage and samples_exons_coverage to use calls to d4tools instead of the pyd4 library
• Speed up coverage report creation by collecting SQL intervals before looping through samples stats
• Refactored gene overview to use d4tools instead of pyd4 lib to compute gene-level intervals stats
• Removed pyd4 lib and all remaining code which was still using it

Fixed

• coverage.d4_interval_coverage endpoint crashing trying to computer coverage completeness over an entire chromosome
• Samples mean coverage values a hundredfold higher on coverage reports
• Install software packages using poetry v<1.8 to avoid problems installing pyd4 (pyd4 not supporting PEP 517 builds)
• Typo in report template with unclosed span/div causing cramped genes not found message
• Mariadb container not passing healthcheck when runned from demo docker-compose file
• Fixed an error on gene overview that made the coverage seem 100-folds higher
• Return error when genes are not provided in the request form to create a coverage report