Skip to content

Restructuring of backup and restore functionalities

Nicola Stoira requested to merge sphnc-789 into main
  • New endpoint /Backup_database which returns ZIP file with insert statements: schema, tables, and types creation for the existing projects, sphn_tables patient data, internal tables.
  • New endpoint /Restore_database which reingests all the database data into a clean database
  • User MUST save the S3 content by their own. If that's not done the backup and restore might be useless as all the data will be missing.
  • The old endpoints have been dropped completely, likewise the .env parameter for backing up external data
  • Suggested edits on the documentation, when approved I can include the new docs in this MR.

I think we should be very confident that the old functionalities can be removed otherwise it might be complicated to re-introduce them.

Example of extracted file: SPHN_Connector_Database_Backup-2024-12-03T15_39_26.zip

@philip.krauss One thing I noticed in the Insel installation is that they probably define the POSTGRES_USER parameter as "postgres". This means that the internal tables are in the same database as the Airflow tables. Therefore, when we check after the reinstallation if the database is empty we find that it is not empty because some Airflow data has been written to those tables. The restore is therefore blocked. We do have two options I think here:

  1. In the setup.sh we forbid POSTGRES_USER="postgres" so that there is no overlap
  2. With a bit more effort we can try to change the default database for Airflow so that it writes to a database named e.g. airflow_db and then we forbid it in the setup.sh as well.
Edited by Nicola Stoira

Merge request reports

Loading