Skip to content

Adapt Connector to use external S3 storage for storing Einstein data

Nicola Stoira requested to merge sphnc-183 into einstein-main

The changes are not shown correctly. To see the full diff check out: https://git.dcc.sib.swiss/sphn-semantic-framework/sphn-connector/-/compare/einstein-main...sphnc-183?from_project_id=474&straight=true Changes are going to be merged in reference branch for Einstein einstein-main

Main changes:

  • All interactions from Connector to Einstein related bucket are not executed with boto3 (therefore ideally also supporting S3 compatible storage)
  • In the .env we can define if the storage is external or not. If it is external the connection parameters must be defined
  • Tested with an additional Minio storage. Connector was able to write to the external storage and notify the data to Einstein. Einstein was able to pull the data from the external storage and load it into Virtuoso.

Merge request reports