====== The internal data workflow with tipps and comments ====== FIXME | **Software** | **ToDo** | **tipp** ^ Done? | | | | | | | | | | saving the **''//raw data//''** | | | | | Intranet | 1a) save **''//internal collection data//''** (Excel tables) in folder of section at P:\ | "P:\CollDig\Sections\//sectionfolder//" | | | | Easydb | 1b) import **''//multimedia files of internal collections//''** to easydb | | | | | Intranet | 2a) save **''//external collection data//''** (Excel tables) to Raw folder in GFBio folder structure at P | "P:\GFBio\//projectfolder//\Raw folder | | | | MorphDBase | 2b) import **''//multimedia files//''** in MorphDBase and the raw folder at P:\ | | | | | DiversityCollection | **''//import data in DC//''** | | | | | | **''//check excel tables//''** for completeness | compare to [[https://datacenter.zfmk.de/wiki/abcd_overview|ABCD overview]] | | | | | check consistency of columns | content matches the header ? (is there need to revise the data with [[https://openrefine.org/|Open refine]]; use of only one separator in each column | | | | | check completeness of columns | has every column a header, are there all necessary columns? | | | | | check the formats of the data | coordinates (WGS 84, conversion see[[https://www.geoplaner.de/#geographische_Koordinaten|here]] or [[https://datacenter.zfmk.de/wiki/internal/doku.php/dwb:koordinatenumrechnung|here]] ) and dates (ISOdate or another clear date format) | | | | | save excel sheet as txt-file (Unicode-format) for import | | | | | | **''//create import scheme//''** and import data | | | | | GitLab | save the latest import scheme in **''//gitlab//''** | | | | | DiversityCollection | check completeness and correctness of **''//imported data//''** | | | | | | check correct withhold for data under **''//embargo//''** | see [[https://datacenter.zfmk.de/wiki/internal/doku.php/biodivinf:digitization:withhold|wikipage with withhold demands]] | | | | | **''//delete withhold//''** for released data | | | | | DOI Fabrica | create DOI for dataset | | | | | | create **''//new DOI as draft//''** at [[https://doi.datacite.org/sign-in|DataCite Fabrica]] | LogIn and schemes see [[https://datacenter.zfmk.de/wiki/internal/doku.php/doi:mds|DOI Fabrica]]; fill in ORCID and ROR | | | | | create **''//DOI metadata-xml//''** | use [...]\Nextcloud\DOIs\datacite_metadata_generator.html | | | | | | or update the DOI via the form function of DOI Fabrica page (see also: https://github.com/mpaluch/datacite-metadata-generator and https://schema.datacite.org/) | | | | | save **''//DOI metadata xml-file//''** | in folder [...]\Nextcloud\DOIs | | | | DiversityProjects | **''//import metadata in DP//''** | | | | | | check, if the latest version is installed | | | | | | import data from GFBio with special import in DiversityProject | LogIn to GFBio ticket system see [[https://datacenter.zfmk.de/wiki/internal/doku.php/gfbio:jira|here]] | | | | | check **''//withhold demands//''** of each section | see [[https://datacenter.zfmk.de/wiki/internal/doku.php/biodivinf:digitization:withhold|wikipage with withhold demands]] | | | | | check completeness and correctness of **''//metadata//''** | compare to [[https://datacenter.zfmk.de/wiki/abcd_overview|ABCD overview]]: title, description, authors/contacts, affiliations, licenses, citation, key words, Identifiers and DOI | | | | | **''//insert DOI//''** for new major version | new DOI only for a new major version (if data have been added), keep existing DOI with new minor version (if only typos are corrected) | | | | | **''//trigger transfer to BioCASe//''** | triggered by: new version number (minor or major) + the box "Create archive" must be checked + embargo must have expired (script for transfer is automatically running between 8 and 16 o'clock) & to identical numbers are not accepted: Version 1.1 is not working, only version 1.2 oder 2.1 | | | | BioCASe Provider Software | create **''//new archive//''** for dataset | title is project-title, use GGBN (ABCD 2.1), Paging size: 1000 | | | | | control new archive | check , if data are complete transferred, delete the browser cache (Firefox) if necessary | | | | Intranet | **''//save the archive//''** | in [...]\Nextcloud\GFBio\Archive-easydb | | | | | **''//add date to archive-title//''** | | | | | Easydb | **''//upload new archive with date//''** | new minor version: upload only new version, no new DOI, (Upload as new version under Datei - Versionen, only active, if you press "Bearbeiten" in the upper right corner) | | | | | | new major version: new entry, new DOI, new metadata | | | | | create new **''//thumbnail//''** | for templates and examples see: [...] \Nextcloud\GFBio\Bilder für Archive in easydb (size complete: 290 x 170px; size collection: 180 x 135 px) | | | | DOI Fabrica | if all is ready, **''//set DOI to "findable"//''** | | | | | BioCASe Monitor Service | **''//insert new entry//''** or update old entry | | | | | | **''//insert link//''** to archive in BPS | copy link with right mouse button in BPS and insert in BMS | | | | | check data in **''//"Consistency check"//''** | | | | | GFBio-Portal /JIRA | **''//create DEVOPS-ticket//''** to harvest new dataset | assign Andree Behnken, no need to create a second ticket for VAT: the georeferenced data are automatically harvested in the night (by VAT/ Michael Mattig) | | | | | **''//check entries for correctness and presence//''** | sometimes they need days or weeks to harvest the new entries | | | | GFBio-Wiki | insert new entries in **''//GFBio-Wiki//''** or update old ones | at page: https://gfbio.biowikifarm.net/internal/Overview_on_ABCD_datasets_published_via_BioCASe_data_pipelines | | | | JIRA | add DOI to original ticket | | | | | | send a **''//Service Quality Survey//''** - link to the data producer(s) | see : https://gfbio.biowikifarm.net/internal/Checklist_for_ABCD_datasets_published_via_BioCASe_data_pipelines | | | | BioDivInf Wiki | insert new entries | at https://datacenter.zfmk.de/wiki/internal/doku.php/gfbio:datasets | | | | | | | | |