.. _data_submission:

Data Submission
===============


These are an overview of the steps to be followed by a researcher willing to store his/her data collections in es-FEGA, *i.e.*, the submitter:

.. contents:: Overview
   :depth: 2
   :backlinks: top


Step 1: Application for a new data submission 
---------------------------------------------

::

    How?                Contacting via email es-fega.helpdesk@bsc.es
    Expected outcome    


To begin the submission process, just contact the es-FEGA helpdesk team with some **information on the data collection** willing to be archived, along with any doubt you may have about the service. Please, make sure to provide as much information as possible on the data as well as the expected access control. Consider to include the following items: 

* Contact information
        * Submitter Name
        * Institution
        * Group/Department
        * Research Project/Consortium
* Dataset/s description
        * Dataset/s size (GB)
        * Number of files
        * Number of samples
* Data access
        * Is data consented for research purposes?
        * Does data require controlled access?

  
Given the above information, the helpesk team is going to consider the particulars of your case and will send you further instructions on the following steps (here summarized).

.. raw:: html

    <embed>
        <a style="font-size:1.2em;padding: 5px 50px;border: 1px solid;margin: 20px;text-decoration: none;" href="mailto:es-fega.helpdesk@bsc.es?subject=New Data Submission&body=Submitter%20Name%0D%0AInstitution%0D%0AGroup%2fDepartment%0D%0AResearch%20Project%2fConsortium%0D%0ADataset%2fs%20size%20%28GB%29%0D%0ANumber%20of%20files%0D%0ANumber%20of%20samples%0D%0AIs%20data%20consented%20for%20research%20purposes%3f%0D%0ADoes%20data%20require%20controlled%20access%3f">Contact es-fega.helpdesk@bsc.es</a>
    </embed>
   

Step 2: Agreement of legal requirements
---------------------------------------

::

    How?                Contacting via email es-fega.helpdesk@bsc.es
    Expected outcome    


According to the General Data Protection Regulation, submitters must confirm they have the authority and permission to deposit the data collection in es-FEGA. It is proven by providing and/or agreeing on a serie of legal agreements involving several actors, like the Barcelona Supercomputing Center, as the Data Processor and Service Provider of Spanish Federated EGA node, and the Data Producers and Controllers, usually corresponding to the submitter, his/her collaborators, and/or his/her home institution.

Following, a list of the legal terms and agreements applicable to the data deposition phase. The submitter can share the following document/s with the legal service of his/her home organization for revision. Some of them require to be filled in and send back to es-FEGA to es-fega.helpdesk@bsc.es.

**Data Processing Agreement**  (DPA) signed by the submitter's organization, as Data Controller, and the Barcelona Supercomputing Center, as the Data Processor and Service Provider of Spanish Federated EGA node.


.. list-table:: Formal and legal requirements
   :widths: 25 25 50
   :header-rows: 1

   * - Name
     - Involved parts 
     - Document
   * - Data Processing Agreement (DPA)
     - Agreement between:
         * Data Controller: usually, submitter's home organization
         * Data Processor and Service Provider: the Barcelona Supercomputing Center
     - The following legal notices are accepted when using es-FEGA services:
         * `BSC Data Processing Agreement`_
         * `BSC general terms of use`_
   * - Data Access Agreement (`DAA`_)
     - Agreement between:
         * Data User: usually, the submitter
         * Data Access Comittee (`DAC`_): usually, submitter's collaborators
     - The following document is to be filled and send to es-fega.helpdesk@bsc.es in step 4
         * `EGA default Data Access Agreement`_

A more comprehensive guide and overview of the formal requirements will be provided here soon.

Step 3: Registering
-------------------

::

    Who?        Submitter
    How?        contacting via email es-fega.helpdesk@bsc.es


The Submitter will need to obtain a Central EGA account prepared for submission to our Spain Federated EGA node. This is simply requested through an email to the Central EGA helpdesk (helpdesk@ega-archive.org).


Step 4: Data encryption and upload
----------------------------------

In order to deposit your data, it must be encrypted first with Crypt4GH format, the Global Alliance for Genomics and Health (GA4GH) standard for securetly sharing human genetic data.

Visit `Crypt4GH cryptor documentation page <https://crypt4gh.readthedocs.io/en/latest/>`_ to learn more on the format. The site also include the instruction on how to `install <https://crypt4gh.readthedocs.io/en/latest/setup.html>`_ it,

The `public key of our Spanish FEGA Node <_static/es-fega_master.key.pub>`_ is required for encrypting your data. Please, download it.

.. code-block:: console

   curl https://fega-test.bsc.es/docs/_static/es-fega_master.key.pub > es-fega_master.key.pub
   
   crypt4gh generate -n <key-name>

   crypt4gh encrypt -f <file to encrypt> \
     --sk  <key-name>.sec.pem \
     --recipient_pk es-fega_master.key.pub > [encrypted file generated]



Step 5: Metadata submission
---------------------------

Step 6: Data release
--------------------

.. _BSC Data Processing Agreement: https://coming.soon
.. _BSC general terms of use: https://www.bsc.es/legal-notice
.. _EGA default Data Access Agreement: https://ega-archive.org/files/Example_DAA.doc
.. _DAA: https://ega-archive.org/submission/dac/documentation#DAA
.. _DAC: https://ega-archive.org/submission/data_access_committee