Skip to content

Latest commit

 

History

History
93 lines (66 loc) · 5.11 KB

04.b-upload-metadata.md

File metadata and controls

93 lines (66 loc) · 5.11 KB

Uploading to the metadata catalogue (CKAN portal)

This document describes how to upload metadata to your CKAN instance.

Environment

Ubuntu 14.04 server

Prerequisites

1. Working CKAN server

  • You need a running CKAN instance, of course. For installation see the previous module Install CKAN.
  • CKAN manager account To be allowed to upload and update datasets in your CKAN instance, you must first create a CKAN sysadmin account. How this is done is described in the previous module, CKAN configuration.

2. Internet connection and browser

The supported and tested browsers are:

  • Firefox, version 45 or later
  • Google Chrome, version 51 or later

3. Spreadsheet template

For this part, you can document in the Mapping V n.m (YYMMDD) tab in the excel spreadsheet Community-B2FIND.template.xlsx. Examples of how the metadata id represented in the original community portal (as far it exists) and in your (or the B2FIND) portal should come in the columsns of this tab. Typically at this stage, a version for your community is created with the tabs Harvest and Mapping already filled out (by the B2FIND team and/or the community/project members).

4. The uploader software

For uploading, we use the python script sh mdmanger.py again, this time in the ‘u’, upload, mode.

5. Some JSON samples

The uploader process expects validated CKAN-compatible JSON files to be available in the directory oaidata/<projectname>/<subset>/json. This is the case if the associated mapping and validation modules have been excecuted successfully.

The Upload process

1. Create a CKAN Organization

The CKAN group denotes the owner of the data. First add a CKAN Organization named rda. To do this, you have to login as thw CKAN admin in the CKAN website by clicking on Login.

Then go to Organizations and click on Add organization. Now enter RDA as the organisation name and add a description.

Note: We have to use RDA here as the organisation name, because this is used and needed in the upload source code. Additionally, you can add a logo by providing a link to a picture.

Confirm by clicking on Create Organization and the result should look like:

2. Create a CKAN group for your project/community

To upload metadata for your project, you must create a correspending group in your CKAN instance (i.e. community in the B2FIND case), to group all metadata of the project under one umbrella.

To do this you have to login as CKAN admin in the CKAN website by clicking on 'Login'.

Then select 'Groups' from the main menu on the top and click on 'Add group'.

We enter fishproject as group name and add a description of the project.

Note : The group name is internally changed by CKAN to a lower case string. You should only take care that this string is the same as used in the harvest and mapping procedure in the previous modules. Additionally, you can add a logo by providing a link to a picture.

Confirming by clicking on Create Group results in an empty CKAN group:

3a. Manual upload of metadata to CKAN

You can use the graphical user interface of CKAN to create metadata and link it to an organization and a group. Click on the tab Datasets and then on the button Add Dataset. You are now asked to fill in some metadata.

In the next tab you can also add some data to the entry. Click on finish. To link the dataset to a specific group click on the dataset and open the tab Groups. Here you can choose under which groups the dataset shall appear.

Exercise Create some datasets. How can you add some real data to an entry? How can you link a dataset with some external data resource, e.g. another entry in a repository?

3b. Uploading JSON files as datasets with the mdmanager script

Before you can upload data to your CKAN instance, you need an appropriate API-key. While logged in as admin user, click on your admin name (top panel on the right). You will find your key in the left corner below your username. You can either pass your API key every time you upload data with the option --auth=<API key>, or create a file /home/<user>/.netrc and add the following line to the file:

<ip-address, fully qualified domain name>:8181	<API key>

Uploading is executed using the option --mode u, and in addition to the options used before, the IP adress of the CKAN instance has to be specified with the option -i.

./mdmanager.py --mode u -c fishproject -s http://localhost:8181/oai/provider --mdsubset sample_1 --mdprefix oai_dc -i localhost

3. Check the uploaded files

If all worked fine, you should now see the uploaded datasets under the group <ProjectName> in the CKAN website.