Architecture
The idea of the current architecture is to require as little of a backend as possible. By using local stores and the Linked open data vocabularies API we remove the need of managing relations and classes. And by using a local RDF store, which implements a SPARQL endpoint, we have no need of a backend structure except to provide the page
Within this design, the web-application is responsible for:
- Transforming CSV's to RDF
- Storing data using open RDF-endpoint
- Visualising the result for the users
- Querying and visualizing the result
The vocabularies are responsible for:
- Supplying the app with popular vocabularies
The outside SPARQL endpoints are responsible for:
- Supplying the app with popular data
During development of the service a multitude of changes. When starting this project the first architecture was a more traditional one.
The idea behind this structure is that the web-application will provide an interface for the user to convert their data and store it remotely in order to make it accessible to the world. The extra database is created in order to store metadata about the graphs and the user information.
The functionalities of the vocabularies and the external SPARQL endpoints are the same as described above
When the project started the following architecture was planned. As the Data conversion functionality was planned to only run locally there was no need for a server during the development of that feature.
The difference between previous architecture is that within this architecture the triple store is responsible for storing metadata and users. This would be accomplished by using the default graph to store meta- and user-data. This meant we needed to design/maintain one less database.
At the end of the implementation of the data transformation, there was no sight on a usable SPARQL endpoint to store the data. And one of the key points was that we needed a structure which allowed the user to have full rights over their data but also make it publically available to use. Therefore the use of an online file sharing API (ex Dropbox) was considered as a use. Where the user would have to login using the API's authorization system and the files be stored in a folder where both they and the application have access to in order to share their datasets with each other.
Another feature which needs redesigning was the feature that user was able to rate each other's data. In order to develop this feature, the service needs to store its own index of rating which is solved by keeping a document as a database which would update each time the user loads the page.
While further researching into the Dropbox API we discovered a sour apple. The Dropbox API cannot be used to create another file-sharing application. Which is unfortunate as the service is within a gray area, while the foremost functionality is to create and query Linked Data the feature where people are able to use each other's data makes it so the application could be seen as a file-sharing application. Therefore the choice was made to implement a local triple store with a SPARQL endpoint which removes the need for remote storage of user data. By using SPARQL queries instead of supplied methods we are able to quickly switch between endpoints.
After a meeting with the Product Owner, he suggested and supplied an open SPARQL endpoint. Because the connection with the local SPARQL-endpoint was already using SPARQL queries the transfer to the open endpoint was easy. Because this is an open endpoint it wouldn't be nice to use the default graph and let everybody name their datasets without a format.
As our default graph, we use 'http://www.gerwinbosch.nl/rdf-paqt/metadata' where the metadata of the data is stored. The data is stored with the people's own title of the dataset with 'http://www.gerwinbosch.nl/rdf-paqt/data/' prepended to make sure people do not overwrite graphs which have nothing to do with the application.
Of course, there is still the possibility of misuse of the endpoint.