openEngiadina

openEngiadina is a platform for creating, publishing and using open local knowledge.

1 Overview

1.1 Local Knowledge

There are thousands of little pieces of information and data that describe the social, cultural and natural environment of an area, we call such data local knowledge.

Local knowledge may include:

Local knowledge is produced and used by various people and organizations (actors). Actors that may interact with openEngiadina include:

1.2 Open Knowledge

We believe that local knowledge should be freely available to everyone to use and republish as they wish, local knowledge needs to be open data. However, having access to data is not enough. Data needs to be accessible (useful and efficiently useable). Local Knowledge needs to be open knowledge.

Technically local knowledge is made open and accessible by describing it as linked data.

1.3 Crowdsourcing

The vast amount of diverse data points that comprise local knowledge make local knowledge practically impossible to centrally manage or control. We observe that the various actors themselves need the means to publish data (e.g. a local book store can publish its changed opening times without appealing to any central authority). Note that large tech giants also rely on individual actors to publish data in a structured way.

Local knowledge can be expressed in an accessible way as linked data. The ActivityPub protocol provides the means to publish such knowledge in a decentralized manner. Additionally the ActivityPub protocol allows for social interactions such as commenting on pieces of knowledge, liking them or sharing them - features well known from popular social networks.

1.3.1 Example: Publishing an event

Currently the Engadin tourism office offers an online form for submitting a new event. The form causes an e-mail to be sent to a content manager working for the tourism office. The data is manually checked and then uploaded to the database of a commercial service provider. Finally the event is retrieved from the commercial service provider and displayed as plain text with minimal markup on the online event calendar.

With openEngiadina the user or local organization creates the event trough a interface similar to one offered by popular social network platforms. The tourism office is notified about the creation immediately and can tag the created event for publishing, this is very similar to "liking" an event. The online event calendar is generated by querying for all events that have been "liked" by the tourism organization. The event can be published almost instantenously, openEngiadina can be used as a crowdsourced content management system. Further advantages include:

A more complex example might be status of hiking trails. Currently, publishing changes in status of trails requires a high level of coordination between various public services, the commune and the tourism office. openEngiadina might be able to simplify the work flow for handling status of trails by acknowledging the fact that publishing such data is mostly a social endeavor.

1.4 Open Development

openEngiadina is not only meant to be useable by a wide population but should encourage development of the platform itself and applications interfacing with it. In particular openEngiadina should be useable for projects at local schools and hackatons.

This is accomplished by:

Some ideas for extensions or applications using openEngiadina include:

1.5 Objectives

The objectives of the project, ideas and current work-in-progress is documented here.

2 Semantic Social Network

openEngiadina is a cross between a knowledge repository and a social network. Knowledge is created trough social interactions and knowledge can be used in social interactions. We call this a Semantic Social Network.

The two fundamental technolgies used to achieve this are Linked Data and the ActivityPub protocol.

2.1 Linked Data

Knowledge may be represented as a graph consisting of vertices which represent things and edges which represent relations between things. To make this consistent and machine-understandable it is important to name everything (things and relations) so that semantically same things (and relations) can be identified across different sets of data.

A clever convention is to use internet addresses (URIs) as names and place data about the thing at its URI. Now things can be referred to by their URI and data about the thing can be retrieved by simply looking up the URI. This allows data that is published in different places on the internet to be linked together and to be used very efficiently by machines, this is the core idea of linked data.

Large collections of names and rules (schemas) for describing plenty of things are already defined and widely used. There are many existing linked data sets (e.g. Wikidata).

openEngiadina primarly uses the JSON-LD format to represent linked data.

2.1.1 Triplestore

openEngiadina uses a Triplestore as underlying database allowing for highly optimized semantic queries. This is a key novelty and technical challenge of the project. Most existing projects use a relational database.

Initial experiments on how to use a Triplestore have been conducted.

2.2 ActivityPub

ActivityPub is an open and decentralized protocol for distributed publishing and social networking. It resembles traditional e-mail in the sense that actors (persons, organizations or bots) have accounts on servers of their choice and can publish messages to other actors (who might be on different servers). The protocol handles delivery of the messages to the recipient actors. Unlike e-mail the messages are structured data, linked data.

The structure of the messages is defined in the ActivityStream standard. Messages are activities such as creating, updating, deleting, liking or sharing an object. ActivityStream is serialized as JSON-LD and objects can be any linked data.

Historically the ActivityPub protocol was born out of improvements to previous protocols for federated social networking.

2.2.1 Fediverse

A number of services already exist implementing the ActivityPub protocol for federation. That is content from one service is available on an other (provided the client can display the content). The universe of these federated services is called the fediverse. openEngiadina will be part of the fediverse.

Some existing services:

Currently there are almost 3 million users on the fediverse spread out on 4'500 servers.

3 Related projects

3.1 DATAtourisme

Open Data platform for touristic data. DATAtourisme also uses a Triplestore as underlying database (Blazegraph).

3.2 Mastodon and Pleroma

Popular Microblogging platforms that use the ActivityPub protocol.

Both projects are restricted to a small set of object types that can be shared. openEngiadina aims to support any object type.

3.3 CKAN

An open-source data portal platform widely used by public institutions to publish open data (e.g. Open Data Portal of Z├╝rich Tourism).

openEngiadina aims to be compatible with CKAN portals to a high degree. Data available in CKAN portals should be readily usable from within openEngiadina.

3.4 CommonsPub

CommonsPub is a project to build a generic federated server, based on the ActivityPub web standard.

Unlike CommonsPub openEngiadina uses a triplestore as underlying database.

3.5 Solid

Sold aims to create decentralized web where users are in control of where and how their data is stored.

openEngiadina intends to make use of many of the standards published by the project (e.g. Web Access Control).

3.6 Linked Data Fragments

A project exploring how to query Linked Data that is spread across the web.

openEngiadina aims to use results by the project for federated queries and for user interfaces that provide autocompletion by live querying existing data sets.

See also the Comunica Framework published by the project.

3.7 Openki

A web platform for promoting access to education by enabling peer-to-peer courses.

openEngiadina shares the idea of crowdsourced creation of events but builds on the ActivityPub protocol.

3.8 Spritely

Spritely aims to create a federated social network and is conducting various small experiments towards that goal. openEngiadina makes use of the research being conducted.

4 About

For questions, feedback and comments please contact pukkamustard (pukkamustard [at] posteo [dot] net).

openEngiadina is supported by miaEngiadina.