Exploring scholarly publications using DBPedia concepts: an experiment
This post is about a recent prototype I developed, which allows exploration of a sample collection of Springer Nature publications using subject tags automatically extracted from DBPedia.
DBpedia is a crowd-sourced community effort to extract structured content from the information created in various Wikimedia projects. This structured information resembles an open knowledge graph (OKG) which is available for everyone on the Web.
Datasets
The dataset I used is the result of a collaboration with Beyza Yaman, a researcher working with the DBpedia team in Leipzig, who used the SciGraph datasets as input to the DBPedia-Spotlight entity-mining tool.
By using DBPedia-Spotlight, we automatically associated DBpedia subject terms to a subset of abstracts available in the SciGraph dataset (around 90,000 abstracts from 2017 publications).
The prototype allows searching of Springer Nature publications using these subject terms. Additionally, DBpedia subjects include definitions and semantic relationships. While we are currently not using these features, one can imagine how they could be raw material for generating more thematic 'pathways' through the content.
Results: Serendipitous Discovery of Scientific Publications
The results are pretty encouraging. Despite the fact that the concepts extracted are sometimes only marginally relevant (or not relevant at all), the breadth and depth of the DBpedia classification makes the interactive exploration quite interesting and serendipitous.
You can judge for yourself. The tool is available here: https://dbpedialinks.michelepasin.org/
The purpose of this prototype is to evaluate the quality of the tagging and generate ideas for future applications. Any kind of feedback or ideas is very welcome!
We are working with Beyza to write up the results of this investigation as a research paper. The data and software are already freely available on GitHub.
A Couple of Screenshots
For example, see the topic 'artificial intelligence':
One can add more subjects to a search in order to 'zoom in' into a results set, for example by adding 'China' to the search:
Implementation Details
- Main webapp is using Python and Django. I finally found a good excuse to upgrade to the latest release (2.1), so very proud of myself!
- Bootstrap for UI components
- d3.js for visualizations, in particular I learned a lot from this great force-directed layout example
- The Springer Nature SciGraph dataset of scientific publications
- The DBpedia Spotlight tool for entity extraction
- See the GitHub source for more details