Inductive Learning of Concept Representations from Library-Scale Corpora with Graph Convolution

Published in INFORMATIK, 2019

Recommended citation: Lukas Galke, Tetyana Melnychuk, Eva Seidlmayer, Steffen Trog, Konrad Foerstner, Carsten Schultz, Klaus Tochtermann, "Inductive Learning of Concept Representations from Library-Scale Corpora with Graph Convolution." INFORMATIK, 2019.

Access paper here

Research conducted in context of the Q-AKTIV project.

Abstract: Automated research analyses are becoming more and more important as the volume of research items grows at an increasing pace. We pursue a new direction for dynamic research analyses with graph neural networks. So far, graph neural networks have only been applied to small-scale datasets and primarily supervised tasks such as node classification. We propose to use an unsupervised training objective for concept representation learning that is tailored towards bibliographic data with millions of research papers and thousands of concepts from a controlled vocabulary. We have evaluated the learned representations in clustering and classification downstream tasks. Furthermore, we have conducted nearest concept queries in the representation space. Our results show that the representations learned by graph convolution with our training objective are comparable to the ones learned by the DeepWalk algorithm. Our findings suggest that concept embeddings can be solely derived from the text of associated documents without using a lookup-table embedding. Thus, graph neural networks can operate on arbitrary document collections without re-training. This property makes graph neural networks useful for dynamic research analysis, which is often conducted on time-based snapshots of bibliographic data.

    author = {Galke, Lukas AND Melnychuk, Tetyana AND Seidlmayer, Eva AND Trog, Steffen AND Förstner, Konrad U. AND Schultz, Carsten AND Tochtermann, Klaus},
    title = {Inductive Learning of Concept Representations from Library-Scale Bibliographic Corpora},
    booktitle = {INFORMATIK 2019: 50 Jahre Gesellschaft für Informatik – Informatik für Gesellschaft},
    year = {2019},
    editor = {David, Klaus AND Geihs, Kurt AND Lange, Martin AND Stumme, Gerd} ,
    pages = { 219-232 } ,
    doi = { 10.18420/inf2019_26 },
    publisher = {Gesellschaft für Informatik e.V.},
    address = {Bonn}