Using the AIDA Language to Formally Organize Scientific Claims
Sixth International Workshop on Controlled Natural Language (CNL 2018)
27 August 2018
These Slides: https://tinyurl.com/cnl2018kuhn
Optimized for Reading Single Work
Bad for Getting More General View
Unused Potential of Software/Databases
We Have: Network of Publications
typical edge: paper cites paper
We Need: Network of Results
typical edge: study supports hypothesis
We Need: Network of Knowledge
typical edge: gene causes disease
Networks of Results/Knowledge
Letting researchers communicate their findings in a way that
- allows for machine-interpretable representations
- is simple and intuitive
- is flexible and practical
- is general
We introduced the approach of AIDA Sentences in earlier work:
- Controlled Natural Language
- English sentences that are Atomic, Independent, Declarative, and Absolute
Kuhn, Barbano, Nagy, and Krauthammer. Broadening the Scope of Nanopublications. In Proceedings of the 10th Extended Semantic Web Conference (ESWC). Springer, 2013.
AIDA Sentence: Definition
- Atomic: a sentence describing one thought that cannot be further broken down in a practical way
- Independent: a sentence that can stand on its own, without external references like "this effect" or "we"
- Declarative: a complete sentence ending with a full stop that could (at least in theory) be true or false
- Absolute: a sentence describing the core of a claim ignoring the (un)certainty about its truth and ignoring how it was discovered (without "probably" or "evaluation showed that")
Linking AIDA Sentences
Informal, Semi-formal, Formal
Linking Informal, Semi-formal, and Formal Statements
Related Controlled Natural Languages
Basic English (~1930): designed to improve human communication in politics, economy, and science.
Other CNLs (ACE, CLEF) proposed for scientific results: focus on precision
AIDA is unique as a CNL for knowledge representation that focuses on expressivity instead of precision.
About AIDA sentences:
- How easily can researchers write them?
- How easily can they be automatically extracted?
- Are they really general and cover everything?
- Do researchers like them?
- Can they be easily linked and formalized?
First study on manual creation of AIDA sentences from abstracts by untrained researchers
Second study on automatic creation of AIDA sentences from texts with a simple algorithm
In both studies, about 70% of the created AIDA sentences were perfectly accurate
Alzheimer's Case Study
Manually representing the main statements of meta-review as AIDA sentences:
Alzheimer's Case Study: Results
Open Access Case Study
Representing the statements of another meta-review:
- On Open Access citation advantage
- General AIDA sentence: "Open Access publications receive on average more citations than similar publications that are not Open Access."
- More specific ones, e.g. "Open Access publications in astronomy and physics receive ..."
- Result: AIDA sentences for 70 publications
AIDA sentences in the classroom setting:
- Course entitled "Knowledge and Media"
- AIDA sentences used for summarizing papers (2015-2017, 20 papers each year)
- Should help students remember and understand the main content of the papers
- Questionnaire about AIDA at end of course
User Study Questions
1. AIDA Sentences: Were the AIDA sentences, as presented during the lectures and on the slides, helpful for you to understand and remember the content of the papers?
- Yes, the AIDA sentences were helpful.
- Maybe. I am not sure whether the AIDA sentences were helpful.
- No, the AIDA sentences were not helpful.
2. AIDA sentences compared to classical text summaries: Did you find the AIDA sentences, as presented during the lectures and on the slides, to be more or less useful than classical text summaries?
- I found the AIDA sentences to be more useful than classical text summaries.
- I found the AIDA sentences to be about as useful as classical text summaries.
- I found the AIDA sentences to be less useful than classical text summaries.
User Study Results
70% found AIDA sentences to be helpful;
only 1.6% found them not helpful
User Study Results
45% prefer AIDA sentences;
only 3.6% prefer classical summaries
Data: 659 AIDA Sentences
- Manual extraction study: 51 AIDA sentences
- Automatic extraction study: 189 AIDA sentences
- Alzheimer's: 62 AIDA sentences
- Open Access: 70 AIDA sentences
287 AIDA sentences
Linking and Network Study
Analyzing the effect of simple post-hoc partial formalization of AIDA sentences.
Use of DBpedia Spotlight: text annotation tool producing links to DBpedia (Wikipedia pages).
Linking and Network Study Results: Accuracy
Manual evaluation of a random sample of 10% of the resulting annotations (173 out of 1726):
Example:The treatment of Alzheimer's disease with one of the three cholinesterase inhibitors donepezil, galantamine or rivastigmine has a higher probability of at least one adverse event of anorexia before the end of the treatment as compared to a placebo treatment.
Network: AIDA Sentences and Papers
332 network components: largest covers 10%
Network: Existing Concepts Added
167 network components: largest covers 24%
Network: DBpedia Spotlight Links Added
66 network components: largest covers 48%
Results indicate that approach is promising:
- Students found AIDA sentences useful
- AIDA sentences can be automatically connected to Linked Data identifiers at good accuracy
- Linking seems to lead to a dense and broad network of scientific findings
Publishing AIDA Sentences with Provenance
Thank you for Your Attention!