A research project coordinated by Alexandru Ioan Cuza University advances the idea of a new technology able to understand by itself the content of a book and to offer the readers suplimentary context information. „MappingBooks” Project is led by Professor Dan Cristea (Ph.D.) from the Computer Science Faculty and its first result/product – an 8th grade Geography textbook – is close to being launched.
The idea of an intelligent book started from an Orhan Pamuk novel, a Nobel Laureate writer. On his way to attending a conference in Istanbul, Professor Cristea also took in his luggage the novel „The Museum of Innocence”, the fiction story about a museum that displays objects bearing witness to a love story from the 80’s Istanbul.
One of his Turkish peers told him that Pamuk actually inaugurated a real museum in Istanbul, with the artefacts mentioned in the novel. At the entrance, the lady selling tickets asked Professor Cristea for the book, instead of money. She opened it at page 613, at the very part mentioning the Museum of Innocence and where a ticket was drawn. There, on the picture, she applied the museum access stamp.
Professor Dan Cristea
This crossroads between reality and fiction gave Professor Cristea the first idea about designing a „smart book”. An intelligent book would have alerted him to the possibility of visiting the Istanbul Museum of Innocence, even take the tour of the 80’s metropolis, as described by Orhan Pamuk in his book. The intelligent book could have signalled potential meetings with the author or could have put him in contact with other Pamuk fan readers from the surroundings. The concept could be developed furthermore: the intelligent book would be able to understand the relations between the characters and the plot, to create by itself genealogical trees, abstracts, timelines and character relations maps.
The first smart book, the result of a 4-year research and code writing, is the 8th grade Geography textbook and will be available to students by the end of the school year. The textbook will function as a mobile app for devices connected to the Internet and GPS services. On the mobile device, the book could be read as a regular textbook, but will also have other functions. Each mention of a geographical location is automatically recognised and the student is informed about the possibility to find out supplementary information. Moreover, each time a student passes by one of the hundreds locations mentioned in the textbook, he is made aware that the objective figures in the Geography textbook and is reminded relevant information about it.
How the smart book works. The app has to be downloaded on the user’s tablet or mobile phone. Afterwards, it will automatically generate intelligent maps of the book content, based on the understanding of the text by the machine. Depending on the reader’s location and calendar, the application recommends visiting the sights mentioned in the book, signals the places the characters have been to, informs about events connected to the reading and allows communication with others readers.
“The functionalities of the first intelligent text of this kind are few, compared to the rest we have in mind”, Dan Cristea said. “We plan to create much more sophisticated applications. For instance, tourist guides constantly updated with events taking place in town and to suggest activities according to the location. It can be applied to non-fiction books that notify you when you pass by geographic locations, buildings or institutions mentioned in the book. But the most difficult challenge remains the understanding of fiction texts, the relations between the characters, the storyline. How can you determine artificial intelligence to understand the sudden translation to the perspective of another character or a diary entry of recounting a memory? A human being is able to decipher them, as easy as it breaths. Well, for a machine is much more difficult.”
How to train artificial intelligence to understand people
The answer to the question “How do you make a computer program to understand a text?” derives mainly from the hope that it will learn by himself. “Automatic learning”, an entire chapter of Computer Science, plays an important part in understanding language. Basically, the program is given the texts in which things to be understood are tagged or marked in a certain manner. Using tags, the program needs to „figure it out”, to derive rules, then to apply them on new texts. It works probabilistically, meaning that sometimes, the understanding is not adequate, but when it gets feedback and the problem is fixed, the program increases its performance.
One of the first fiction texts used as trial was Henryk Sienkiewicz’s historic novel “Quo Vadis”, whose annotation took two years. “I worked with my students, fragmenting the text into several parts. They indicated to the program where the emotional and family-related relations occur, then the separate parts had to be connected and corrected – both automatically and manually. Now, we hope to be able to use our results in order to provide the program with learning material.” Thus, after gathering several such examples, the intelligent book would be capable to understand independently if the characters have a cordial relationship or, on the contrary, a turbulent one, if they are kindred and what are those relations.
The relations in ”Quo Vadis” have been tagged manually by the students and professors, in order to train the program to recognise them subsequently. In the image can be noticed: the map of the emotional relations between characters, the map of the main characters’ relationships, Vicinius; the statistics of the relations of Vicinius, the main character.
A similar approach exists when it comes to recognising the temporal development of a story, although the issue is even more complicated than that. For the recognition of temporal plans, Professor Dan Cristea works with master students and Ph.D. student Andreea Macovei on a different novel, ”Map of the invisible world”, by Tash Aw. The same as in the ”Quo Vadis” case, the objective consists in indicating the programme the transitory moment from a temporary dimension to another, to be capable of understanding by itself.
“We have found inspiration from the rail tracks”, Professor Cristea explained. ”In a train station, there are several rail tracks that combine, get separated, combine with others… it’s a tangle. Just like people’s destinies. If a rail track would be a person’s fate, presented in a book, he meets with another character, spend some time together, then they separate and meet other people. We’ve named them time tracks and our aim is to detect them separately.”
Books that suggest to the publisher how they should look like
In the future, the research team from the University hopes that the intelligent book applications will also be attractive to the publishing industry. In a new research project proposition, Professor Dan Cristea, together with researchers from Brunno Kessler Foundation from Trento and Vrije Universitaet Amsterdam, computer scientists from ArtSoft Consult Cluj-Napoca and a group of publishers from Italy and Hungary advances the idea of tool development for book publishers. Starting from text understanding, programmes would be capable to suggest the publisher digital artefacts meant to generate interaction: event references or real places, timelines, character interrelation maps, family trees and so on. These specific applications are suitable especially to textbooks and tour guides, but will also be used for fiction books.
The declared purpose of the book re-configuration project is to bring reading in children’s world. ”Children are so obsessed with electronic gadgets, they have no idea how many satisfactions they can find holding a book in their hands, reading it. We thought that by exploiting young people’s passion for these devices, adapting books to them, making the text interactive, we could revive the interest of the young generations for reading, a universe that fades away slowly”, Dan Cristea explained. The intelligent book would also provide, in its final version, the possibility to create connections between its readers, to generate social groups for readers to meet and discuss about the content – a Facebook where the only acceptable posts are comments on books.
„MappingBooks – Let’s jump in the book!” project is financed by the Ministry of National Education. The Alexandru Ioan Cuza University team of project is composed of Professor Dan Cristea, Ph.D., Lecturer Ionuț Pistol Ph.D., scientific researcher Daniela Gifu and scientific researcher Lucian Gâdioi from the Faculty of Computer Science, Lecturer Mihai Niculiță Ph.D. from the Faculty of Geography and Geology and Dorina Ticu.
