Data organization and interaction

This is the 3rd and last post of a series of post about the technology behind novoseek. In the first issue we talked about the problem of synonyms, in the second we showed the challenge of dealing with homonyms, in our third issue we would like to share some thoughts behind data organization and its representation which is a common issue to any type of Web application.

I must confess, I am fond of data visualization. I love all those keynotes, or graphics that have great colors, shapes… They catch my attention despite the fact that I might not understand them or that they provide me with irrelevant information. However some of them are really amazing. When I was doing my research on bioinformatics I was desperate looking for ways to represent all the data I had on protein interactions in a way that I could get a big picture at first and then focus on the details. I found a few amazing things at Visual Complexity but not flexible enough. I must confess that I failed in my intention to apply my programming skills to this task.

When I joined Bioalma and I started promoting our first product AKS, I was really excited with one of its main features that represents the relations among concepts based on the co-occurence in the literature. Is a great piece of software that lets you see at-a-glance which concepts are more related and visualize clusters. However, the information behind it was not always understandable.

When we started the novoseek project we decide to embrace the KIS (Keep It Simple) principle. Although we  try to keep up with this philosophy, I must confess that in our meetings the development manager, marketing director and an art director, its hard to say if we are even close to this philosophy.

Regarding the novoseek interface

As you might remember from previous posts, novoseek analyzes all the literature with an algorithm that integrates database information and takes into account the context of terms to annotate them in the literature. So when we started the project and we had all the data from the analysis of all the literature, we asked ourselve “what should we do with it? How could the user take advantage of all this analysis?”. Obviously, putting it in a search engine that is simple, clear and easy to use was our best choice. We needed to start organizing the data and designing a visualization interface to interact with it.

We needed to arrange all that information in a data structure that could give a fast, efficient and scalable service. The scalability issue was a really important concern. We didn’t want to change the data model when the system needed to serve millions of simultaneous petitions.

We also needed to have a picture of what type of information we wanted to display and how the user could interact with it. Based on our experience we knew that we needed to develop something not only simple but also familiar to the end user. We knew that designing an advanced interface with lots of information would be likely to disconcert the users. Our CEO was always telling us “we need to do something that doesn’t need to be explained to use it and understand it”. And so we did.

So the indexing technology and the automatic disambiguation method enabled novoseek to search faster and more efficiently the most relevant documents. We decided to take advantage of that and build what we called Profile. This profile is the result of the analysis that novoseek does taking advantage of the results of our text-mining analysis to build a list of the most relevant concepts to the query. We thought that this list would be really helpful since it gives a quick idea of what are the main themes related to the query. As we thought this list of relevant concepts needed to be interactive, we then added some functionality to it. Whenever you click on one of the terms of the list you get all the documents that take into account the very query term and the clicked concept. You can check examples with our user cases.

After that, we added many other features, some of which are really handy! Others may be a bit more hidden for advanced users that want to make the most out of the system.

However, understanding the users, the way they interact with us, what is useful and what can be removed to keep up with the KIS philosophy is an endless and ongoing process. At Bioalma, we are always studying what would happen if we put this menu here, if we choose this color or if we set up this log-in box there. Indeed, we mix our own craziness with the user suggestions and it is clear that sometimes we come up with a different (or strange) interface. So stay tuned and find out soon the results of our conversation with users and our own schizophrenia.

We recommend you to read:

• The importance of context in text disambiguation
• Your handy my novoseek guide v1.0
• Why Publication Types matter when looking for relevant scientific literature in Pubmed
• Tip #3 to improve searches in novoseek – use labels to organize your readings
• Tip #2 to improve searches in novoseek – use Boolean operators