The secret to outstanding research is to identify your research goals and work towards them. The more experienced researchers might find natural identifying them, but novices usually struggle to come up with strong research objectives. Well-defined objectives will help you to remain on-topic and use your time more efficiently.
Upon starting a new research project, it is very important to spend some time to understand which are the objectives to reach and what you need to achieve them. In this article I introduce a simple procedure to define your goals in terms of research questions.
These questions will help you to take educated decisions such as when identifying relevant articles for reviewing the literature. I also introduce 5 simple criteria to organise by importance multiple research-related aspects so that you use the right resources to cite in your articles. I also introduce Academic Scout: a tool that will coordinate your research efforts.
1. Understanding what is relevant
Any time you approach a new research task, you should first spend some time to understand what you are hoping to accomplish. Being conscious of it can help you:
- Decide in which area to work or specialise
- Explain why your research is important
- Find important journals to read and publish in
- Identify the most influential top researcher for collaborations
Even though it is so important, this skill is rarely taught in academia. Researchers usually acquire it in a trial and error process by reading a lot of literature, leading to much wasted effort. A more efficient way, is to think about your work in terms of research questions.
These questions describe what you are trying to answer through your research. Being aware of them, allows you to skip most of the guesswork that you would have to do instead. Great, but how can you easily identify them?
Often people distinguish between the Big Question and Specific Questions. The big question coincides with the goal of your broad topic, most likely the topic that interests you the most. The specific questions (or question) address a practical problem or a research problem (a class of practical problems) within a specific niche of the broad topic.
From Diagrams To Research Questions
Identifying your research question is a process that takes time. You may be given a topic by your supervisor or you may come up with one for which you are really passionate on your own. For example, let’s say that your topic is Machine Learning. This topic is too broad to write about, so you would need to refine it.
Researchers often use graph diagrams to explore the available research questions. They usually start by writing the main topic on a piece of paper, then they add the subtopics they are aware of. The more they have read and are knowledgeable on the subject, the more precise will be the diagram. If you don’t know from where to start, start by familiarising with the topic by searching it online. Simple Google searches and Wikipedia articles can help to quickly understand the domain and find valuable keywords. Add them to your diagram as you discover them.
When you feel comfortable enough with the topic, pick an aspect in which you are particularly interested and think of an open-ended question about something you’re genuinely curious about. For example, you might be interested in Unsupervised Learning. Then, a possible research question could be: how effective is Unsupervised Learning with respect to precision in Machine Learning?
Congratulations! You’ve just come up to your first research question!
With this question in mind, ask yourself what you would need to answer it. Think of a particular dataset, some reference results, state-of-the-art approaches, etc. These are the resources that you should care about and where you should put your research efforts.
Start to read papers and any time you find an article that contributes an idea, method, benchmark, or dataset that you could use to answer your research question, you will know that it is relevant for your research.
Re-Evaluating your Research Questions
Keep in mind that the amount of information available for a specific question with respect to your research time frame might suggest to reconsider your research questions. This is especially true for 1st year PhD students as they are literally trying to figure out their research plan. Generally speaking, you should re-evaluate your research questions from time to time to make sure that they are still valid goals for you.
For instance, you might realise that you find too many resources to be relevant to your research. This likely means that your specific research question is still too broad. The most common strategy to narrow down your question is to include further constraints. Get out your domain diagram again and for each node think of ways in which you could make the underlying problem more specific. Possible aspects to consider are, for instance:
- temporal (i.e.: in the last year),
- geographical (in London),
- demographic (age, occupation, ethnicity, etc.)
- purpose (for Autonomous Driving, Image Reconstruction, etc.)
See if a combination of constraints make sense together and do a quick search online: do you still find enough material to back up your research but not as much as before? If so, update your question with these constraints and proceed. If not, keep checking other combinations or adding constraints.
Sometime you might end up with the very opposite problem: no matter what you do, you can’t find any relevant information to lead your research. The domain diagram is again your friend: you could try another combination of constraints, or to relax some constraints (i.e.: relax London to United Kingdom, remove the temporal constraint, etc., or even navigate the diagram one node up and pick another question. In any case, make sure that you can find enough information for your updated question.
Remember that you are the only one to decide if a research question is valid or a resource relevant. If you still feel stuck, seek the advice of your research supervisor or discuss your problem with your peers.
2. Ranking your findings
Now is the time to look back to the papers that you selected as relevant. You likely have slightly different versions of the same research or research that has likely been superseded by more recent work, or articles that are more authoritative than others. In other words, how do separate the good from the CRAAP? CRAAP is an acronym used to describe how to evaluate your findings. It stands for:
- Currency: Check when the paper has been published or updated to ascertain if it is recent enough. Published results might have been superseded by more recent works. As a rule of thumb, the more recent are the papers, the better. Although the trade-off is that recently released papers won’t have many citations from which you can determine quality. Tools like Academic Scout can extract dates from papers and do this check for you.
- Reliability: Check if the paper is backed by facts or if it contains opinions. In other words, check if it includes enough citations. The more citations, the more likely that the authors have a good understanding of the domain, so it is easier to verify the paper and trust the content. Again, Academic Scout can compute for you the citation per page ratio and verify the quality of the references.
- Authority: Check who are the authors of the papers. Are they well respected names in the field? If you’re new to the field or you don’t know them, how many papers have they published? What’s their H-index number? The higher these numbers, the more likely the author is influential. Academic Scout conveniently reports these numbers to you for evaluation.
- Accuracy: Are the results backed by data? Does the paper include or link to any dataset needed to compute the results? Is the methodology to compute the results described in enough details to be reproducible? Are the results clearly organised and discussed? The more the answers to these questions is yes, the more likely the paper is good.
- Purpose: Does the authors have a second purpose? Are they expressing a biased opinion? Ask yourself if they are trying to sell a product or an idea.
Keep CRAAP in mind when evaluating a research paper to determine whether it really is worth your time and your citation. As you become more experienced as a researcher, you’ll get better at making these decisions more quickly. By evaluating your reading list in this way, you’ll make the most of your time by focusing on only the most important papers that can truly influence your research.
It takes a long journey to become a successful, focussed researcher. Using research questions to identify relevant content for your research and the CRAAP criteria to prioritise it, makes it less stressful and more resilient. The adoption of a tool like Academic Scout also makes it more productive!