Thoughts and Tips on Navigating Research Project Planning

Personal approaches for managing uncertainty and constrains in project planning

Having been involved in R&D projects across various fields for over a decade, I have become convinced that successful research project planning is kind of an art. It’s unlikely that any single planning framework can meet all needs. Unlike development projects, which typically start with a clear understanding of both the goal and the method—“I know what I want and I know how to do it”—research projects usually begin with a clear goal but no defined method: “I know what I want but I have no idea how to do it”.

One of the distinguishing features of research projects is the degree of uncertainty involved. In this post, I will share several practical ideas on research project planning, aiming to provide a perspective that makes navigating the uncertainties of research more manageable and less frustrating.

Bayesian Spacecraft

Any project, in essence, is a movement from point A to point B. Point A represents the starting point, encompassing all prior data about the project. This includes initial beliefs, expected hypotheses, promising preliminary experiments, relevant findings from the literature and other knowledge. Point B is the desired project outcome, a set of goals whose achievement marks project success. The space between points A and B is filled with uncertainty, which must be carefully navigated. An effective project plan addresses this uncertainty by strategically placing milestones to guide the path. Each milestone should not only answer specific research questions but also refine our understanding of the project’s trajectory.

Imagine a spacecraft trying to find its way to star B. Its initial trajectory was calculated on planet A with the best possible precision, but there are no guarantees it’s correct. However, along its journey, the spacecraft can encounter navigation stations that help adjust its trajectory. If there are enough stations and they function well, the chances of correcting the trajectory and reaching star B are high.

Proper milestones should act like those navigation stations, and we should use the “thrusters” of our project to adjust its trajectory based on new knowledge. A lack of milestones may lead to divergence from the track that leads to the goal. Too many milestones can be exhausting and time-consuming, while poorly set milestones may lead us down the wrong path.

Consider a synthetic data generation project aimed at expanding a small real dataset to train models on an augmented dataset. Some straightforward milestones would be: implementing a synthetic data generation pipeline, generating synthetic datasets, and then training and evaluating models using this augmented data. While it may be tempting to jump straight to data generation, this approach is not always advisable as it can be time-consuming and might not yield meaningful results. Adding an initial evaluation of the potential improvement in model performance by incorporating real data from the same or a closely related domain could be beneficial. If the potential improvement appears promising, proceed with synthetic data generation. This milestone can reveal whether the addition of data leads only to marginal improvements and may also provide insights into which aspects of the data should be prioritized to maximize utility. The initial choice between these two project trajectories may appear crucial for the project’s success.

Setting good milestones is not an easy task, but I find it helpful to keep this mental picture in mind while planning. It helps shift the perspective from steps we know we can take to steps we need to take to gain more knowledge and adjust the project’s trajectory.

• The rule of thumb is: the more uncertainty, the more milestones should be set.
• Prioritize milestones that challenge the project trajectory the most.
• A biased initial trajectory is normal, but we should do our best to refine it during the project.

To plan or not to plan

Long story short — always plan, but the real question is how detailed and how deep your plan should be. A project without any planning will be more like a random walk with only an accidental probability of getting to the right place. On the other hand, a too detailed plan is likely to appear useless as most of its parts will not happen in reality.

I like to consider planning like a mental beam search: starting from initial knowledge, you imagine the most probable outcomes, branch decisions for specific results, and then explore them further. Here is where the tradeoff comes into play: it is possible to grow this tree of beams as big as your imagination, but on the other hand, it is possible to start acting with an incomplete tree and update it on the fly. This creates a planning-execution tradeoff. The longer the time horizon for planning, the more uncertainty appears. It is usually tempting to over-plan, but reality is often more complex, and there is a high chance that many plan branches will become obsolete.

I once spent about a month meticulously planning experiments, building a vast tree of “ifs” and “elses”, creating tables of model architectures and hyperparameters to evaluate, along with sources of additional data and numerous hypotheses I wanted to test during the project. When the time came to execute, I encountered an unexpected result at the first step — it appeared that the initial approach did not fit the inference latency constraints, and I had to completely reconsider the project’s approach. All that detailed planning turned out to have little in common with reality.

Finding the optimal balance between plan detail and execution is closely related to the concept of milestones. Here are several references that may help you build an optimal plan:

• Getting actual data is better than building a hypothetical plan. If it takes approximately the same resources to obtain data, it’s better to get it and then build a further plan on this stronger basis.
• The longer-term your plan is, the more uncertainty there is. It makes sense to have the level of detail inversely proportional to the stage: earlier steps should be more detailed, while further steps should have fewer details.
• Avoid over-planning. It is tempting, but it is usually better to have a more general plan and do a reality check as fast as possible.
• More than three branches for a single event is usually too much.
• Try to cut the search space as much as possible.

Wild West of Estimation

Have you ever faced a situation where you thought, “Oh, I/my team will manage this task in a week” but then many complications appears, and you finished in a month at best? Do you frequently encounter situations like this? If not, congratulations! You either have a great talent for planning or haven’t encountered this issue yet, and you might not need this chapter.

People are prone to overoptimistic planning, which frequently leads to running out of time or budget. We usually hope for the best-case scenario, but life tends to be more complicated. Many unexpected situations may arise and influence our plans. This is extremely common and, to the same degree, often ignored. How often have we heard that some big project, like a movie production, the opening of a new subway station, or a rocket launch, has been delayed or its budget doubled?

I’d like to refer to an example from the book Thinking, Fast and Slow. Daniel Kahneman described the case of planning the development of a decision making course. During the preparation, all colleagues were asked to estimate the time needed to finish the course (plan exercises, write a textbook, etc.), and the answers ranged from 1.5 to 2.5 years. Then he asked another colleague if he remembered similar projects and what their durations were. It turned out that for similar projects, only about 40% managed to finish, and for those that did, it took about 7 to 10 years. Finally, the project was completed, and it took 8 years.

I personally faced this issue quite often. Once, while leading a project, we reached a final stage that was running out of time. I had planned a “perfect” scenario for this stage, designed to fit the time constraints and accomplish the project goals. The main weakness of this plan was the assumption that everything would work flawlessly. However, in reality, everything that was not under direct control went wrong — one colleague get sick, the computation server went down for a week, and another colleague needed urgent extra days off. Consequently, the initial “perfect” plan fell apart, highlighting that relying on perfect conditions is not robust approach.

So we should accept the fact that we usually rely on optimistic or even idealistic scenarios rather than the worst case. Knowing this, we should verify our plan using this lens. Here are several points to consider:

• If you can find historical data for similar projects, use it as a fair reference. It may not look as good as your estimations, but it is likely to be more reliable.
• Ask yourself how idealistic your plan is. How many events in this plan are supposed to work only in a perfect case? The more such points, the lower the total probability that they will all work together.
• Provide some room for unexpected problems and time extensions. Then multiply it by a coefficient greater than one.
• Expect to run out of time and don’t freak out about it. If you finish on time, consider it good luck.

Don’t Beat a Dead Horse

It’s no surprise that initial beliefs about the project may appear to be wrong and desired results may seem unreachable, at least at this moment. Trying to finish the project at some point may become similar to trying to ride a dead horse. At some points in the project, options for goal pivoting or even project termination should be considered. This is always a challenging task as we have already invested time and resources into the project. We are already strongly involved, and it’s tempting to think that the next idea may solve the problem and everything will be fine. In reality, that project-saving idea may never appear, or it may take years to find it (like the more than 10-year-long blue LED development). If your resources are not unlimited (which is likely the case), project termination or pivoting points should be considered.

For sure, this is one more tradeoff: continue the project with the chance of finding a solution or terminating it and spend resources in a different, potentially more promising direction. One of the most reliable approaches here is to apply time limits to your milestones: if some intermediate result is not achieved, take time to consider whether it is just a duration estimation error or a bigger problem. One compromise is pivoting. If the main goal of the project seems unreachable from a certain point, consider defining another still important goal that may be reachable from the results already gained.

Sometimes being too focused on the initial goal may lead to blindness to other opportunities that may appear during the project. So always pay attention not only to the desired results but also to all signals that come in. Some of them may lead to better solutions.

To sum up, here are several pieces of advice on termination and pivoting:

• Define time bounds for project milestones and evaluate perspectives when the time comes.
• Be fair with yourself and the project’s potential at this point; consider termination or pivoting as valid options.
• Be tolerant of possible failure. A good attitude is to treat it as doing your best with the knowledge you had at the beginning. Now you know more and can do better, update your plan and try again.
• Try to avoid tunnel vision on the initial project goal. Intermediate results may provide interesting alternatives.

Closing Thoughts

Having an elegant plan at hand makes it easy to imagine how everything will work smoothly. This is one of our common optimistic biases. It’s easy to start believing that everything is accounted for and will work like a well-oiled machine. However, while this plan is clearly in front of you, all the unexpected cases are unseen and almost impossible to anticipate. They are likely to affect the project, so be prepared for it. Embrace the unexpected challenges as integral parts of the process rather than disruptions. This mindset allows for flexibility, adaptation, and continuous improvement.

Embrace the beauty of plan’s imperfection much like calligraphers approach ensō circles. These circles, with their incomplete forms, symbolize the acceptance of flaws. However, unlike the traditional unfinished ensō circle, feel free to revisit your plan. Consider your plan to be a flexible guide which is adapted to new experience and data paving the way forward.

Thoughts and Tips on Navigating Research Project Planning was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.