EDGE OF THE RABBITHOLE

November 7th, 2020

It’s one thing to have an idea, it’s quite another animal to make that idea come to life.  Bringing any idea to life boils down to a series of problems that need to be solved - it requires poking around and searching for solutions.  The reason why so few people hunt down the solutions required to bring ideas to life is perhaps because so many attempts to find solutions lead down long and winding rabbit holes that end with no solution.  The experience of needing to start over with nothing to show for one’s effort can be incredibly demoralizing. There is a meta skill that anyone competent in their field eventually develops for problems in that field: it’s the ability to get a sense for whether or not a rabbit hole  is promising or not.

As a beginner in a new field, it is usually next to impossible to come equipped with this intuition of judging a rabbit hole of research and effort just from it’s vague appearance from the outset.

Is this the path to the correct solution?  Or is that other path better?  Which will work faster?

The quickest way to get a good answer is to simply ask someone with who is a lot more familiar with the topic.  An expert with seniority in a field is often just someone who can take a look at a problem and say “I’ve seen something like this before.”

But without an expert on hand, there are two crucial aspects to judging the promise of a rabbit hole.

One is testing as soon as possible, as often as possible and as fast as possible.  If the first step down a rabbit whole of investigation can be tested in some way, then it should be tested before going any further.

For example in classical painting, there is a technique for establishing correct proportions of the object being drawn by comparing ratios of different lengths mapped on the object and comparing those ratios to the drawing.  It’s a simple form of measurement that allows an artist to  gain a greater and better approximation of the shape and size of the object, and this starts before any paint has touched the canvas, while it’s still just a drawing.  It would be silly to wait until the entire canvas was painted in order to start measuring to see how the underlying drawing could be improved.

Something similar happens in coding.  Instead of writing hundreds of lines of code in an attempt to bring an entire idea to fruition all at once, often just a small portion of code is written and then run to see if it works as expected.  The error here is less grave than with the painting because any part of the code can be written.  Unless of course all that code was written to serve a purpose that it doesn’t end up serving even when it does work as expected.  But writing small pieces and testing as progress is made is likely to reveal that fact sooner, before all the time required to write what’s in mind has come about.

This leads to the second crucial aspect of judging rabbit holes: pivoting against the sunk-cost fallacy.

We all experience the fallacy of sunk-cost, often on a daily basis.  Getting in one grocery line because it’s shorter can end up seeking like a bad idea when it moves incredibly slowly compared to the next line over.  Do we make the switch?  Often not because of the time already sunk in the commitment to the current line.  The same thing applies to our efforts.  Spending an inordinate amount of time chasing a phantom solution down a rabbit hole seems to justify more chasing, otherwise, what was the point of all the effort so far?  Developing the skill to let that go and back away and start from scratch and try a different approach is invaluable for two reasons.  Not only is it likely that a solution will be found elsewhere, faster, but trying out a different solution often quickly turns out to be an illuminating counter-point to the first attempt.  Being able to compare the progress of two paths gives a greater sense of which might actually be more promising.  

In this vein it can even be useful to purposely try a variety of paths, but dabble in each very quickly so that an array of possible approaches can be compared and rated.

Combining this ability to pivot with the necessity to test fast and frequently creates a simple rubric: if the current step of the rabbit hole can’t be tested, consider pivoting to an entirely different approach.  Often what will happen is that some progress on a new path will reveal something about the prior rabbit hole that had seemed to grow cold, but now yields progress when the pivot is made back with a new perspective, new fact or realization that came about only by looking for answers elsewhere.

It all comes down to pivoting and testing.