Discovery Tools for Unknown Problems: Building Lenses for Insight

When you are staring at an unknown problem, the goal isn’t to build a finished structure; it’s to build a “lens” that lets you see the truth of the problem clearly.

Depending on whether your “unknown” is in the data, the logic, or the environment, your choice of tool changes the way you gain that insight.

1. The Jupyter Notebook (Data-First Insight)

If your unknown problem is buried in a mountain of information—patterns you can’t see or distributions you don’t understand—the Notebook is your laboratory.

The Approach: Highly iterative, non-linear execution. You run a cell, visualize a plot, and then go back up two cells to tweak a filter.

Best for: Finding the “shape” of a problem. If you don’t know if the data is clean or if the correlation even exists, you use this to get visual feedback.

The Trap: It’s easy to create “hidden state” (running cells out of order), which can lead to false insights. As Hamming would warn, you might get a “number” that is actually a ghost in the machine.

2. Exploratory Developer Tests (Constraint-First Insight)

This is essentially “TDD for Discovery.” You write a test not to verify a solution, but to poke the system and see how it reacts.

The Approach: You write a test for a behavior you assume exists. When it fails (or behaves strangely), the error message becomes your data point.

Best for: Exploring API boundaries or complex logic trees. It forces you to define the “What” before you know the “How.”

Insight Style: It provides binary feedback. The system either behaves as you hypothesized or it doesn’t.

3. The REPL (Language-First Insight)

Using a Read-Eval-Print Loop (like ipython, node, or irb) is like having a conversation with the computer.

The Approach: Immediate, ephemeral experimentation. You test small syntax ideas or library methods in real-time.

Best for: Lower-level technical unknowns. “How does this library handle nulls?” or “What is the exact structure of this nested dictionary?”

Insight Style: Tactile feedback. It’s the fastest way to build “muscle memory” for a specific technical domain.

4. Prototyping in a Scratch File (Architecture-First Insight)

Sometimes called a “Spike,” this involves writing a single, throwaway script that tries to solve the problem from end to end as messily as possible.

The Approach: Ignore all best practices. No error handling, no logging, no tests. Just get the data from point A to point B.

Best for: Investigating “The Unknown Unknowns.” You often find that the biggest hurdle isn’t the logic you were worried about, but a permission error or a missing dependency you hadn’t even considered.

5. The “Pencil and Paper” / Whiteboard (Logic-First Insight)

Hamming also emphasized that thinking is the most important part of computing. Sometimes, the computer is a distraction.

The Approach: Sketching the flow, the state transitions, or the data relationships before touching a keyboard.

Best for: High-level system design and identifying circular dependencies.

Insight Style: Conceptual feedback. It stops you from over-engineering a solution to the wrong problem.

Comparison of Discovery Styles

Tool	Feedback Loop	Primary “Insight” Gained
Jupyter	Visual	Patterns, Trends, and Outliers
Tests	Assertive	Boundaries and Edge Cases
REPL	Conversational	Syntax and Library Behavior
Scratch File	Flow-based	Integration and Feasibility
Whiteboard	Abstract	Structure and Logic

Summary for the Hamming-Minded

If the purpose is insight, you must choose the tool that reduces the latency between a question and an answer.

• If you’re asking “Is this data junk?” → Notebook.
• If you’re asking “Does the system handle this edge case?” → Exploratory Tests.
• If you’re asking “How does the whole thing hang together?” → Scratch File.

The tool shapes the lens, and the lens shapes what you can see.