The Smallest Experiment With a New Technology
Told to add video calls, the reflex is to learn WebRTC's API. The move is to ask what problem it solves, which of its concepts your problem touches, and what the smallest experiment is that proves the pieces connect for you — and to build that, not the feature, first.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You have been asked to add a capability that needs a technology you have never touched. What is the smallest experiment that tells you whether it will work for your problem, and how do you find it without first learning the whole thing?
The chat app's next requirement is "users can start a video call from a conversation". You have never used WebRTC. Its documentation is large, its vocabulary is unfamiliar — peer connection, offer, answer, ICE, STUN, TURN, signalling — and every tutorial builds a full video chat app. You have a week before anyone asks how it is going.
Learn the API. Read the peer-connection reference, follow a full video-chat tutorial, and get its demo working on your laptop. It produces a video on screen in a day, and a video on screen looks exactly like progress on the requirement.
The demo works between two tabs on one laptop and the first test between two colleagues on different networks fails, because the concept the demo never exercised — traversing NAT — is the one the chat app's users will hit every time.
- The demo works between two tabs on one laptop and the first test between two colleagues on different networks fails, because the concept the demo never exercised — traversing NAT — is the one the chat app's users will hit every time.
- The vocabulary is learned as a whole, so signalling — which is not part of WebRTC at all and is entirely the chat app's problem — is treated as a detail, when it is the part that has to be designed.
- The tutorial's architecture — a small signalling server it wrote — gets copied, and the chat app now has two realtime channels, one for messages and one for call setup, with no reason for the split.
- A day of API learning did not answer the only question that matters this week: can two of our users, on their real networks, connect at all? The demo answered "can two tabs on one machine connect", which nobody doubted.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Refuse to start with the API. Start with what problem the technology solves, in one sentence: WebRTC lets two browsers exchange media directly, without the media going through your server. Then ask which of its concepts your problem actually touches, and which it does not; the concept list is a map, and your problem is a route through part of it.
- Find the riskiest concept for your situation — the one that, if it does not work for your users, changes the design. For a chat app whose users are on home and office networks, that is connectivity across NAT, not media, not the API surface. The smallest experiment is the one that exercises that concept and nothing else (Risk-First Development).
- Design the experiment as a question with a prediction: "two browsers on different home networks, with only a public STUN server, exchange one data message directly — I predict it works for most pairs and fails for some". Build only what the question needs — a data channel, not video; a signalling step done by copy-pasting text between the two people, not a server — and run it with real users on real networks.
- Let the result decide what to learn next. If it connects, the next experiment is media and the signalling design; if it fails for some pairs, the next thing to research is a relay, and the design has a cost line it did not have before. Either way the week ends with a fact about the chat app's users, not with a demo.
The concept map, before any API
The unknowns board is the concept map with the risk attached. The known side is what the overview gave in an hour; the unknowns are the concepts the requirement touches, each turned into a question with an experiment; and the ordering of the experiments is by what would change the design. Signalling appears on the known side on purpose — it is not WebRTC, and recognising that is the first finding.
- ✓WebRTC exchanges media directly between browsers; the server is not in the media path.
- ✓Signalling — exchanging the offer, answer and candidates — is not part of WebRTC; the chat app's existing realtime channel can carry it.
- ✓Media capture from a camera and microphone is a browser API that works on the laptop; low risk.
- ~Calls are between exactly two users in V1. A group call changes the topology and the cost; written down so the design is not silently sized for it.
- ~Calls need not be recorded or moderated server-side. If that changes, direct exchange may be the wrong tool.
? Will it work for our users?
becomes Can two browsers on our users' typical networks — home, office, mobile — connect directly with only a public STUN server?
experiment A page with a data channel and copy-paste signalling, run between pairs of colleagues on different real networks; record connect or fail per pair.
? Do we need a TURN server?
becomes For the pairs that fail directly, does a relay connect them, and what would relaying media cost per call-minute?
experiment Only if the first experiment fails for some pairs: add a hosted relay to the same page and rerun the failed pairs; read the relay's pricing page.
? How does signalling fit?
becomes Can the offer, answer and candidates be sent as ordinary messages on the existing conversation channel without changing its contract?
experiment Send an offer as a message payload through the current channel between two clients and see whether it arrives intact and in order.
? What about video quality?
becomes Deferred: quality questions are meaningless until connectivity is known, and the browser's defaults are acceptable for V1.
experiment None this week; revisit once calls connect.
The experiments are ordered by what their result would change. Connectivity first, because a failure there adds a relay and a cost line; signalling second, because it decides whether there is a second server; quality last, because nothing depends on it yet.
The experiment as a slice
The experiment is a vertical slice through the technology, not through the product: it touches each concept the connectivity question needs and replaces everything else with the crudest stand-in. What it proves and what it does not are both stated, because a connected data channel is not a video call and must not be reported as one.
- PageOne button to create a peer connection and a data channel; text boxes to paste the offer and answer; a log.
- SignallingCopy-paste between two people over the existing chat — the crudest stand-in, deliberately.
- ConnectivityOne public STUN server; candidates gathered and exchanged; connection state logged.
- TransportA data channel carrying one text message — the observation that the connection is real.
One order of experiments, and the one that replaces it
The order below runs from the riskiest concept toward the feature, and reads the API only when an experiment needs it. The alternative is real: when connectivity is already known — the users are all on one corporate network, or the team has shipped calls before — the risk moves to signalling or to media, and the order starts there.
- 1Purpose and concept map from the overview
because It marks which concepts the requirement touches and which are not this technology at all, before any reading is aimed.
- 2Connectivity experiment: data channel, copy-paste signalling, real networks
because A failure here adds a relay and a cost line, which changes the feature's scope before any other work is worth doing.
- 3Relay experiment, only for the failed pairs
because It turns "some users cannot connect" into a cost the requirement's owner can decide on.
- 4Signalling over the existing channel
because It decides whether there is a second server to build, and it uses infrastructure that already exists.
- 5Media: replace the data channel with camera and microphone tracks
because Now the API reference is read for the calls the experiment needs, and the reading is aimed.
- 6Failure paths: network change mid-call, one side closing the tab
because The happy path connects; each failure is injected against something that works (Failure-First Questions).
How to do it
Most important first.
- Write the technology's purpose in one sentence and your requirement in one sentence, and check they match. If media must be recorded or moderated server-side, direct browser-to-browser exchange may not even be the right tool (What Problem Does It Solve?).
- List the technology's concepts from the documentation's overview and mark each one: touched by my problem, not touched, or not part of the technology at all (signalling). Do not read past the overview yet.
- Pick the riskiest touched concept: the one whose failure would change the design. Write the experiment as a question with a prediction (Experiment Design).
- Strip the experiment to that concept. Replace everything else with the crudest stand-in: copy-paste for signalling, a text message instead of video, a public server instead of your own.
- Run it with real conditions — different networks, real users — because the concept you chose is about conditions, and a laptop has none. Record what happened for each pair (A Prototype Answers a Question).
- Decide the next experiment from the result, and only now read the reference for the API the next experiment needs.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- Purpose: two browsers exchange media directly. Requirement: two users in a conversation see and hear each other. Match. Concepts, marked: peer connection (touched), offer/answer (touched, but mechanical), ICE candidates and STUN (touched — this is connectivity), TURN (touched only if STUN fails; a cost), media tracks (touched, but low risk — cameras work), data channel (a tool for the experiment), signalling (not WebRTC; entirely ours, and the chat app already has a realtime channel). Riskiest: connectivity across the users' NATs, because if it fails the design needs a relay server and the cost of the feature changes (NAT: Many Private Hosts Behind One Public Address).
- The experiment: a page with a button that creates a peer connection with one public STUN server and a data channel; it prints its offer as text; the other person pastes it in and prints an answer; the first pastes that in; then one types "hi". No video, no server, no framework. Prediction: works between two home networks, may fail from the office. Result across a handful of pairs: home-to-home connected; home-to-office failed for the pairs behind the office firewall. That is the fact the week was for: the feature needs a relay for some users, the relay is a cost line, and the founder can decide whether office users matter before any video code exists.
- What was deliberately not learned this week: media constraints, renegotiation, the full ICE state machine, any signalling server. Signalling was answered by observation instead: the chat app's existing realtime channel can carry the offer and answer as messages, so there is no second channel to build (WebSockets in the UI).
How you know it worked
What now exists that did not before, and what question you can now ask.
- You can state the technology's purpose and your requirement in one sentence each, and they match.
- You have a concept list with each item marked touched, untouched or not-this-technology, and you have read nothing past the overview.
- The experiment exercises one concept, ran under real conditions, and produced a fact about your users rather than a demo.
- The next thing to learn was chosen by the experiment's result.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What problem does this technology solve, in one sentence, and does my requirement have that problem?
- ?Which of its concepts does my problem actually touch — and which of the things I am worried about are not this technology at all?
- ?Which touched concept, if it fails for my users, would change the design or its cost?
- ?What is the smallest experiment that exercises that concept alone, under my users' real conditions, and what do I predict?
- ?What did the result decide, and who needs to hear it before I learn anything else?
What can go wrong
- The experiment grows: the data channel gets video "while we are at it", then a signalling server, and by Friday it is the tutorial's demo with a different name. The experiment answers one question; when it has, stop and write the answer down.
- The riskiest concept is chosen by what is hardest to learn rather than by what would change the design. The ICE state machine is hard; whether users can connect is what matters.
- The experiment is run only on the laptop, so the concept it was meant to test — conditions — is never exercised, and the result is the demo's result again.
- The result is "it failed for the office" and the response is to learn the API harder, rather than to bring the cost of a relay to whoever owns the requirement. The experiment produced a decision for someone else; deliver it.
- The week produces no visible feature, and a demo would have. If the stakeholder needs to see video to keep faith, the experiment may need a small demo alongside it — clearly labelled as not the experiment.
- Stripping the experiment to one concept means the other concepts are still unknown, and one of them may hold the next surprise. The method finds the largest risk first; it does not find all of them.
- Real conditions cost real people's time: the experiment needs colleagues on other networks, and scheduling that is slower than opening two tabs.
- "So do not learn the API." Learn it when an experiment needs it, which is soon. The point is the order: the concept map and the riskiest experiment come first, so that API learning is aimed at something.
- "The smallest experiment is the tutorial's demo." The demo exercises every concept at once under no conditions. The smallest experiment exercises one concept under real conditions; it is smaller in scope and larger in what it proves.
- "This is specific to WebRTC." WebRTC is the example because its concept list is long and one concept dominates the risk. The same move applies to a payment provider (the riskiest concept is the confirmation path, not the charge), a search engine (relevance on your data, not the query syntax) or a message broker (redelivery, not publishing).
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALPurpose, concept map, riskiest concept, one-concept experiment under real conditions applies to any unfamiliar technology attached to a requirement; what changes is which concept carries the risk, and that is found by asking what would change the design.
- TEAM-SPECIFICA solo learner runs the experiment and reads the result; on a team, the result is a decision for whoever owns the requirement — a relay cost, a scope reduction — and delivering it is part of the experiment. Under a deadline, the experiment is the estimate's foundation and skipping it is how the estimate becomes fiction.
- ILLUSTRATIVEThe video-call requirement, the handful of pairs tested and the office firewall result are invented to show the shape of the experiment; WebRTC's concepts are described at the level of its overview, not any browser's implementation.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The manifesto at /manifesto/delegating: a WebRTC wrapper library delegates the offer/answer mechanics; whether your users can connect at all stays yours, and no library will run that experiment for you.