10 Comments
User's avatar
Yetvart Artinyan's avatar

Great thoughts, and thank you for openly sharing them, Stephen.

Your point that idea generation is becoming abundant seems to shift the real bottleneck somewhere else: from producing possibilities to deciding which possibilities deserve scarce attention, time, and capital.

This becomes even more difficult when abundance is collective, coordinated or not. Individuals can generate more ideas with AI, but so can teams, departments, and entire organizations. The number of possibilities can therefore grow much faster than our capacity to test them.

That makes me wonder about the logical status of judgment at this stage.

AI may help us compare, structure, challenge, or rank ideas. But AI itself is an abstraction of the world rather than direct contact with the world. Sometimes that abstraction may be extremely useful; sometimes it may be misleading precisely because the relevant future has not happened yet.

So when several ideas are internally coherent and none has yet encountered reality, on what basis can we legitimately prefer one over another?

Is there a logically defensible way to judge which novel idea deserves to survive before any of them has been exposed to the real world, or can AI at best help us decide which uncertainty to test first while remaining fundamentally unable to tell us which idea will actually survive contact with reality across desirability, feasibility, and viability (let's call this survivability)?

And if survivability can only emerge through contact with reality, does the scarcity created by AI-generated abundance shift from “good judgment” to something even more specific: the ability to design the smallest real-world encounter that can discriminate between competing possibilities before too many resources are committed?

Stephen Pao's avatar

Hi Yetvart, good point! I think AI can suggest a small real-world encounter. What it can't do is know in advance whether that particular test will actually discriminate for this particular idea or read an ambiguous result once it comes back. The suggestion is still just another idea until it's run.

We ran into this at Barracuda. We had a prototype load balancer built on open source that we were debating on whether to productize. Instead of debating it internally, we ran Google Ads to see who clicked through. This was our small, real-world encounter.

Still, the test itself wasn't the hard part. The hard part was extracting and reading the thin but real signals of some gaps that F5 and Citrix weren't addressing. I'm not sure that AI could have done that on its own.

Yetvart Artinyan's avatar

Thank you for sharing this example, Stephen. What first made you think that customers were not being served well enough by F5 and Citrix, and that the problem was important enough to start building a prototype? Once people clicked on the ads, how did you determine what they were trying to achieve and whether the problem mattered enough for them to switch or pay? What made those thin signals difficult to interpret, and what ultimately convinced you that they reflected more than curiosity?

By asking these questions, I am essentially doing a mini user research and trying to understand causality, much as I suspect you did in this case. AI may assist, but interpreting ambiguous information and judging what it means and what to do next remain fundamentally human.

Stephen Pao's avatar

Honestly, the prototype was kind of an accident. An engineer built it for our own internal infrastructure. We didn't want to pay for an F5 or Citrix box for what we needed. I'd used F5 at two other startups, so I knew the shape of the problem. But that's really all the conviction we had going in. Everything else was against us. No data center expertise in-house. No channel selling into that space. And a brand known for spam filters, not traffic management. Not exactly a no-brainer.

So we went looking for real signal instead of just assuming it was there. We ran Google AdWords into landing pages, but the early messaging was mostly guessing. We tried a bunch of use-case framings before a couple actually got people talking to us instead of just clicking. That's really where I'd push back on the idea that AI could've figured this out. A click just means someone was curious about a headline. We had to iterate the message itself before we got anything worth calling signal.

What we found wasn't anything like traditional data center stuff. Things like keeping a site up during a software upgrade by failing traffic over to a backup server. Or load-balancing Windows terminal services for a small company. Weird, narrow use cases. Not what our brand had any business winning, but real pain nonetheless.

The harder part was telling curiosity apart from real intent. What solved that for us wasn't analysis, it was structure. Our resellers ran "free" 30-day evals, but they almost always got a credit card or PO upfront, invoiced at the end. Nothing was technically required, but once a reseller was in the loop, it felt more like a paid beta with a money-back guarantee. That little bit of friction is what separated mildly interested from willing to commit. Some came back. A lot stuck.

Even after we had real customers, there was constant back and forth on what to build. The big platforms had persistence models, deep reporting, stuff our use cases just didn't need. Figuring out what to cut without breaking what made it useful was its own kind of friction. That came from going back and forth with actual users, not from any data set.

So no, I don't think that gets shortcut by AI. The click told us basically nothing. What actually told us something was building just enough real cost into the funnel to separate curiosity from intent, and then sitting in the ambiguity long enough to read it right.

Yetvart Artinyan's avatar

This resonates with many technology inventions I have seen myself. Thank you for unpacking the case further, Stephen.

An engineer encounters a problem and responds by building something because building is what engineers are trained to do. The evidence may initially be no more than one internal experience and the assumption that others probably struggle with the same thing. In your case, however, that experience already contained a meaningful switching decision: you knew F5, found the established solutions too expensive and excessive for your situation, and chose an internal workaround.

From a Jobs to Be Done perspective, the opportunity was therefore not “a market for another load balancer.” It concerned the progress people needed to make in particular circumstances. Keeping a website available during an upgrade and balancing Windows terminal services are different use cases, but perhaps they reflect the same underlying job: maintaining sufficient availability without the cost, expertise, and complexity of enterprise infrastructure.

The Google Ads helped identify which descriptions attracted attention (correlation), but a click could not explain why someone was looking. That required understanding what had happened before the search, what people were doing instead, why that became inadequate, what concerned them about changing, and what finally made action necessary (causation). The conversations revealed the forces behind the possible switch (switching and paying forces); the credit card or purchase order showed that some people were prepared to act.

The reseller arrangement adds the business model innovation perspective. It did not merely measure willingness to buy. It may have created the conditions that made buying possible. The reseller offered credibility to an unfamiliar brand (channel), translated the technology into the customer’s situation (communication), provided a familiar purchasing route (ecosystem/distribution), and reduced the perceived risk through the 30-day evaluation. At the same time, the reseller had its own Job to Be Done: finding a product it could confidently recommend, support, and sell profitably.

This is why I argue that you were not developing a product. You were developing a survivable business model in which the product was only one component. The customer’s job, the reseller’s job, product scope, channel, trust, payment structure, risk allocation, price, and delivery costs all had to reinforce one another. A technically useful product could still have failed if the reseller had no reason to sell it, customers did not trust the brand, procurement was too difficult, or the economics could not support those narrow use cases.

Your decisions about what not to build were therefore more than product prioritization. Adding persistence models, deep reporting, and other enterprise features could have increased development costs, support requirements, sales complexity, and price until the offer recreated the same problem that caused customers to avoid F5 and Citrix. Removing features protected not only the product’s simplicity but the survivability of the entire business model.

AI could help generate messages, structure interviews, and identify patterns. Synthetic interviews could produce plausible hypotheses. But neither can create independent evidence of a real switching decision. That evidence came from people experiencing constraints, considering alternatives, accepting risk, committing money, and continuing to use the product.

This is why I advocate conducting user research before building anything intended to become a product. The cost of learning is much lower at the beginning than farther down the path, when code, capital, internal identity, and commercial promises have already accumulated. I understand why meeting potential users without a solution can feel uncomfortable. There is nothing tangible to present, and the conversation can feel awkward without a prototype to anchor it.

Yet the purpose is not to present a solution. It is to understand what people are trying to get done, where they struggle, what they currently use instead, why those alternatives are inadequate, what triggers them to search, and what happens if they do nothing. This provides a much stronger basis for deciding whether anything should be built, which capabilities matter, and which should be excluded.

It also helps distinguish whether you are entering an established market with another competing offer or addressing people who remain underserved because the existing options demand too much money, expertise, or complexity. Your case seems closer to the latter: the innovation lowered the threshold for customers who needed reliable traffic management but could not justify an enterprise solution.

Looking back, did Barracuda discover one common Job to Be Done beneath those narrow use cases, or did the survivable business model emerge because the reseller channel allowed you to serve several small but valuable jobs that F5 and Citrix could not serve economically?

Stephen Pao's avatar

There was one underlying job, but a lower-stakes version of it. The mission-critical version, keeping revenue flowing 24x7x365 or delivering VDI reliably across a global enterprise, is what actually justified a pair of F5’s. Our customers had a related but genuinely smaller version of that same job. They needed availability, but not at a level that warranted that price or that complexity. F5 and Citrix weren’t ignoring a different job. It was a deliberate choice to leave that end of it alone, since their go-to-market model, their revenue and margin expectations, and the product itself were all built for the high-stakes end.

That’s also why the business model mattered as much as the job itself. We could serve that lower-stakes version profitably because we already had the admin UI, logging, and monitoring built for other products, so the incremental R&D was low. We had brand and channel to reach those customers without building distribution from scratch. And it never needed to be a big number. This was never a $50 million a year line for us, and that was fine, because the company was small enough that a modest product line wasn’t a distraction. F5 and Citrix couldn’t have shaped a business that way even if they’d wanted to. Their model was built for something else entirely.

Yetvart Artinyan's avatar

Thank you for sharing this very insightful real-world case.