Why software-based products are so difficult

Creating a good digital product involves understanding the very nature of software.

Despite being its primary material, it is often ignored, if not misunderstood.

Software is not wood, nor is its treatment the way we have historically developed around architecture and other disciplines with which we operate in the non-digital world.

It often seems that this is what we would like to happen, but reality is stubborn and always ends up making us realize that this framework doesn't fit.

Companies whose business is the digital product itself often suffer from this poor understanding of software and too often seem to struggle against its nature.

From this perspective, many expectations are misguided, producing organizations that are less productive and professional than they could be.

Daily empirical experience is the best antidote to avoiding this misunderstanding.

Let's explore some examples that I believe will be evident to anyone even slightly familiar with this industry.

One input, many outputs --------------------------- Software obliterates our aspirations for complete control over the final result.

Our brains have not yet fully understood that something we project may not materialize in a practically identical way to the model.

Companies that invent digital products constantly face this challenge, and the anecdotal collection of amusing situations, despite the dramatic background against which they often occur, is truly abundant and rich: designs that pursue a precise color and end up with different modulations depending on the monitor and configuration; visual arrangements that pursue a certain effect and end up with a different result or simply appear broken depending on a certain resolution, a certain type of device, or a version of a certain program within a certain type of device; interaction flows involving data and communications that never reach their recipients either because they depend on something they must activate, or because, given certain environmental conditions—network, location, time of day—they never take place; designed experiences, where image, movement, and sound may intervene, that don't have the expected outcome or only partially do so, because something isn't right in the system, that is, in the set of conditions that must be met.

The brain trained on beautiful assembly lines where the fitting of elements at the input ultimately produces the intended output—from a car to a hamburger—experiences its first crisis here.

And people cannot help but have to respond to this collapse: sometimes they simply escape—reality is wrong or not yet good enough—and other times they carry the burden of living poorly with something that always produces dissatisfaction.

Black boxes ------------ What you don't see in software is more than what you see.

Digital products bring us closer to what we usually experience with our own bodies: things happen inside that we cannot understand by focusing exclusively on our epidermis.

Doesn't it happen that a clinical analysis—product metrics—gives you correct values, but when you undergo surgery (they open you up) they discover something different based on information that has been made visible?

This black-box nature is something you may frequently experience.

For example, errors in digital products are completely normalized.

So is the inability to satisfactorily explain why they occur.

Sometimes superficial explanations are found, but it's very common to abandon the aspiration to find the root cause, not only because the effort wouldn't be worth it, but also because there's a looming suspicion of something multifactorial that couldn't even be properly tracked in an opaque system.

Inventing with software is accepting that the unexpected is ordinary.

Dynamic Information -------------------- Working with software means accepting that you are working with changing pieces.

Information in digital products is inherently dynamic.

Unlike other types of products, where information is often static once printed or recorded, in software the data and structures are in constant flux.

Starting development is often also the moment you discover that something wasn't quite right; that something changed at some point without you even considering it; or that something just stopped working.

Teams experience this phenomenon daily, but so do clients, and it's often difficult to determine which of the many layers involved is the cause.

Branching -------------- Software flutter can have unintended consequences.

Branching is another unique aspect of software that often surprises those unfamiliar with its nature.

A seemingly simple change can trigger a cluster of unexpected effects elsewhere in the system.

This is due to the complex interconnection of components and the dynamic nature of the information handled by the software.

Sometimes flapping has huge consequences, as in the recent case of Microsoft and CrowdStrike.

In practice, it carries so much weight, often psychologically, that the usual expressions when incorporating something new into a product are cautious, such as "in principle, it shouldn't affect anything." The proliferation of methods for progressive rollouts—by percentage, cohort, or directly by feature flag—is another good confirmation of how internalized this phenomenon is, since the focus is not on how to prevent it from happening but on deploying a safety net, knowing that it will happen to some degree.

Plasticity ----------- Working with software tends to be a creative process without established contours.

When the only limit is your imagination, it's easy to be confused by exuberance.

Digital products contain a certain promise of an abundance economy that tends to scorn what the traditional economy, an economy of scarcity, placed in value.

We know that these types of products don't suspend the logics we already knew, so it's possibly more a problem of perception than of real change.

The malleability of software is a fact, starting with its own general purpose.

This plasticity is something you can often verify, especially when a functionality is conceived with a focus on assembling the integral set of its parts for execution, and the parts themselves become new possibilities that generate others, and these others, and these others, in an open sequence.

In the industry, it's common to use expressions like "design to the maximum" to refer to these types of situations.

A bit of history ------------------- The history of software is young.

Even more so is that of the internet-oriented digital product.

Even so, it offers some relevant references that help us see that this singular nature of software was indeed detected from the very beginning, and that different approaches have been formulated to see how to manage it.

First, we know that since the 1930s, approaches called Iterative and Incremental Development began to be formulated.

They originated with a few engineers dedicated to quality, who began to show concern about certain situations and proposed different solutions.

These types of iterative and incremental approaches were already being applied in projects at such significant organizations as NASA and IBM in the 1950s and 1960s.

We also know that by the 1970s the debate had already been consolidated between this type of methodologies and those that ended up grouped under the label of "cascading developments", which were considered the standard way, since they imported management styles that were already known previously and that had become popular with the revolution of industrial organizations and project management.

Article by Royce expounding on and pointing out the limitations of the waterfall method based on their own experiences.

Objectives in management used to be about meeting deadlines and costs, and in many ways this remains a dominant perspective even today.

Companies also tend to promote the passing of the baton as if it were assembly lines that, instead of joining screws and bolts, join pieces of software.

As these approaches became more established, the trend that sought new ways of programming and organizing teams around programming gained ground.

In the 1990s, introductions to the movement known as Link Placeholder began to be published.

This was the moment when the field of programming began to merge with a vocabulary that until then had remained on the margins, or had simply not existed: the evaluation of success no longer relied on meeting deadlines and costs, but rather on customer satisfaction.

Finally, towards the decade that inaugurated the year 2000, the famous Manifesto for Agile Software Development was published, signed by relevant programmers.

We know, therefore, that the different approaches and ways of working with the software were born from the same people who knew it best.

By now, time and cost have completely disappeared as primary objectives, and in their place, collaboration with customers and the ability to respond to change, among others, have taken center stage.

In just 70 years, the way we look at software development has changed dramatically.

Today, we already use things like atomic development, slicing techniques, continuous delivery, iterative implementations, progressive rollouts, and, in general, a toolbox that stays alive through constant input, as common currency, especially when we focus on tactics surrounding digital product development.

Blurry Customers and Complex Problems --------------------------------------- This shift toward customers, even placing them as collaborators, added a new layer of complexity to the software layer we've focused on so far.

First, because another unknown was added: how do we get to know who those customers are?

The more capacity and relevance digital products have gained, the more users and customers use them.

Currently, you can say without fear of exaggeration that the magnitude of some products has the same impact as non-digital realities as important as, for example, traffic management.

Activities like user research emerged as a way to unravel this mystery, while simultaneously incorporating analytical capabilities to quantify behavior through interactions.

Through a set of attributes—clicks, visits, funnels, sessions, etc.—conversations, surveys, user testing exercises, and other techniques from a toolbox that has grown over the years, we try to satisfy that oft-repeated aspiration of putting people at the center, and of providing them with a certain certainty that they are not directly our own invention.

It's not uncommon for everything to end up as an illusion: companies acting as ventriloquists, putting their demands into the mouths of these unknown customers.

Furthermore, with the opening to customers came a reformulation of problems, which went from being considered projects consisting of a set of requirements and a sequence of tasks, to facing what social scientists usually call complex problems, but which from the perspective of design thinking have also been verbalized as wicked, twisted problems or weakly defined (wicked problems).

A type of challenge that lacks clear solutions or boundaries; it doesn't have a definitive answer or is usually formulated in a classic way, and its possible solutions aren't enumerable.

[](https://world.hey.com/inigo/349a3f76/blobs/BAh7BkkiC19yYWlscwY6BkVUewdJIglkYXRhBjsAVGwrBw lo33BJIghwdXIGOwBUSSIMYmxvYl9pZAY7AEY=--26fc90e5a82832c19832f4e336c54f462319667d/13.png?disposition=attachment "Download 13.png") Buchanan's article where he reflects on the importance of weakly defined problems in design thinking.

After all, this is the terrain in which many of the things we try to solve operate, and in which the social sciences are so involved (how can we solve the aging population?

How can we reduce school failure?

How can we reduce the environmental impact?).

Although in products the questions are usually more modest (how can we get customers to keep their subscriptions longer?

How could we provide a better experience?), they share that open-ended nature: both because there is not always agreement that there really is a problem—it is rare for there to be a fact so obvious that it generates total consensus—and because the goal is never to completely eliminate it, but rather to move with relative indicators (good progress is getting 5% more customers who renew their subscriptions).

So, well into the 20th century, digital products have not only faced numerous challenges related to their primary source of input: software, but, in their consolidation as a significant part of economic activity, they have also incorporated other challenges that bring them closer to many other human activities in which we orient ourselves through conjecture, scratch a few obvious facts to draw conclusions, and deal with risk and uncertainty.

Some new methodologies, such as that of Jobs To Be Done, aim to be a way to reduce these new challenges, incorporating real context and concrete situations as a way to minimize this lack of definition of boundaries.

It is easy to see that the digital product has grown up, although the organizations that invent them often do not keep up with the same pace of maturity and continue to treat it like that child who was merely a technical resource.

There is much room for improvement in organizations that focus on the bottom line or the business model, but that treat the product, which is the source of those finances, either through requests or through strong hunches dumped into a backlog.

A narrow-scope incident cannot be managed the same as a strategic decision, just as the arrangement of a seat on a train is not treated the same as the redesign of the rail network itself.

In many companies, seats and rail networks are confused, and there is a tendency to try to resolve both in the same way: everything is valued solely as the amount of work completed without seeking correlation with expected results.

More productive and professional organizations ---------------------------------------------- The stronger a company's relationship with the invention of a digital product, or what amounts to the same thing, the more its business is the product itself, the more it is affected by these aspects that we have just gone over.

When it does not incorporate this reflection, it often struggles because it works within the wrong framework.

Therefore, neither does the organization provide the appropriate way of thinking, nor does it offer its people the environment in which they could be most productive.

First, these types of companies are uncomfortable inventing from high-fidelity projections.

If Figma is the starting point and someone validates it, be it the CEO—something even more common than is often thought—or any other management figure, the risk of sliding down the slope of seemingly endless developments is very high.

Equally, there is the risk of lagging behind (sometimes the discussion is held hostage in Figma itself).

Likewise, a lot of frustration and loss of talent will have to be noted on the debit side of their teams, since that final result, a projection of that design at its highest high-fidelity, can never be achieved as imagined, because, as we have seen, there is no one-to-one correspondence between input and output.

Digital products treated as an assembly line usually end up being monstrosities.

They are companies, therefore, that are comfortable with the fat point -low fidelity- and touching to understand.

Or also, validating to reduce risk.

They need to encourage going to the real in order to form an opinion.

It's not unusual, therefore, for them to favor a style that emphasizes creating things and sharing them as quickly as possible to see how they're received; involving potential users and customers in the very invention process to incorporate learning as quickly as possible; and insisting that pace, in itself, is a positive value (the mantra "speed wins" is possibly the one that best summarizes this orientation).

Touching to understand also means that these are organizations that struggle to survive when they don't internalize the material they work with: the business you can imagine also depends on your understanding of, for example, how a request works in a browser.

Release early, release often.

But also, along these lines, they are organizations that benefit from reflecting on how they want to organize themselves to invent, since they cannot borrow styles that have worked in other industries in the past and assume that they are an operational template regardless of the material they work with.

In this respect, they feel bad about anything that leads to thinking in terms of merely executive organization: estimates of hours, estimates of costs of requirements, estimates of the number of people that could be added to speed up executions, etc.

They feel good about looking at organizations associated with collective creativity - from the entertainment industry to sports - where a person is not a resource - what matters is who and not just whether it is backend or frontend - and where the organization itself has to incorporate a constant and changing reflection on how it is articulated - the way in which a film shoot is organized or the tactics of a team sport are thought out have many similarities with the challenges faced by these types of organizations.

Software-based products play best with hive minds.

It’s no wonder then that these kinds of companies live in a constant buzz about the latest trendy methodology that might finally give them that sense of how to handle situations that never quite fit them.

The companies that have understood that part of their operations includes inventing how they invent are the ones that are introducing innovations to the market of methodologies, whether it's Spotify, Duolingo, or Basecamp.

The most prudent approach would be to imitate them in their vocation, not their results, although the latter is more common, and that's where you tend to pick up the snake's skin, so to speak: what suits Google may not suit your organization.

Even less so if your company has no similarities in size, business, people, or, in general, culture.

Digital product teams can't stop thinking about how they organize themselves.

They are also companies that are uncomfortable being aliterate, even though they often operate in oral mode.

Decision-making in such a slippery environment, in the sense of lacking a reasonably firm foundation of obvious facts on which to base conjectures, appreciates written deliberative support.

It's the space where you can articulate the amount or lack of evidence you have about something—in quantitative or qualitative form; where you can make explicit the logical connections, strong or weak, behind a decision—so that the very game of scarcity and alternative uses of money have something to back it up; and where you can construct the stories that motivate those decisions.

It's also the way to unite the organization and articulate its own strategy, which can never be simply dumping a few tasks into a backlog.

Even less so if the possible arguments connecting those tasks have been dealt with orally.

It is necessary to streamline the process that leads from capturing a potential opportunity, to shaping it from possible solutions to, finally, building something.

In this function of compensating for uncertainty and risks, there are companies that also need to equip themselves with a specific disposition towards metrics and the use of quantitative data.

Our intuitions improve when we compare them with statistical patterns, and also when we get used to seeking quantitative guidelines.

These types of companies benefit from having people who think in terms of magnitudes rather than accuracy to three decimal places.

This is the path through which uncertainty can be quantified and, from there, probable conjectures constructed.

It's natural that different styles of organizing these companies around metrics—from OKRs to north stars, translated into indicators—will resonate, since metrics help them reduce the risk of the bets they make.

Metrics are a compass that do not prescribe the path to the destination.

Since these are organizations that, due to the material they work with, must give importance to both the achievement of objectives and the increase in learning, they also need to face the challenge of acting as a collective intelligence.

Promoting team autonomy, without subordinating them to management validation processes, becomes a necessity from the moment only those working with the material have adequate and up-to-date information to make decisions.

It's natural for methodologies to proliferate that favor organizations with management exclusively occupied with developing core themes while delegating not only the execution but also, in many cases, the decisions themselves (the numerous ones that occur in the creation process itself) about what to do to implement those core themes.

These are companies, therefore, that benefit from involving anyone with valuable and proven knowledge as early as possible, and not simply leaving them with a small portion of the execution: the person who programs is just as valuable when they program as when they don't, for example, because they are designing different solutions to a problem, or because they are investigating the constraints that may condition those solutions, or because they are experimenting with something new that could also end up being the key to solving something in an unexpected way.

Time and attention are two relevant factors in the development of digital products.

Finally, this type of organizations have the need to put their customers at the center, because they are a relevant source of information and learning: they help to reveal ramifications and hidden aspects of the black box; Its very use sheds light on the generation of errors and experiences where the output doesn't match what was projected at the input, but it also allows for the articulation of decisions based on certain quantitative regularities and helps define functionalities that could otherwise end up becoming a wormhole, compromising the very viability of the company.

In this sense, unlike the style of other types of companies in the past, these companies thrive on breaking down the wall that separates their interior from the exterior.

Companies have a tendency to create walls between the people who invent and the people they invent for.

It’s only natural that slogans like dogfooding become popular, where the organization and the customer become confused when the former uses its own product in order to have the experience of the latter.

Quality becomes, in a way, a collaborative control.

The entire trend we mentioned earlier regarding context awareness and direct conversation with people needs to be incorporated in some way.

Being on the other side of the mirror is so central to these types of companies that even for everyday incidents, screenshots and recordings, situation data for playback, and the ability to log into the customer's account to see what they see are completely naturalized.

The most effective way to understand someone is to live their life a little.

Contrary to popular belief, fostering scarcity brings many benefits to these types of organizations.

Precisely because it counterbalances the abundance (of anomalies, possibilities, and materializations) that characterizes software.

These companies thrive on a decision-making framework based on bets and appetite.

It's a framework that's also easy for anyone to understand, since our hunger for something and what we're willing to sacrifice for it naturally correlate.

Organizations improve when they shift the question from how long something will take—because it forces me to make a rough estimate on uncertain ground—to how much I would dedicate to it.

Whether it's attention, money, effort, or time, which, in the end, are all sides of the same coin.

Appetite helps us establish what is central and what is peripheral.

Not only has digital not invalidated the oft-repeated slogan in economics that "nothing is free", but it pushes us to remind ourselves of it more than ever, so that we work in a more responsible and professional manner.

Ethics is the first to benefit when the people who build companies improve their knowledge of the material they work with, precisely because of all the difficulty that accompanies software-based products.