Category: Thinking out loud

Instigating innovation by enhancing experimentation

“We don’t experiment!”, the operations manager sneered at me. “We know what we are doing. We are experts”. From the shaking of his head I could form my own conclusions. It meant that this business has a very short term focus in terms of innovation, mainly using a consensus based approach to drive incremental improvement. The irony is that the word “expert” implies learning by doing, often over an extended period. The very people that become “experts” through experimentation and trying things become the gatekeepers that promote very narrow paths into the future, thus inhibiting learning in organizations.

The aversion to experimentation and its importance in innovation is institutionalized in management. Many of the textbooks on innovation and technology management does not even have a chapter on experimentation (see below for some exceptions). Many industrial engineers and designers actually narrow the options down so early in a process or product design so that what comes out can hardly be described as an experiment. Approaches such as lean and others make it very hard to experiment as any variation is seen as a risk. In more science based industries, such as pharmaceutics, medicine and health, experimentation is the main approach to innovation.

Most manufacturers do not like the idea of experimentation, despite it being widespread in most companies. If management does not see it (or hear about it) does not mean it is not happening. This is the main problem. Lots of companies (or rather employees) experiment, but the feedback systems into the various levels of management and cross functional coordination are not working. Learning by doing is hard to do in these workplaces. Furthermore, management systems that rewards success or compliance makes learning by doing almost impossible.

Let me first unpack what I mean with experimentation.

Experimentation is a kind of investigation, or an attempt to better understand how something works. It is often associated with trial and error. Sometimes experiments are carefully planned, other times it can be impulsive (like when people press the elevator button repeatedly to see if the machine responds faster). Experiments are sometimes based on a deep insight or research, then it is almost like a authentication or proofing exercise. Other times it is done as a last resort (two attempts to get the machine to work did not work is followed by hitting it with a spanner). This could be naive even a little desperate. (Suddenly the machine works and nobody knows what exactly solved the problem). While experiments can be to prove something, I believe that not enough managers realise that experiments is a powerful way to keep their technical people happy (geeks love tinkering) and a strong way to improve the innovative and knowledge capability of an organisation. What does it matter if this experiment was successful in 1949, why don’t we try it and see if we can figure it out? Remember, innovation is a process of combination and recombination of old, new and often distant capabilities and elements.

Experiments in manufacturers happens at different levels.

  • It happens spontaneously on the work floor, where somebody needs to keep a process going. Ironically often experiments in the work space is the result of resource constraints (like trying to substitute one component/artifact/material/tool for another. A lot of potential innovations are missed by management because feedback doesn’t work, or experimentation is not encouraged or allowed.
  • Experiments could also be directed and a little more formalized. Typically these experiments originate from a functional specialization in the business, like the design office or another function. In these experiments the objective, the measurement and evaluation of the experiment is valuable for management as it could create alternative materials, processes, tools and approaches viable.
  • At a more strategic level experimentation often happens when evaluating investments, for instance making small investments in a particular process or market opportunity. It could also be about experimenting with management structures, technology choices or strategies. Sometimes the workers on the factory floor bear the brunt of these “experiments” which are not explained as experiments but rather as wise and thoroughly through decisions. In companies that manages innovation strategically, decisions at a strategic level could involve deciding how much funds to set aside or invest in tools that enable experimentation, for instance 3D printing, rapid prototyping, computer aided design and simulation, etc.
  • Accidental experimentation occurs when somebody by accident, negligence or ignorance does something in a different way. This occasionally result in breakthroughs (think 3M), and more often in breakdown or injury. Accidental experimentation works in environments where creating options and variety is valued, and where co-workers or good management can notice “experimental results” worth keeping. However, in most of industry accidental experimentation is avoided as far as possible.

The above four kinds of experiments could all occur in a single organizations. At a higher level experimentation can also happen through collaboration beyond the organization, meaning that people from different companies and institutions work together in a structured experiment.

When you want to upgrade industries that have a tendency to under invest in innovation, you can almost be certain that there is very little formal experimentation going on. With formal I mean thought through, managed and measured. Proving one aspect at a time. It is often necessary to help business get this right.

Since this series is about instigating innovation in both firms and their supporting institutions it is important to consider the role of supporting institutions. One important role for supporting institutions is to lower the costs and risks of experimentation for companies. This could be through the establishment of publicly funded prototyping or demonstration facilities. Another approach is for supporting organizations to support collaborative experiments. However, I sometimes find that supporting institutions themselves are not managing their own innovation in a very strategic nor creative way.

Helping industries to improve how to conduct experiments need not be expensive and does not necessarily involve consulting services (many institutions are not organized for this). For universities there are some interventions that align with their mandates. For instance, exposing engineering students to formal experiments with strong evaluation elements (such as chemistry students have to go through) can also make it more likely that a next generation of engineering graduates are able to also plan and execute more formal experiments. Or creating a service where industry can experiment with technology within a public institution. Or arranging tours or factory visits to places where certain kinds of experiments are done, or can be done.

Lastly, not all experimentation needs a physical embodiment. Design software, prototyping technology, simulation software and 3D printing makes are all tools that enable experimentation and that reduces the costs and risks of experiments. Furthermore, experiments need not be expensive, but they should be thought through. I often find that companies want to create large experiments when a much smaller experiment focused on perhaps one or a few elements of the whole system would suffice. Here it is important to consider the science behind the experiment (at a certain smaller scale certain materials and process characteristics are no longer reliable or representative). The experiment must be just big enough to prove the point, or to offer measurement and comparison or functionality, nothing more.

I will close with a little story. I once visited a stainless steel foundry. These businesses are often not known to be innovative, but I was in for a surprise. The CEO of the foundry had a list of official experiments that were going on. Often each experiment had a small cross functional team involved, supported by a senior management champion. The aim was not to succeed at all costs, but to figure things out, develop alternatives AND increase the companies knowledge of what is possible. Everybody in the different sections of the business knew when experiments were taking place, and everybody was briefed on the results. Even though this is a very traditional industry, this company had managed to get their whole workforce to be excited about finding things out.

I promise I will get to the how in a future post in this series.

 

My favorite text books on experimentation in innovation are:

DODGSON, M., GANN, D. & SALTER, A.J. 2005.  Think, play, do : technology, innovation, and organization. Oxford: Oxford University Press. (I think this one is now out of print)

VON HIPPEL, E. 1988.  The sources of innovation. New York, NY: Oxford University Press. (Despite being an old book this is really inspiring)

THOMKE, S.H. 2003.  Experimentation Matters: Unlocking the Potential of New Technologies for Innovation. Harvard Business Press.

HARVARD. 2001.  Harvard Business Review on Innovation. Harvard Business School Press.

If you know of a book more recent then please let me know.

 

How I teach the topic of innovation systems

IMG_2533One of my favorite subjects to teach is about the promotion of innovation systems. I love it because it combines abstract elements that most people grasp, and practical elements that most people enjoy. Most academic literature on innovation systems are quite abstract, and our approach to identifying ways to improve an innovation system from its current state is quite pragmatic. The literature on managing innovation is very broad and contains millions of tips, theories, myths – actually it is overwhelming for practitioners wanting to support industries, firms and organizations to become more innovative. Therefore I try to explain the principles of both innovation systems and innovation management so that people can re-organize and use what they already know, and know where to relate new knowledge that they may encounter along the way.

SC_in_action
Trying to explain how to get exploration and safe 2 fail experiments to work

I typically start by laying some foundations, often using puppets, props or cartoons to make it slightly less serious (I use sheep characters, don’t ask why):

While most people intuitively understand that there are different kinds of innovation, most practitioners are surprised by how different product innovation, process innovation and business model innovation are. A great discussion usually takes place when people reflect on why business model innovation (Tim Kastelle states that it is easy but really hard) is really what hampers growth and productivity improvements, but how most industrial and innovation policies typically targets mostly product and process improvements.

Now that the foundation is in place, I typically move on to the more abstract issue of innovation systems. After explaining the definition (see the bottom of the post) that I like most, it is necessary to explain the importance of the dynamic between the different elements. It is natural to create checklists of institutions and actors and tend to forget that even in economic development weaker actors that interact more dynamically can trump first class institutions that are not accessible to most people that need support.

The importance of building the technological capability beyond the leading firms is important. I have written many posts about this so will not repeat this here, but for me the systemic nature of innovation and knowledge accumulation is critical. But typically we use 6 lines of inquiry to investigate how the dynamism in the system can be improved. There are four really important aspects which include:6 Four lines of inquiry_web

 

The agenda concludes with different ways practitioners and policy makers can intervene in the innovation system to improve the dynamics, the flow of information, the exchange of knowledge and the increased innovation appetite of entrepreneurs.

Duration

To present this agenda can take anything from 2.5 hours to three days. When the participants are experienced in diagnosing enterprises and public institutions, the exercises tend to be more meaningful and fun. When nobody in the room knows anything about the problems companies face on a day to day basis this kind of training is much harder. When I have more time then topics such as mapping formal knowledge flows, detecting unmet sophisticated demand, collaborating for research and development, etc can be included.

I have been presenting this session is various formats at international training events like our Annual Summer Academy in Germany, at different academic departments in universities. I frequently present this in some form to science, technology and industry government officials. In other occasions I have presented this to practitioners, development staff and even to the management of a university wanting to become more innovative itself.

The definition I work from:

The definition of innovation systems that I work from is the one of the earliest definitions on this subject. Freeman (1987:1) defined an innovation system as “the network of institutions in the public and private sectors whose activities and interactions initiate, import and diffuse new technologies.” The emphasis is mainly on the dynamics, process and transformation of knowledge and learning into desired outputs within an adaptive and complex economic system.

The textbooks I teach from:

My favourite textbook that I use when teaching at universities remains FAGERBERG, J., MOWERY, D.C. & NELSON, R.R. 2005.  The Oxford handbook of innovation. Oxford ; New York: Oxford University Press.

If I have more of a business management audience, then I prefer to use a book with more innovation and technology management tools in it such as DODGSON, M., GANN, D. & SALTER, A. 2008.  The Management of Technological Innovation. Oxford University Press.

Of course, this agenda follows the logic of my own book on the promotion of innovation systems that I have published!

 

 

Innovation systems in Metropolitan Regions of developing countries

During 2015 Frank Waeltring and I were commissioned by the GIZ Sector Project “Sustainable Development of Metropolitan Regions” (on behalf of the German Federal Ministry for Economic Cooperation and Development (BMZ), Division 312 – Water, Urban Development, Transport) to write a discussion paper about a hands-on approach to innovation systems promotion in metropolitan regions in developing countries. The discussion paper can be found here.

Frank (left) and Shawn (right) in front of the Berlin Wall Memorial

This assignment was a great opportunity for us to reflect on Frank’s experience on structural change in territorial economic development and my experience on industrialization and innovation systems in developing countries. We also had to think hard about some of the challenges of using a bottom up innovation systems logic in developing countries, as such an approach would rely heavily on the ability of local public management to coordinate strategic activities aimed to improve the dynamics between various public and private stakeholders. It was great to reflect on our past Local Economic Development experience and our more recent work on innovation systems, industrial upgrading and complexity thinking.

A key aspect of this discussion document was to think long and hard about where to start. We know many economic development practitioners in cities are often overrun by demands from both politicians and industries for support. We also know that by selecting promising sectors based on past data and assumptions about job and wealth creation often end in little impact and much frustration. We agreed that an innovation systems approach must be aimed at stimulating the innovative use of knowledge, so we decided to not start with a demand focus (assuming the officials are already responding to some of the demand) or with statistics but a knowledge application focus. The use, generation and recombination of knowledge is central to the technological upgrading of regions, industries, institutions and societies. From our experience in promoting innovation systems and our recent research into non-consensus based decision making (this is where you do not select target sectors based on consensus or assumptions about growth potential, but you look at emergent properties in the system) we decided to start with three questions to understand the dynamics of knowledge flows in the region:

  1. Which enterprises, organisations and even individuals are using knowledge in an innovative way? Obviously this question is not simple and can only be answered by reaching out in the local economy to institutions, firms and individuals.
  2. Which stakeholders are actively accumulating knowledge from local or external sources? Again, this is an exploration.
  3. Who are individuals or organisations that know something about unique problems (challenges, demands, constraints) in the region? These could be buyers, supply chain development officials, public officials, engineers or even politicians that are willing to articulate unique demands on the regional economy that might not have been responded on by local (or external) enterprises.

These three questions are treated as an exploration that will most likely be most intensive at the start. In our experience economic development practitioners should constantly be asking themselves these questions when working on any form of private sector upgrading.

A second dimension is about assessing the interplay between institutions and industries and its effect on innovative behavior within regions. Who is working with whom on what? Why? What are the characteristics of the life cycles or maturity of various kinds of stakeholders in the region? Thus we are trying to understand how knowledge “flows” or is disseminated in the region. While some knowledge flows are obvious, perhaps even formal, some knowledge flows could be more tacit and informal. For instance, while knowledge flows from education is quite formal, the informal knowledge exchange that takes place at social events is much more informal, yet very important.

Apart from the identification of the dynamics and interrelations between the industries and the different locations, one other key factor is to identify the drivers of change who want to develop the competitive advantages of the region.

We also present our technological capability upgrading approach as six lines of inquiry, some of which have been covered in earlier posts on this weblog:

  1. The company-level innovation capability and the incentives of firms to innovate, compete, collaborate and improve, in other words the firm-level factors affecting the performance of firms and their net-works of customers and suppliers. These include attempts within firms to become more competitive and also attempts between firms to cooperate on issues such as skills development, R&D, etc.
  2. The macroeconomic, regulatory, political and other framework conditions that shape the incentives of enterprises and institutions to develop technological capability and to be innovative.
  3. Investigation of the technological institutions that disseminate knowledge.
  4. The responsiveness and contribution of training and education organisations in building the capacity of industry, employees and society at large.
  5. Investigation not only of the interaction and dynamics between individual elements in the system, but of the whole system.
  6. Exploring poorly articulated needs or unmet demands that are not visibly pursued by the innovation system.

We, and of course our GIZ colleagues of the Sector Project Sustainable Development of Metropolitan Regions, are very keen to engage with the readers on these ideas? Please post your comments, questions to this weblog so that we can have a discussion.

Best wishes, Shawn and Frank (Mesopartner)

 

 

2015 in review

The WordPress.com stats helper monkeys prepared a 2015 annual report for this blog.

Here’s an excerpt:

A New York City subway train holds 1,200 people. This blog was viewed about 6,000 times in 2015. If it were a NYC subway train, it would take about 5 trips to carry that many people.

Click here to see the complete report.

TCI 2015 conference – part 2

Wednesday the 4th was jam packed with great speakers, parallell streams, and great conversations in the corridors. It is not possible nor fair to try and summarize everything here, but I wanted to share just a few thoughts. Please excuse the formatting of this post, I have written this on the fly on my iPad.

In the opening plenary session, Prof Christian Ketels made an excellent presentation that laid a foundation for the event. He emphasized that clusters emerge and are not created. Of the many things he said this stood out for me, as my own experience is that a lot of people are trying to create clusters out of groups of homogenous firms – what I would call a collection and not a cluster.

During the afternoon I could listen to many excellent presentations about entrepreneurial ecology, measuring and evaluating cluster performance, etc. Melissa Pogue (Martin Prosperity Institute) made a great point that it is important to find ways to harness the creativity of people in domestic companies that was not focused on global competition (the so-called traded clusters). This involves findings ways  to unleash the potential of people employed in “average” or local companies, increasing their ability to make decisions and to exercise their judgement. She used a great dataset from the US to show how important this is, as focusing only on the “creative” companies lead to increasing inequality created by rapidly increasing prices of housing, services, etc. in more creative or innovative regions. I thought this was a very valueble point because my practice is also focused on trying to get the more traditional sectors to become innovative, however, much of the cluster discussion is focused on the leading or more creative companies. 

In the closing session of the day I heard that Singapore coordinates R & D and innovation interaction with the private sector from the Prime Ministers office. While my opinion that this should be done from the bottom up is well known, the problem we often face is that top down support is often poorly coordinated or maybe even inconsistent.

A second point was raised by one of the wise men of cluster practice, Ifor Ffowcs-Williams. He made several important points, but one in particular is worth pondering. He stressed that clustering should be more about relations and dialogue. While everyone would agree with this statement, many clusters are completely dependent on a hierarchical arrangement with a cluster manager keeping the whole thing together. Resilient clusters emerge from a dense interaction between members, covering a wide range of topics and issues. However, many cluster managers biggest concern is not about building trust nor is it finding ways to stimulate collaboration between members. Rather, it is about raising funds (often public) or justifying continued support to industry.

Tech transfer in South Korea

Four of the partners in Mesopartner are at the TCI Conference in Daegu, South Korea. On the 3rd of November we all went in different directions on excursions to various clustering initiatives in the region.

I signed up for the mechatronics tour. I want to share a few observations about the technology transfer institutions that we visited.

Firstly, technology transfer into the region is focused on stretching existing enterprises. You would think this is obvious. In South Korea, the different levels of government contribute large amounts of funding to buy the latest and most modern equipment that are placed in public institutions. This technology is often identified by leading firms like Samsung. Important criteria for technology selection includes its “platform” ability, meaning that it can be used in several industries. The Koreans refer to this as “convergence”. A second criteria is that it must enable competitive products to be developed with a strong focus on exports.

Secondly, cost recovery is a low priority. At the institutions that we visited they often charge as little as 20% for the use of the latest equipment, basically recovering costs of consumables. The facilities cannot handle production orders, but are used mainly to demonstrate applications or for making prototypes. The facilities consist of open spaces, open labs and cutting edge testing facilities.

Thirdly, the institutions support smaller companies in R & D and product development, often on-site. It struck me that the institutions realized they have to “take” the technology to industry. While most of the effort is focused on new products and new enterprises, there is still on objective of helping incumbent more traditional companies to innovate.

Lastly, we visited a Creative Economy cluster initiative. It was not focused on arty projects, but on hardcore technology like making micro-robots, smartphone attachments, etc. Companies could bid for space (literally a 15m2 space in a modern office environment). There is a strong emphasis on smaller down-scaled technology applications. Entrepreneurs that are selected to join the incubator have 6 months free rent, lots of technical and market development support, and networking and exchange with other incubatees are compulsory. Large companies like Samsung, LG, etc have technicians and coaches on site (24/7) to support any enterprise on almost any topic. Thus the resources of large companies that are partnered with these centres are made available to help smaller startups.

Today the conference starts. Already I feel like I have learned enough to justify the trip from Pretoria to Deagu.

If the culture cannot change then the business cannot change

Originally published in August 2015, revised in January 2018

I received many comments and tweets about the previous post. Thank you for ideas and comments

Some agreed that innovation is the result of culture. Some said that culture is not only created by management, but also by staff. For instance, the admin pool in a traditional engineering company can be very innovative (and creative) even if the rest of the business is stuck in the 1980s.

Somebody told me that creating an innovative culture is in itself a chicken-egg (low equilibrium) situation, because for a leader to create (or enable) an innovative culture takes innovation in itself. You can see where this is going.

Then I discovered a recent cartoon in my inbox by Hugh Macleod of Gapingvoid fame. This cartoon says it all.

An organization that cannot change its culture (due to too rigid systems, due to lack of management capability, due to its people) has become trapped in time. While some organizations may exist like this due to sheer momentum, due to protection (by law), by continuous funding, or for whatever reason, will struggle to adapt to external change. These organizations are not resilient and they are at the mercy of external supporters (a.k.a clients, benefactors, funders or shareholders).

I was also asked how some organizations can still innovative despite a poor innovation culture. Again, it is of course possible to replace a machine, or for a few people in an organization to design something brilliant, or for a new process to emerge. Of course it is possible. But it takes much more energy, determination of a few, and some really tenacity to be innovative in an un-innovative (what is the right word here?) culture.

I am sure more comments will come.

Cheers, Shawn

I appreciated the comments received by e-mail, but wonder why people are not posting comments to this article? Is the WordPress registration process to difficult? Please let me know. And keep those comments coming!

Systemic Insight – Economic development is about introducing options, not bringing solutions

Marcus and I have just posted a new article on the Systemic Insight website. The post is about our recent article that was published in the IDS Bulletin (Vol 46.3) and is titled “Explore, Scale Up, Move Out: Three Phases to Managing Change under Conditions of Uncertainty”. 

Please note that in future all my posts specifically about complexity and resilience will be published on our Systemic Insight website. To my regular blog readers it may be worthwhile to also subscribe to the feed on that site. We promise to write more often now that the foundation for our applied research theme on complexity in development has been created. We already have many customers using our approaches and this area of work is promising and rewarding. We have added a new page to the Systemic Insight site that explains some of our most frequently asked about services related to complexity thinking applied to development.

On this site I will keep on writing about Private Sector Development in general, and particularly on innovation and innovation systems.

Your feedback, comments, emails, phone calls, tweets and likes are appreciated. Let us know what you think and what you would like to discuss, read about or just “air”.

Best wishes,

Shawn

Post 3: The role of various organizations involved in education

This is the 3rd post in the series on building technological capability.

Contrary to common belief, building technological capability does not start with eduction. That being said, education matters and certainly makes any upgrading effort much easier. It is not only about the basic qualifications, but also about the ease with which existing employees can further their education.

With education institutions we include all forms of education, from basic to technical, vocational to post tertiary education. We also include public as well as private providers of education. Education lays an important basis with regards to skills, but also increases the absorptive capacity of a society. Advanced forms of education includes research.

In general, when we assess the role of education in assisting enterprises or industries to upgrade, we want to know how responsive the education system (broadly speaking) is. If a new standard is agreed upon in industry, how long does it take for the technical Universities to include this in their programmes? If a new disruptive innovation takes place, how fast is the curriculum updated? I remember back in the 90s when I studied for the first time, how the software we used at the university was no longer available commercially. Fortunately, all those Lotus 1-2-3 shortcuts still worked on Excel, I still use it today 😉

While the responsiveness of the education sector is important, what is ideal is a education system that not only responds to the needs of the private sector, but it preempts or anticipates what is needed next. This is very important for the manufacturing sector. Graduates must know not only what is now mainstream, but also what is expected or coming soon. This is where large parts of our South African university system lags behind. But this is not unique to South Africa.

A last point I want to make about the education sector is that it is also important to understand the role and contribution of the private sector to the education sector. In many countries there is a close relationship between the private sector and for instance universities. Companies contribute to to not only basic education infrastructure, they often fund research positions and projects. An industry that is complaining that they don’t get from the education sector what they need is most likely also not contributing through finances or advice.

On market failures – perhaps you are too close

I am often involved in coaching and capacity building a different kinds of private sector development experts working in the developed and developing world. I am sometimes shocked when I realize that a practitioners or programme managers in the field involved in market development do not understand some of the basics of how markets work or how to address market failure. This is often made worse by the broad ideological blindness of the organizations that promote market development approaches. I state this based on my experience that when markets and its alternatives are properly explained to teams in organizations, many problems resolve themselves, largely because the way markets function and evolve are better understood. Don’t get me wrong, I love markets. They are amazing in that they can emerge almost anywhere but where we often seem to need them. But I am not blind to their limitations (like how unfairly they allocate gains), nor am I naive about what it takes to get market systems to work.

If you are trying to solve market failures by bringing suppliers and buyers of a particular good or service together you may be too close to the action to really make a difference in the medium to long term. Actually, you might be making it harder for markets to evolve, as trust that is weakened when something does not work as it should or as promised is not easily forgiven in the real world, making 2nd attempts very hard if not impossible. There are many reasons why I say this.

Firstly, a market failure is a symptom that something else is wrong. It could mean that knowledge about the product or service, or how or why it is used, is not available or costly. This could imply a deeper failure (knowledge related) that people do not understand the value, the impact or the modalities of the good or service, or how the good or service will affect them or what it might depend on. Or the supplier is not able to demonstrate or explain how a good or a service can be used, or that it will address a particular need.

Secondly, modern markets are tightly intertwined and interdependent on other markets and other forms of allocation beyond markets. For instance, the service for quality management advice needed by food producers is dependent on many other services, including management consulting, HR consulting and sufficient demand for companies that are for quality accredited. It may also depend on some technical expertise in the form of a service about the product itself and the regulations it must comply with. These different markets co-evolve and depend on each other. Furthermore, this quality management service is also shaped by domestic and international regulations, standards and norms. Lastly, this service may also be specific to a particular service or product type, so the potential impact of the service or particular good may be easier (or harder) to guess so that a potential buyers of the service can figure out if this money might be spent in a better way. Remember that spending money on the wrong thing (adverse selection) is also a market failure if this is caused by an inability to thoroughly evaluate the expected benefits of alternative choices.

Thirdly, most services and products traded in markets also depend on related or supporting networks and hierarchies. For instance, few market services or products used by businesses can be used if that business (a hierarchy) does not have a management capacity, or absorption capacity (to figure out how the product or service will impact the rest of the business) or a functional capacity (internal expertise to use the product/service optimally). Many first time users of products and service depends on social networks to evaluate alternatives.

Fourthly, many services are not provided only by the private sector, but also by public providers and not-for-profit organizations (and even via networks). The more generic the service, the more likely that it wont succeed as a private service (because business typically pays for additionality, generic solutions can often be developed in-house (via hierarchy). Many “business services” in developed countries are provided by private, public, not-for-profit (networks) or hybrid models. Multilateral development organizations often promote “commercial” business services even when in their own countries these services are also available as public or hybrid services. Often services are first provided by the public sector, and the complimented by the private sector as demand becomes more specialized. Or services are provided by the private sector, until the public sector realize that it is in fact a public good or service and that it should in fact be provided by the state. But often, in the long run, products and services provided in the public sector are also provided in the private sector, and vice versa. The order depends not only on the context, but also on the dependency and interdependency of the markets, as well as the costs and efficiency of the alternative means of provision.

Lastly, in the words of Mark Granovetter, markets are deeply embedded within a societal context. Markets are part of the society, it reveals what a society values, how much it trusts, and how much it values people keeping their promises. You cannot isolate a market from the context, optimize it and then insert it back in the society. The societal context provides the trust, the enforcement and even information flows that makes it possible for markets to work. Out of this society a whole range of institutions emerge, some in the form of organizations, others in the form of norms, habits and routines.

During training sessions on how markets work, practitioners are often surprised to find out that markets are only one way a society allocates goods. The other way is through networks (often not in exchange of currency), or through hierarchies (organizations that allocate resources internally). When markets are new, they often emerge first as networks. Over time a group of people that know each other socially formalize their transactions, and out of this markets emerge. This is why we often advise practitioners that when one form of allocation fails, the solution is often to stimulate the others. So when a market fails, first try networks or hierarchies.

We often use a case study to illustrate the point. A service provided in one country by the private sector as a commercial service, is provided in another country as a public service. In a third country, the service is provided by an association as a network good. Pairs of practitioners from different countries then assess the three cases and must make a recommendation. It is quite funny to see how people from different parts of the world disagree on what constitutes a commercial service (market transaction), what constitutes a public good (allocation via hierarchy) and when a network transaction is better.

On the point of designing markets. While it is true that some markets are designed, these designs are often carefully planned and regulated. Think of mobile phone spectrum or broadcasting rights. It is not so easy to design markets that needs many actors to cooperate and that depends on many other variables that you cannot control through regulations. Even if you could use regulations, you might have the problem of not being able to change something if you need to.

In the end, markets learn and adapt. Actors in markets experiment, they learn from each other, and they adapt. This takes time, much longer than the life of a development programme. Ask yourself, why does a market for cigarettes develop in a prison within hours, but a market for tomatoes can take years? We have to understand the preconditions and the evolution of markets much better if we want to assist the evolution of societies and their markets.

To solve market failures, we often have to move one level up to where societies turn broad and generic policies about the society into organizations or targeted interventions. This may still mean working with the people doing the transactions to learn from them, but often the solutions will lie in institutions, policies and eventually maybe in regulations and standards.