In order to avoid misconceptions about the term market failure as it applies to the study of economics, start with a definition:
Market failure: When the free market does not allocate resources to their most efficient uses.
Market failure does not occur just because somebody doesn't agree with the outcome of free markets. That would be a normative issue. Economic thinking requires dealing with positive, not normative issues.
Market failure involves the concept of externalities. Externalities occur when actions of producers or consumers affect third parties - people not involved in the production, purchase, or sale of a particular good. Externalities are sometimes referred to as spillovers.
Externalities can be positive or negative. An externality can be a spillover benefit or a spillover cost.
A few more relevant definitions:
Private benefit: A benefit received by a party to a transaction.
Private cost: A cost paid by a party to a transaction.
Social benefit: Total benefit of a transaction: equal to private benefit plus external benefit.
Social cost: Total cost of a transaction: equal to private cost plus external cost.
An example of a positive externality: Flu vaccinations.
Each year, many people receive flu vaccinations. The private cost is paid by the individuals receiving the vaccinations (or their insurance companies). The private benefit is the lower health risk and the associated gain in peace of mind for the individuals receiving the vaccinations. While many people are involved in these transactions, many others are not. For a variety of reasons, many people choose not to receive this vaccination. Those who do not receive the vaccination still receive a benefit: As a larger number of people receive vaccinations, the less likely it becomes for those who choose not to receive vaccinations to be exposed to the flu. The general health of the population benefits from flu vaccinations, not just those individuals who pay to receive them.
With a positive externality, the free market price and quantity does not factor in all of the benefits. Resources are under-allocated. Too little production of the good in question occurs. The free market price and quantity will be determined by the equilibrium where the supply & demand curves intersect, based only on private benefits and costs. If all of the social benefits were included, the demand curve would be farther to the right, creating an equilibrium with a higher price and a higher quantity.
An example of a negative externality: Air pollution.
If the unregulated production of a particular good involves polluting the air, this production imposes a social cost that is not factored into the market price and quantity of the good being produced. The market price and quantity will be determined by the private costs and benefits only. The social costs of air pollution are many: Poor health for residents and animal life; increased medical expenses; shorter life expectancies; missed work and production due to illness. Even obscured scenery is a social cost.
With a negative externality, resources are over-allocated. Too much production of the good in question occurs. If all of the social costs were included, the market supply curve would be farther to the left, resulting in a higher price and a lower quantity.
The socially optimal level of externalities is not necessarily zero. In the example of the flu vaccinations, a solution that involves every single person in the population receiving a vaccine may indeed add in a cost that exceeds the social benefit. In the example of air pollution, a law that requires firms to emit zero pollution may force firms to shut down, resulting in no production. Shutting down production may very well involve social costs in excess of the social costs caused by air pollution.
Possible solutions to the problems of externalities:
For positive externalities: The government could pay a subsidy to consumers who purchase the good. This would mean that the cost to consumers would be less than the amount received by producers. If the amount of the subsidy is the amount that would induce enough demand to equal the socially optimal level of production, then this would have the effect of shifting the demand curve to the right. This would result in the socially optimal level of production at the socially optimal price. Efficient allocation of resources would result.
The problems with this solution: No market exists to determine what level of production is socially optimal. That level requires some guesswork, and the result would be rather arbitrary. Also, government-paid subsidies involve problems relating to the methods that governments have for financing expenditures.
For negative externalities: Three possible solutions for negative externalities are a tax on production, a government command, and marketable permits.
Tax on production: A tax for a negative externality has the opposite effect as that of a subsidy for a positive externality. A tax on production will increase the cost of production of the good in question, resulting in a leftward shift in the supply curve. If the tax is set at the proper level, the result would be the socially optimal level of production at the socially optimal price. Efficient allocation of resources would result.
The problem with this solution: No market exists to determine the socially optimal level of production. Guesswork is involved, which would produce rather arbitrary results. One added benefit for the government would be additional tax revenue.
Government command: Instead of imposing a tax on a negative externality, the government could pass laws setting the legal upper limits on the amount of the negative externality that is produced.
The problems with this solution: The limit would not be market-based; it would instead be somewhat arbitrary. Those who create the negative externality would have no incentive to lower the production of the negative beyond the legal limit. With a tax, a firm would have an incentive to find new technology that would lower the tax along with the externality. With a government command, no such incentive exists. A command will not necessarily lead to an efficient allocation of resources.
Marketable permits (also known as cap and trade): Cap and trade is a system that is designed to eliminate many of the problems associated with the tax and the government command methods for decreasing a negative externality. It is the only method of the three that uses market forces to determine the amount of a negative externality that is produced by any individual firm. Cap and trade works like this: The government sets a limit on the maximum amount of a negative externality that will be allowed, but this limit is on an entire industry instead of an individual firm. The government then issues permits for the negative externality. The total permits issued equal the maximum externality that is allowed. Any individual firm can produce as much of the externality as it wishes, as long as it has enough permits to cover that amount. If a firm wants to produce more of the negative externality than it has permits for, it can only do so if it buys more permits from other firms in the industry. The total of the externality is controlled by the government industry-wide, but the market for permits determines the amount of the externality produced by individual firms. The permits will have their own market, and their market price will create an incentive for firms to reduce production of the negative externality.
The market for permits could also be used to reduce the industry-wide amount of a negative externality. For example, an environmental group could decide to purchase permits with no intention of producing the externality. By holding permits, the environmental group could prevent the permits from being used by firms that would produce the externality.