Tag Archives: planning

accuracy of a model vs. its “decisional quality”

Leave a reply

I like the way the abstract of this paper distinguishes between (1) the accuracy of a model as measured by comparing it to physical observations (always assuming those are an accurate or at least unbiased measurement of the true state of the universe and (2) the appropriateness of a model to be used in decision making. I find these concepts very, very difficult to get across even to scientists and engineers.

Ecological forecasting models: Accuracy versus decisional quality

We consider here forecasting models in ecology or in agronomy, aiming at decision making based upon exceeding a quantitative threshold. We address specifically how to link the intrinsic quality of the model (its accuracy) with its decisional quality, ie its capacity to avoid false decisions and their associated costs. The accuracy of the model can be evaluated by the [Greek symbol rho – I don’t know what they mean by this just from reading the abstract] of the regression of observed values versus estimated ones or by the determination coefficient. We show that the decisional quality depends not only of this accuracy but also of the threshold retained to make the decision as well as on the state of nature. The two kinds of decisional errors consists either in deciding no action while an action is required (false negatives) or to act while it is useless (false positives). We also prove that the costs associated to those decisions depend also both of the accuracy of the model and of the value of the decision threshold.

Ecological Modeling

September 2019 in Review

Leave a reply
Most frightening and/or depressing story: Most hopeful story:
  • I think Elizabeth Warren has a shot at becoming the U.S. President, and of the candidates she and Bernie Sanders understand the climate change problem best. This could be a plus for the world. I suggested an emergency plan for the U.S. to deal with climate change: Focus on disaster preparedness and disaster response capabilities, the long term reliability and stability of the food system, and tackle our systemic corruption problems. I forgot to mention coming up with a plan to save our coastal cities, or possibly save most of them while abandoning portions of some of them in a gradual, orderly fashion. By the way, we should reduce carbon emissions and move to clean energy, but these are more doing our part to try to make sure the planet is habitable a century from now, while the other measures I am suggesting are true emergency measures that have to start now if we are going to get through the next few decades.
Most interesting story, that was not particularly frightening or hopeful, or perhaps was a mixture of both:
  • I mentioned an article by a Marine special operator (I didn’t even know those existed) on how to fix a broken organizational culture: acknowledge the problem, employ trusted agents, rein in cultural power brokers, win the population.

special operations culture

Leave a reply

This article by a Marine special operator says special operations have a culture problem. That doesn’t surprise me too much. Anyway, here is the prescription the author gives for addressing an organization’s culture problem:

  • Acknowledge the problem. It’s hard to spot a slow change from within an organization. One solution is to have a peer organization do a review.
  • Employ trusted agents. These are sort of the blue collar leaders.
  • Harness and rein in the cultural power brokers. These are more like the middle management.
  • Win the population. This is an idea for counter-insurgency where you try to peel the bulk of the population away from a few bad actors within their ranks.

The article mentions “core values”. My own observation about core values is that strong, well-functioning organizations tend to already have them implicitly, and when you have to make a big deal about training people in them explicitly your culture is already lost. I’m not sure you can change individuals’ core values all that much. You can try to weed out people with bad ones and bring in people with good ones.

value of learning curves in climate change planning

Leave a reply

This article gives an example of how to put an economic value on climate change adaptation incorporated in a larger planning framework.

The Economic Value of Climate Information in Adaptation Decisions: Learning in the Sea-level Rise and Coastal Infrastructure Context

Traditional methods of investment appraisal have been criticized in the context of climate change adaptation. Economic assessment of adaptation options needs to explicitly incorporate the uncertainty of future climate conditions and should recognise that uncertainties may diminish over time as a result of improved understanding and learning. Real options analysis (ROA) is an appraisal tool developed to incorporate concepts of flexibility and learning that relies on probabilistic data to characterise uncertainties. It is also a relatively resource-intensive decision support tool. We test whether, and to what extent, learning can result from the use of successive generations of real life climate scenarios, and how non-probabilistic uncertainties can be handled through adapting the principles of ROA in coastal economic adaptation decisions. Using a relatively simple form of ROA on a vulnerable piece of coastal rail infrastructure in the United Kingdom, and two successive UK climate assessments, we estimate the values associated with utilising up-dated information on sea-level rise. The value of learning can be compared to the capital cost of adaptation investment, and may be used to illustrate the potential scale of the value of learning in coastal protection, and other adaptation contexts.

gamification and water planning

Leave a reply

This article is about gamification and water planning.

A review of water-related serious games to specify use in environmental Multi-Criteria Decision Analysis

Serious games and gamification are nowadays pervasive. They are used to communicate about science and sometimes to involve citizens in science (e.g. citizen science). Concurrently, environmental decision analysis is challenged by the high cognitive load of the decision-making process and the possible biases threatening the rationality assumptions. Difficult decision-making processes can result in incomplete preference construction, and are generally limited to few participants. We reviewed 43 serious games and gamified applications related to water. We covered the broad diversity of serious games, which could be explained by the still unsettled terminology in the research area of gamification and serious gaming. We discuss how existing games could benefit early steps of Multi-Criteria Decision Analysis (MCDA), including problem structuring, stakeholder analysis, defining objectives, and exploring alternatives. We argue that no existing game allows for preference elicitation; one of the most challenging steps of MCDA. We propose many research opportunities for behavioral operational research.

leading implementation of complex programs

Leave a reply

This is just something I have wanted to write down a few thoughts on for awhile. My field is engineering and planning, and I have been involved in a number of programs that are complex technically, financially, and on the people side. I’ve seen some things done well, I’ve seen some things done badly, and I’ve done a few things well and learned a few lessons the hard way myself. So here are my thoughts:

  1. Organize the entire program around achieving a vision and set of goals which everyone understands. Create a crystal clear vision and set of goal statements for the program. Make sure these are thoroughly understood by all senior and mid-level decision makers – communicate, market, train, drill, test – whatever it takes to make sure they get it. Then, set specific objectives for individual functional units within the organization, and for all individual staff members, that advance these goals, all these goals and only these goals. Make each objective SMART – specific, measurable, achievable, realistic, and time bound. Then track every individual’s and every unit’s progress towards meeting the objectives, and hold individuals and managers accountable for meeting their objectives.
  2. Make sure the knowledge level of the entire staff is up-to-date with industry standards and best practices, then encourage system thinking, creativity and innovation to advance the leading edge. Create a formal training and continuing education program for staff. Create a psychological “safe space” for discussion of ideas that are outside the typical daily functions of the organization. Organize talks, discussion groups, and other events. Bring ideas and speakers in from outside the organization. Encourage and reward staff to spend time reading and attending events outside the organization, then bringing back ideas and communicating them to colleagues. Be on guard for the development of group think, and actively encourage and reward the sharing of ideas that are new to the organization.
  3. Focus on communication of system behavior, risk, and other complex information. Continuously improve staff knowledge of communication approaches, strategies, and tools by weaving these into the training and innovation program. Bring in specialized staff with communication and visualization skills. Set up a specific job role, group or committee whose job it is to oversee communication approaches in all aspects of the organization.

scenario analysis

Leave a reply

Maybe this is not of interest to everyone, but I am always looking for new ways to analyze and communicate the results of alternatives and scenarios.

The diversity of socio-economic pathways and CO2 emissions scenarios: Insights from the investigation of a scenarios database

The new scenario framework developed by the climate change research community rests on the fundamental logic that a diversity of socio-economic pathways can lead to the same radiative forcing, and therefore that a given level of radiative forcing can have very different socio-economic impacts. We propose a methodology that implements a “scenario discovery” cluster analysis and systematically identifies diverse groups of scenarios that share common outcomes among a database of socio-economic scenarios. We demonstrate the methodology with two examples using the Shared Socio-economic Pathways framework. We find that high emissions scenarios can be associated with either high or low per capita GDP growth, and that high productivity growth and catch-up are not necessarily associated with high per capita GDP and high emissions.

infrastructure infrastructure infrastructure

1 Reply

Hillary wants to spend $250 billion on infrastructure. Bernie wants to spend $1 trillion. Infrastructure investment is good. Economists all say so. Politicians who care at least a little about reality all say so. The American Society of Civil Engineers says so. I happen to be a civil engineer so it is definitely good for me.

The idea is that the infrastructure we have in the ground now wears out gradually, and we almost never do enough maintenance to keep it in its original condition. So there is a constant loss of value to the economy. You can also think of infrastructure as reducing friction, transaction costs, wasted time, effort, and energy in the economy. If people and goods can get where they need to go efficiently and cheaply, they can do more productive work and produce more value each day with less waste. The same idea applies to getting energy and water around. Finally, there is the economic idea that when economic growth is below capacity, as it is now, you could pay people to dig holes and fill them back in again, and there would be a net economic gain. So any infrastructure investment, even if it is not optimal, has even more benefit and it is a sort of free lunch, something for nothing, two birds with one stone. And interest rates are so low that it makes sense for the government to borrow money, or even print it, and realize this almost automatic, guaranteed, magical return on the investment. Politicians who oppose infrastructure investment on debt or deficit grounds just don’t understand or don’t care about reality, or else they are telling a cynically calculated story to people who don’t.

Optimal, planned infrastructure investment would be even better, of course. We really don’t do any planning at the national scale in the United States. Maybe we are still trying to differentiate ourselves from the Soviet planned economy, but seriously, it is time to get over that particular hangover.

I like Elon Musk’s Hyperloop because it is a big idea and it asks if we should be considering something bold, big, and different, not to mention more efficient and cost-effective than the same old ideas from the 1950s! But we need to remember infrastructure is not just about transportation, it is about energy, water resources, food, commerce, trade, information, environmental quality and ecosystems. Infrastructure has an ecological footprint which needs to be measured and considered in decisions. Let’s think big, take a holistic, long-term look at the whole system, and ask what kinds of investments would be best. What kind of integrated infrastructure system do we think would make sense in the future, and what steps would we take today to get there? One of the hard things about infrastructure, though, is that it is long-lived. The technology available actually changes much faster than the infrastructure wears out, so blindly repairing and perpetuating old infrastructure ends up retarding the pace of change. A good plan needs to take a long-term view, but it has to be flexible enough to adapt to changing technology, environmental, climate and socioeconomic conditions during the course of its implementation. This is the essence of good planning, but it is hard for many of today’s hyper-specialized, local- and short-term-thinking professionals to pull off.

groundwater

Leave a reply

This paper in Water Resources Research is about global groundwater depletion and pollution, and how groundwater can be managed better.

With rivers in critical regions already exploited to capacity throughout the world and groundwater overdraft as well as large-scale contamination occurring in many areas, we have entered an era in which multiple simultaneous stresses will drive water management. Increasingly, groundwater resources are taking a more prominent role in providing freshwater supplies. We discuss the competing fresh groundwater needs for human consumption, food production, energy, and the environment, as well as physical hazards, and conflicts due to transboundary overexploitation. During the past 50 years, groundwater management modeling has focused on combining simulation with optimization methods to inspect important problems ranging from contaminant remediation to agricultural irrigation management. The compound challenges now faced by water planners require a new generation of aquifer management models that address the broad impacts of global change on aquifer storage and depletion trajectory management, land subsidence, groundwater-dependent ecosystems, seawater intrusion, anthropogenic and geogenic contamination, supply vulnerability, and long-term sustainability. The scope of research efforts is only beginning to address complex interactions using multi-agent system models that are not readily formulated as optimization problems and that consider a suite of human behavioral responses.

They get something important right here, which is that if you are formulating a question in a way that the answer can be “optimized”, you have probably defined the question much too narrowly. Water resources are one part of much larger complex natural and social systems. Modeling and technical analysis is important to pare the universe of all possible decisions down to a smaller set where each possible decision is close to “optimal” or efficient in the technical and economic senses. But then this information needs to be fed into a stakeholder or political process where a much wider range of factors can be considered and decisions made.

I am concerned that the current laser focus on “science, technology, engineering, and math” in education is pushing people too far down the path of expecting clear-cut technocratic answers to questions that have messy political and cultural dimensions in reality. All these subjects are good to study, but they need to pared with solid education in planning processes and tools, and an appreciation of systems in general.

planning theory

Leave a reply

This article in the Journal of Planning Education and Research (free for the month of February only apparently) is a nice review of planning theory. It amazes me that the profession of planning seems to be so unsure of itself, and yet has so many important theories and tools to offer to other disciplines. There is a lot of planning going on outside the small field of academically trained urban and regional planning. I like to think of planning as similar to mathematics – it’s a profession for a few, but its theories and tools are used every day by professionals across many fields. Many of us can do moderately complex math by ourselves, and we know we can call on the mathematicians and statisticians for help with the really complex stuff. Similarly, a lot of professionals like engineers and economists are entrusted with the keys to the planning machine. But often, we do it badly because we are not well trained in the theory and tools of planning.

Almost all professionals – planners, engineers, and economists at a minimum – would benefit from better education in general systems theory – what the building blocks of systems are, how they interact with their boundaries, and how their behavior over time is driven by their structure and interaction with boundary conditions, and how they can be manipulated to achieve desired outcomes. Among the professions, engineers and economists probably have the best understanding of systems today, but we tend to define the system boundaries, and the range of desired outcomes that can be achieved, much too narrowly. That is one place planners can come in – facilitating the interaction between technocratic problem solving being done by engineers and economists with the larger socio-economic and environmental context.

What I call “technocratic problem solving” here is essentially what the planners call “rational-comprehensive” planning. In my view, it works very well for the elements of systems that we understand well (managing water resources, food production, and employment, for example). Where it has come under criticism (for example, the failed “urban renewal” programs in the U.S.), I believe the problem is not in the approach, but rather applying the approach to systems we do not understand well has given us a false sense of precision and a false confidence, which has led to failure. A hybrid approach that works very well, in my experience with water resource and environmental planning, is to apply the rational-comprehensive approach to the parts of the system we understand well, and then feed the results into a stakeholder or political process that can deal with the social aspects of the system we understand much less well. Planners can play the critical role in making this process reach a functional outcome. This is how I like to think of the planning profession – as the critical glue that can hold together a coalition of engineers, economists, bureaucrats, businesspeople, interest groups, and members of the public into a coherent whole that can set a direction for our society, then continue to guide it with incremental course adjustments as we go forward.