Information Underload and Overload






 * "Before you become too entranced with gorgeous gadgets and mesmerizing video displays, let me remind you that information is not knowledge, knowledge is not wisdom, and wisdom is not foresight. Each grows out of the other, and we need them all." (Arthur C. Clarke; British science fiction author as well as author of a great many non-fiction books; 1917–2008.)


 * “...a wealth of information creates a poverty of attention...” (Herbert A. Simon; Nobel laureate, American political scientist, economist, sociologist, psychologist, and computer scientist; 1916-2001.)


 * "Everybody gets so much information all day long that they lose their common sense." (Gertrude Stein; American writer, poet and feminist; 1874–1946.)

Brief Introduction and Definitions
We are currently living in the Information Age. This document is about some educational aspects and implications of information. The three quotations given above suggest that information overload has long been a problem. Indeed, the Library of Alexandria, built more than 2,200 years ago, may have held as many as a half-million scrolls. And, they did not have a card catalog in those days! I consider this situation as an information overload and an information retrieval problem. The problem was exacerbated by the fact that the library was not "conveniently" located for most people. It could take weeks of travel for a scholar to get to the library.

Now (in 2016) my 128 gigabyte thumb drive that I purchased for about $40 can store several thousand times the content of the greatest libraries of the so-called ancient world. With an appropriate computer and software, I can easily search this database while seated in a comfortable chair in my home. And, with connectivity to the Internet, I can search the Web—now by far the world's largest library.

The Internet and Web represent a huge change in how we store and retrieve information. Now, instead of having to "go" to the library, in essence the library comes to us. Now, instead of major libraries being few and far between, a major library is available to all of us. Our educational system has not yet fully grasped what this change in access to information means in terms of what constitutes a "good" education. What is Information? The term information has quite a variety of meanings. The five-point cognitive understanding scale given below is sometimes called the Clarke scale, after Arthur C. Clarke (see the quote given above.)



Arthur C. Clarke Cognitive Understanding Scale.

Many people use the term information to stand just for the data, information, and knowledge part of this scale. Others use it for the data, information, knowledge, and wisdom part of the scale. Thus, information as in Information Age has different meanings to different people. This document discusses all five aspects of the Clarke scale.

My 2/25/2016 Google search of the expression information overload produced over 10 million results. Many people believe that one aspect of our current information age is information overload—too much information. A later part of this document argues that, while many people do suffer from information overload, the reality is that we have an information underload. Often people cannot readily find the information that they need to deal with a problem situation.

This may be due to poor information retrieval and use skills, but it may also be because the needed information does not exist, or because the needed information has not been put into a storage facility that allows easy location and retrieval of the information. It also may be because the information retrieval system charges more than the people want to pay to access and use the information.



In summary, we have the key issues:
 * Information retrieval and use skills. Through informal and formal education and practice a person can greatly improve their information retrieval and use skills. This is facilitated by learning to read and write, and it is also facilitated through gaining an understanding of the discipline areas in which one wants to retrieve and make use of stored information.
 * Availability of information. The Web contains a huge amount of information and is steadily growing in size. But, just think about the information stored in the heads of the nearly 7.5 billion people (2016 data) currently living on the earth. Memories not only fade with time, memory recall often produces incorrect information. Or, think about the fact that several billion people have smart phones and/or digital cameras, and the pictures they take constitute a huge amount of information that is relatively private and not readily available to most people.


 * Ease and cost of access to information. For the most part, the availability of free public libraries is less than 300 years old. There is substantial information available on the Web that is not available free. There is much information available in free public libraries that is not available for free use on the Web.

Storing Information in Useful and Usable Forms
It is easy to understand information in terms of printed or computerized documents containing text and pictures. This section will extend your insights into what constitutes information.

Consider a simple set of tools, such as the utensils we use for eating. They contain (represent) information of their inventors and designers. That is a key idea. Tools extend our physical and cognitive capabilities, and they embody the knowledge and skills of their inventors and developers. Some tools are much easier to learn to use than others. A skilled user of a particular set of tools has learned to effectively combine information inherent to and stored in the tools with knowledge and skills gained through instruction and practice. Humans exceed all other animals on earth in their ability to develop and use tools, and to pass knowledge about these tools on to future generations. In this observation, the term tool has a very broad meaning, so humans make use of fire as a tool, various animals make use of twigs and stones as tools, reading and writing are tools, eyeglasses are a tool, and a Smartphone is a tool.

I think of a tool and how to use it as a very important type of information storage and retrieval. Abraham Maslow said in 1966, "I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail." As humans develop more and better tools, they are faced by a growing challenge of educating people about how to (appropriately) use the tools.

For example, consider the time, effort, and practice it takes to become skilled in using a set of carpenter's "hand" tools. Over the centuries not only has the number of these tools grown, but they have been supplemented (and sometimes supplanted) by a wide range of power tools.

Think about the range and nature of human-developed tools that produced the Agricultural Age and changed humans from being hunter-gathers. At the beginnings of the Agricultural Age about 12,000 years ago, farming was very labor intensive. The development of hand tools, and much later use of animal power, slowly changed this situation. The genetic engineering done by farmers (for example, selection of the more productive plants and their seeds, and selection of the "better" animals for breeding) greatly increased yields. Over thousands of years, researchers have developed these seed and animal selection processes into the discipline we now call genetic engineering.

The discipline of agriculture has transformed large parts of the world. For example, employment in the United States has changed from being 90% in agriculture at the time of the American Revolution, to less than three percent today. Very roughly, this means the agricultural productivity in the U.S. has improved by a factor of about 30 over a period of about 240 years. We can do a similar type of analysis of the Industrial Age. In the U.S., peak employment in industrial manufacturing exceeded 50% of the labor force in the early 1950s. Continued improvement in automation and increasing levels of imports of such goods has driven this employment level down by a factor of three. Automation of a manufacturing processes can be thought of as developing and using tools that enhance human physical capabilities. Another way to think about this is that the tools become both physically more capable and "smarter." This concept has become more important as computers have become more "intelligent" and the industrial manufacturing tools have become increasingly computerized.

However, it is more difficult to determine a "improved by a factor of" for the Information Age. One often quoted measure is speed of travel. Walking or using a horse and wagon tended to be at a rate of about three or four miles per hour. Cars easily travel 15 to 20 times as fast, modern trains are somewhat faster than cars and jet airliners travel about 150 times as fast as a person on foot. However, this is only a modest part of the whole story. Consider living in a city that has good water, sewage, and electrical systems. What factor of improvement does this constitute over the past 250 or so years? Or, consider progress in photography, medicine, telegraph, telephone, audio recording, radio, refrigerators, television, air conditioning, and so on. We now take for granted a great many aids to our current quality of life that just did not exist at the start of the Industrial Age. My conclusion is that it is likely not meaningful to try to make a simple "increase by a factor of …" to summarize the Industrial Age.

Information Age
The year 1956 is often stated as the beginning of the Information Age. In that year, the number of workers in the United States with "white collar" jobs first exceeded the number with "blue collar" jobs. The mass production of computers had begun, but had not yet had a significant impact on employment. Since then, the memory size and speed of computers has increased by more than a factor of a million, as has the cost effectiveness of such machines.

Earlier in this document we mentioned changes wrought by progress during the Agricultural and Industrial Ages. These were very large. However, the basic underlying tool of the Information Age has improved by a factor of more than a million since it first became commercially available in the early 1950s. There are certain tasks that humans used to do by hand or by use of non-computerized tools/aids where the productivity of a worker has increased by a factor of more than a million since the start of the Information Age.

As a simple example, I keyboard the search expression Information Age into the Google search engine. My computer tells me that the search engine took .70 seconds to find about 4.6 million exact matches of my search expression. The computer system searched through a collection of billions of Web documents. Not only that, the Google computer system has ordered the results on the basis of a "secret" algorithm that takes into consideration about two hundred different aspects of the documents that it discovered. This ordering process is designed to provide the user with results that he or she may find most relevant.

In browsing the search results, the following article caught my eye:


 * Newitz, A. (5/01/2015). The Information Age is Over. Welcome to the Infrastructure Age. Gizmodo. Retrieved 2/27/2016 from http://gizmodo.com/the-information-age-is-over-welcome-to-infrastructure-1701477972.

Quoting from this article:


 * … the problem isn’t our gadgets. It’s that the future of consumer tech isn’t going to come from information devices. It’s going to come from infrastructure.


 * That’s why Elon Musk’s announcements of the new Tesla battery line last night were more revolutionary than Apple Watch and more exciting than Microsoft’s admittedly nifty HoloLens. Information tech isn’t dead — it has just matured to the point where all we’ll get are better iterations of the same thing. Better cameras and apps for our phones. Virtual Reality that actually works. But these are not revolutionary gadgets. They are just realizations of dreams that began in the 1980s, when the information revolution transformed the consumer electronics market.


 * But now we’re entering the age of infrastructure gadgets. Thanks to devices like Tesla’s household battery, Powerwall, electrical grid technology that was once hidden behind massive barbed wire fences, owned by municipalities and counties, is now seeping slowly into our homes. And this isn’t just about alternative energy like solar. It’s about how we conceive of what technology is. It’s about what kinds of gadgets we’ll be buying for ourselves in 20 years
 * Once you accept that the thing our ancestors called the information superhighway will actually be controlling cars on real-life highways, you start to appreciate the sea change we’re witnessing. The internet isn’t that thing in there, inside your little glowing box. It’s in your washing machine, kitchen appliances, pet feeder, your internal organs, your car, your streets, the very walls of your house. You use your wearable to interface with the world out there.
 * Once you accept that the thing our ancestors called the information superhighway will actually be controlling cars on real-life highways, you start to appreciate the sea change we’re witnessing. The internet isn’t that thing in there, inside your little glowing box. It’s in your washing machine, kitchen appliances, pet feeder, your internal organs, your car, your streets, the very walls of your house. You use your wearable to interface with the world out there.

Hmm. It may well be that a few years from now people will look back and say that the Information Age ended in 2015, and will have coined a name for the next "Age." The following book uses the name, The Second Machine Age:


 * Brynjolfsson, E., and McAfee, A. (2014). The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. New York: W. W. Norton.

Futurists make predictions of when computers will become more intelligent than people. They call this the singularity, and their estimates range from perhaps as little as 40 years to hundreds of years.

From an information point of view, computers are steadily growing more intelligent. This progress will have a profound impact on humanity. Our current K-12 educational system may well be educating many students who will live to see the singularity. Even if this event does not occur during their lifetimes, they will certainly experience computerized robots and other intelligent machines that can do more and more of the things that humans can do.

How Much Information and How to Store It
During my adulthood, computer speed, storage capacity, and cost effectiveness have improved by a factor of many millions. This section provides a little information about how much information is being collected and a possible future technology that will aid in its storage and retrieval.

First how much information? Quoting from a 2011 IBM report(which, of course is now considerably out of date):


 * Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. This data is big data.

In the United states, the word quintillion refers to the number 1 followed by 18 zeros. It is a billion billion. That is approximately a billion bytes a week for every person on earth. A billion bytes is roughly equivalent to a thousand full-length novels. So in total, this is about the equivalent of a thousand different novels for each person on earth—per week!

The following article is used to help discuss technological change:


 * Gu, M., Cao, Y., & Gan, Z. (6/20/2013). How to Fit 1,000 Terabytes on a DVD. Science Alert. Retrieved 6/28/2013 from http://www.sciencealert.com.au/news/20131906-24498.html.

The article reports on recent progress in developing a DVD of greatly increased storage capacity. We are talking about really large numbers. The following may be useful to you as you read the short quote given below. If you have learned the metric system, you know that a Kilometer is 1,000 meters. The computer industry instead uses the preface Kilo to mean 1,024, which is 2 raised to the 10th power.


 * 1 Kilobyte = 1,024 bytes. This is approximately 1,000 bytes.


 * 1,024 Kilobytes = 1 Megabyte. This is approximately 1,000,000 bytes, which is approximately the number of bytes in a 250-to-300 page novel.


 * 1024 Megabytes = 1 Gigabyte. This is approximately 1,000,000,000 bytes.


 * 1024 Gigabytes = 1 Terabyte. This is approximately 1,000,000,000,000 bytes or a million 250-to-300 page novels.


 * 1024 Terabytes = 1 Petabyte. This is approximately 1,000,000,000,000,000 bytes. When the Large Hadron Collider is running, its computers are processing about one petabyte of data per day. Think of a petabyte as one billion books.

Quoting from the article:


 * We live in a world where digital information is exploding. Some 90% of the world’s data was generated in the past two years. The obvious question is: how can we store it all?


 * In Nature Communications today, we, along with Richard Evans from CSIRO, show how we developed a new technique to enable the data capacity of a single DVD to increase from 4.7 gigabytes up to one petabyte (1,000 terabytes). This is equivalent of 10.6 years of compressed high-definition video or 50,000 full high-definition movies.

Here is a somewhat odd piece of information. How Many Movies Are There provides an estimate that there were about 260 thousand feature-length films produced from 1888 to 2017. It would take about six of these yet-to-be-produced DVDs to store this many films.

On 2/25/2016 I did a Google search trying to find more up-to-date information on the work described in the article given above. A 9/3/2014 article, 1000TB on a CD indicated Dr. Gan (third author) had been awarded a Fellowship to help fund his continuing work. But, I did not find information about how soon (if ever) this technology will produce a marketable product.

Generally speaking, it takes a long time for a successful research development to become a widely distributed product. Quoting from David Moursund's 2012 article, The Pace of Technological Change:


 * Even in Silicon Valley, it takes most technologies 20 years to become overnight successes,” says Mr. Saffo, a consulting professor at Stanford’s school of engineering.

Problem Solving
Problems and problem solving are part of the background needed in understanding the topics of information overload and information underload. We use computers to store and process information, and we use information to help us represent and solve problems.

Posing, understanding, representing, and attempting to solve problems lies at the heart of each academic discipline. Problem solving includes:


 * * Question situations: recognizing, posing, clarifying, and answering questions.
 * * Problem situations: recognizing, posing, clarifying, and then solving problems.
 * * Task situations: recognizing, posing, clarifying, and accomplishing tasks.
 * * Decision situations: recognizing, posing, clarifying, and making good decisions.
 * * Using higher-order critical, creative, wise, and foresightful thinking to do all of the above.

Sometimes the question, problem, task, and decision situations are driven by "idle" curiosity, but often they are driven by a purpose of some sort. A four-year-old child asks, "Why is the sky blue?". This is just one of a steady stream of "why" questions one hears from young children.

A more mature scientist who asks, "Why is the sky blue?" may be seeking information about how moisture in the atmosphere polarizes light rays from the sun, or perhaps whether "blue" is roughly the same range of wavelengths for most of the people in a country or most of the people in the world.

Reread the last item in the bulleted list. It focuses on higher-order cognitive processes such as critical thinking, creative thinking, wise thinking, and foresightful thinking. From an informal and formal education point of view, we want students to become better in the higher-order thinking aspects of problem solving.

The discussion of information overload and underload in this document focuses of the higher-order thinking aspects of our educational system.

GIGO: A Major Difficulty
Data, information, knowledge, wisdom, and foresight are all really important ideas for all of us. Even if we don’t always consciously think about these ideas, throughout the day we all make use of them. As an example, think about data. It seems like a simple enough concept. For example, someone designs and carries out a scientific experiment. During this process, data is collected, stored, and analyzed. It may be analyzed in real time (that is, as the experiment is proceeding) in order to help people and machines make quick decisions about processes going on in the experiment. Some or all of the data may be stored over a long period of time and made available to others who want to analyze the data to help test or develop theories, or to make data-driven decisions.

But, what happens if a recording instrument is faulty, and "incorrect" data is gathered? Similarly, data can be recorded incorrectly. Or, data can be deliberately falsified and presented to the world as if it were "correct" data.

How good is your brain at remembering telephone numbers, email addresses, and Web addresses? Probably not so good. In all three cases, a small error produces an incorrect connection. Fortunately, we have writing, electronic devices, and other aids to circumvent this common mental frailty.

Next, think about two different people observing the same event. Each is gathering data by the use of his or her senses. But, data gathered through one's senses is being processed through one's brain. Some of it is stored in short-term memory, and some of that is eventually stored in long-term memory. The collection, processing, storage, and eventual retrieval operations are not exactly the same in any two brains. No two brains/minds are exactly the same, and constructivism (building on current knowledge) plays a significant role in the overall processes.

As a somewhat different kind of incorrect data or information example, consider Newton's laws of motion. Newton believed these were correct, and eventually a huge number of people came to believe that Newton's laws of motion were correct. Much later, the work of Albert Einstein and others in the field of relativity proved that Newton's laws of motion are not correct.

Physics and many other disciplines have theories that they make use of, but that may well be incorrect or not applicable in the situation in which they are being used. As the Newton's laws of motion example illustrates, even a highly mathematical and research-based discipline such as physics has this difficulty. It is easy to see why disciplines such as the social sciences and education face major difficulties as they try to make sound decisions, and attempt to have good foresight about the consequences of these decisions.

What about information that comes from processing, interpreting, and understanding stored data and information? It is easy to see how incorrect information can result from the situations described in the previous paragraphs. Computer people have coined the statement, "garbage in, garbage out" (GIGO) to talk about the difficulty produced when computers are given incorrect information to process. Just because a computer produces a result does not mean the result is correct. GIGO also applies to human brain processing of information.

In brief summary, data, information, knowledge, and wisdom that one accesses from one's own brain, from the brains of other people, from hard copy storage media, and from computers and other electronic storage media are not necessarily correct. We routinely make decisions under conditions of uncertainty, a topic discussed later in this document.

The following book discusses the types of information sources most used by students and their teachers.


 * Moursund, D., and Sylwester, R., eds. (10/9/2015). Validity and Credibility of Information. Eugene, OR: Information Age Education. Download a free Microsoft Word file from http://i-a-e.org/downloads/free-ebooks-by-dave-moursund/275-validity-and-credibility-of-information/file.html. Download a free PDF file from http://i-a-e.org/downloads/free-ebooks-by-dave-moursund/277-validity-and-credibility-of-information-2/file.html.

The book focuses on helping students learn to determine the credibility and validity of the information they retrieve and use in their lives.

The next three subsections discuss biased information, misinformation, and disinformation. These three types of information add to the difficulty of making good decisions and having good foresight into the potential results of decisions that one is making.

Biased Information
The above discussion suggests that, in some sense, data and information are either correct or incorrect. That is too simple a way to look at this situation. My recent Google search of the expression biased information produced more than 50 million results. For example, a person may be biased against people of a different race or residents of a different country, and this bias can lead to misunderstanding and misrepresenting information about the race or country. We receive considerable amounts of biased information through the advertising and marketing media, the news media, and other sources.

Quoting from a 2004 medical article by Thomas Kramer:


 * There is increasing concern and disdain among practitioners of psychopharmacology about information provided to us from the pharmaceutical industry as a function of their marketing efforts. Some practitioners will not talk to pharmaceutical representatives for fear of receiving biased information. I struggle with this attitude. Although it is clearly true that the accuracy and integrity of information varies and some sources are better than others, I have always found that more information is better when one is trying to make decisions about anything, including prescribing psychotropics. I have never been impressed that ignorance is bliss, or even useful or protective of something. To a certain extent, all information is subject to some bias from its source. The only questions are how much bias, and what it is biased toward or against. [Bold added for emphasis.]

Misinformation
This presents still another very important difficulty. You have undoubtedly heard the term misinformation—something that has the look, feel, and "smell" of information, but is incorrect and misleading.

Misinformation is an ongoing part of each person's everyday life. It is not just the steady stream of often misleading advertising that we are subjected to. It is also information transmitted among people in their everyday conversations. You have heard the expressions white lie, shading the truth, and biased information. You are aware of people routinely being biased in the manner in which they interpret data and information, and then represent their own biased opinions as "true, correct information."

The types of analysis given above for data and information also apply to knowledge and wisdom. So, think about the problem of educating students in a world where much of the data, information, knowledge, and wisdom they are routinely exposed to is flawed. How does one educate students to effectively deal with this situation?

Disinformation
Disinformation is an important aspect of information underload and information overload. My 2/27/2016 Google search of the expression disinformation produced nearly 3 million results. Quoting from the Wikipedia:


 * Disinformation is false or inaccurate information that is spread deliberately. It is synonymous with and sometimes called Black propaganda. It may include the distribution of forged documents, manuscripts, and photographs, or propagation of malicious rumors and fabricated intelligence. Disinformation should not be confused with misinformation, information that is unintentionally false.


 * In espionage or military intelligence, disinformation is the deliberate spreading of false information to mislead an enemy as to one's position or course of action. In politics, disinformation is the deliberate attempt to deflect voter support of an opponent, disseminating false statements of innuendo based on the candidates vulnerabilities as revealed by opposition research. In both cases, it also includes the distortion of true information in such a way as to render it useless.


 * Disinformation techniques may also be found in commerce and government, used to try to undermine the position of a competitor. It is an act of deception and blatant false statements to convince someone of an untruth. Cooking-the-books might be considered a disinformation strategy that led to the Sarbanes-Oxley Act.


 * Unlike traditional propaganda and Big Lie techniques designed to engage emotional support, disinformation is designed to manipulate the audience at the rational level by either discrediting conflicting information or supporting false conclusions.


 * Another technique of concealing facts, or censorship, is also used if the group can effect such control. When channels of information cannot be completely closed, they can be rendered useless by filling them with disinformation, effectively lowering their signal-to-noise ratio and discrediting the opposition by association with a lot of easily-disproved false claims.


 * A common disinformation tactic is to mix some truth and observation with false conclusions and lies, or to reveal part of the truth while presenting it as the whole (a limited hangout). [Bold added for emphasis.]

Biased information, misinformation, and disinformation are all important aspects of GIGO. People making decisions based on such forms of information may well make wrong decisions.

Foresight
The top of the data, information, knowledge, wisdom…chain is foresight. It is different from the first four topics in the Clarke scale. You use foresight to attempt to predict or forecast the results of actions that you are contemplating. In predicting possible outcomes from a proposed activity, you can draw upon your full range of informal and formal education, training, and experience. We are all futurists, practicing our forecasting skills routinely throughout the day.

If you are in a situation that calls for immediate or relatively quick action, you are left to your own devices. You do not have time to accurately communicate the situation and your possible actions to other people who might be able to add their foresight and make helpful suggestions to you. Neither do you have time to access information retrieval tools for assistance in making your decision.

But wait. How about airbags and collision avoidance radar in cars? These are instruments (tools) that have a type of foresight and can take very quick action when necessary. These two examples illustrate that people have learned to design and build some automated foresight and action-taking tools.

For a more frequently occurring situation, think about modern street corner crosswalk lights. They contain count-down timers along with a combination of visual and audio signals. You, the walker, make use of this information in real time as you make a decision whether to start across the crosswalk and/or how quickly to walk. Likely you have confidence in the correctness of the information being provided to you. However, you still use caution because you know that drivers sometimes don’t obey the traffic light rules.

If a decision you need to make does not need to be made quickly, you can draw upon the collected data, information, knowledge, wisdom, and foresight of others. In brief summary, we want to educate students to:


 * 1) Make wise, foresightful decisions very quickly when the need calls for it.


 * 1) Learn to use one's own insights and to draw upon the collected data, information, knowledge, wisdom, and foresight of others (including what is available in storage systems such as the Web) when time permits.

In addition, we want to develop machine aids (tools) to decision-making and action-taking that supplement the capabilities of people. Nowadays, Information and Communication Technology often plays a role in designing such tools.

Triangulation
Triangulation refers to the idea of making use of multiple sources of information and checking each source against the other. While the word triangle may suggest making use of exactly three sources of information, the term triangulation in the context used here typically means to make use of two or more sources of information. “Or more” might well be way more than three sources.

As you draw on multiple sources of information, work to understand the various viewpoints being presented and learn to look for flaws and inconsistencies among the various sources.

In solving any problem or in making any decision, you can draw upon your own accumulated data, information, knowledge, and wisdom, and your own foresight capabilities. You, yourself, are one part of the multiple sources of information used in triangulation.

Adults certainly want (and expect) children to develop and make use of "common sense." But, common sense is not so easily achieved. In many cases it takes rather deep understanding of a problem situation to have the insights needed for reliable common sense. Good common sense tends to develop through transfer of learning from years of life experiences and problem-solving experiences.

Many people "bounce their ideas" off of trusted friends, colleagues, mentors, advisers, counselors, and other groups of people to gain their insights. Crowd sourcing has gained in importance. The Internet is a major aid in the process of crowdsourcing.

Peer Review
In each academic discipline, scholars make use of peer review in evaluating their scholarly publications. Roughly speaking, this means that research articles appearing in the "refereed" journals have been carefully read by about three scholars in the discipline before they are published.

As illustrated in the picture, grant proposals to various foundations and government-funded agencies usually undergo careful peer review. In these types of competitive settings, the more highly rated proposals are the ones that receive funding.

Somewhat similarly, academic books being published by reputable publishing companies first undergo review by experts in the discipline. A publishing company may decide not to publish a particular manuscript based on the feedback from its reviewers. Of course, there are vanity presses where an author pays to have a book published, and there is the self-publication process. Thus, there are many scholarly-looking books in print that have not undergone peer review.

Consider the document you are now reading. It has not undergone peer review. Thus, you should be suspicious of its contents. You can look into my credentials, and they might increase your confidence in this document. I am a retired professor, having worked in two different math departments, two different computing centers, one computer science department, and one College of Education. I am the author or co-author of more than 60 books). My academic credentials, extensive publications, and a variety of other scholarly activities have helped to establish my professional reputation.

Here are four more review ideas that may be of particular interest to teachers:


 * 1) In team-based project-based learning, members of the team provide feedback to each other on their individual work and writing. Teachers can organize this into a reasonably careful team-based peer review system.
 * 2) In many project-based learning and other situations, students do relatively formal presentations to the whole class. The class members can be educated to provide a type of peer review.
 * 3) Students can learn to read each others' written assignments and provide peer review to their classmates. See, for example, http://serc.carleton.edu/introgeo/peerreview/index.html.
 * 4) A teacher reading a student's homework or viewing students in class participation is in a position to provide useful review or feedback. Thus, for example, a language arts teacher can provide feedback on the depth and quality of the arguments in a paper. A math teacher can provide feedback on the depth and quality of the "show your work" aspects of student seatwork and homework.

In summary, it is very important that students learn to understand and to use the general concept of peer review. Students can learn to give and to accept high-quality analysis and feedback about the correctness and quality of their work and the work of their classmates.

Decision Making Under Uncertainty
The previous sections of this document are intended to help convince you that in our everyday lives we are regularly called upon to make decisions under conditions of uncertainty.

A recent Google search of the expression decision making under uncertainty produced nearly 2.4 million hits. Indeed, this topic is now an area of study and research and one can become a high-level expert in the area. Here, for example, is a list of research concentration areas from the reference given above:


 * Basic methods based on probability, decision and utility theories
 * Techniques for bounding the effect of missing and/or incorrect information
 * Trading time and space resources with certainty
 * Fusing uncertain information of different kinds
 * Real-time inference algorithms
 * Hybrid dynamical systems
 * Machine learning algorithms
 * Causal reasoning
 * Problem structuring for optimal understanding by human decision makers
 * Analysis and evaluation of courses of actions
 * Cognitive aspects of interaction between human decision makers and automated decision-aids
 * Methods for evaluating the effectiveness of automated decision aids

Risk analysis is an important aspect of decision making under uncertainty. What are the risks involved in making a particular type of decision? To give you a feeling for the depth and complexity of risk analysis, here is a quote from the Harvard Center for Risk Analysis:


 * The Harvard Center for Risk Analysis (HCRA) is a multidisciplinary group of faculty, research staff, students, and visiting scholars who work together to improve decisions about environmental health and other risks. We conduct state-of-the-art research, educate the next generation of leaders in risk analysis and related disciplines, and encourage public discourse about risk topics.
 * Our newsletter, Risk in Perspective, periodically communicates the results of our research in a short and accessible form that emphasizes its policy relevance. Explore past issues below by following the links to the complete issues.
 * Our newsletter, Risk in Perspective, periodically communicates the results of our research in a short and accessible form that emphasizes its policy relevance. Explore past issues below by following the links to the complete issues.

Your brain is designed for decision making under conditions of uncertainty. With appropriate informal and formal education, it can become increasingly successful. This situation presents our educational system with an interesting challenge. How do we educate students to become better at decision making under uncertainty? How do we educate students to recognize when they are facing uncertainty in their decision-making situations? It is important for them to understand that through appropriate research and information retrieval they can reduce the uncertainty and therefore be more likely to make better decisions.

Note that the previous paragraph is suggesting a significant change in our educational system. It proposes a substantial decrease in rote memorization of what to do in certain specified situations. Instead, it would increase students' learning how to make better decisions based on one's current knowledge and skills, combined with the added information one can get from other sources (including the Web) at the time when one needs to make an important decision.

This proposed educational change is not a new idea. Quoting from a 2/23/2015 article by Randy Bass:


 * Our experiences suggest in practice what broader research suggests in theory: If we want to cultivate adaptive and flexible thinkers, then we have to structure experiences for students that put them in positions to engage in intellectual discovery and exercise what Lee S. Shulman called “judgments under uncertainty.” This must be done repeatedly, from the earliest opportunities.


 * There’s nothing simple or determinative about how new media technologies enable these kinds of learning experiences, but their inherent openness, complexity, and ubiquity should make them an inescapable part of our designs for essential learning.

In brief summary, here are two educational suggestions:


 * 1) Help students to understand the uncertainty and the risks involved in many of the decision-making situations they encounter throughout the day. The vocabulary and the basic idea of decision-making under uncertainty should become part of each student’s knowledge and understanding about the world in which they live.
 * 2) Routinely provide students with problem-solving and decision-making situations that involve uncertainty. In each case, help students learn about ways to decrease the uncertainty and risk.

Information Underload
Many people complain that they are suffering from information overload. Some time ago, I responded to a reporter who was writing an article on this topic. I thought about the information overload idea for a while, and suggested a contrary point of view.

I propose that we are not suffering from information overload. Rather, we are suffering from information underload. We often do not have ready access to the specific information we need to answer questions, solve problems, and accomplish tasks we want to accomplish. If we had lots more information and better ways to retrieve the specific information that we need, then the so-called information overload would be substantially decreased.

This response did not go over well with the reporter, and I never heard from him again. However, I have continued to think about my somewhat contrary point of view.

I find it useful to think about information underload from the point of view of the question, "What should we teach in schools?" Our current approach is perhaps best summarized by, "When in doubt, add it to the list of what is to be taught." I think of this as an "appease the interest groups" approach to curriculum redesign. The result is that the curriculum is too full. There is not enough time for reflective practices. There is not enough time for students to pursue independent investigations.

In most courses or units of study, almost all of the available time is used up in "covering required curriculum content." This is one sign of an information overload approach to education, an approach that tends to be governed by the idea that more is better. Hold this (bad) idea in mind as you read the section on Information Overload that follows this current section.

Meanwhile, think about the following three ideas to help deal with information underload. These are suggestions for teachers.




 * 1) As you help your students learn a discipline, give them specific instruction and quite a bit of practice in making use of the Web and other information storage systems in retrieving information that is relevant to what they are studying. That is, focus on the idea that building on the previous work of others is one of the most important ideas in problem solving. The recommendation is to spend more time helping students develop and hone their information retrieval skills.
 * 2) Within any discipline that you teach, look for places where a computer system or other automated systems can solve or greatly help in solving the problems you want students to learn about. This is a suggestion that less time should be spent on learning to do things that a computer can do much better than people.
 * 3) Consider adopting and teaching the mantra, "Less is more." This reminds me of one of my favorite quotes:


 * "I have made this letter longer than usual, only because I have not had the time to make it shorter." (Blaise Pascal; French mathematician, physicist, inventor and writer; 1623-1662.)


 * Here are two more quotations of a similar ilk:


 * "Simplicity is the ultimate sophistication." (Leonardo da Vinci; Italian polymath known for his painting, invention, sculpting, science, math, etc.; 1452-1519.)


 * "Art is the elimination of the unnecessary." (Pablo Picasso; Spanish painter, sculptor, painter, and playwright; 1881-1973.)

Information Overload
Here is a way to think about the issue of information overload:


 * 1) I am constantly bombarded with information from ads, spam and other junk mail, blogs, email from my acquaintances, posting in the social networking systems I belong to, phone calls, text messages, and so on. In some sense the whole world seems to conspire against me by continually interfering with or intruding on my time.
 * 2) When I need specific information, I often make use of the Web. The trouble is, my search engine frequently identifies many thousands (sometimes millions) of results. I have to use a lot of time to find the specific information I need, and sometimes I cannot find it. In the latter case, it is often because I don’t know how to formulate a search expression that uses the correct words to find what I want to find.

Lots of people have these two problems, and these problems continue to persist even as the discipline of computer and information science continues to make remarkable progress.

My 2/28/2016 Google search using the expression information overload produced over 10 million results. The following short article provides a nice introduction to the topic:


 * Ingebrightsen, N. (n.d.). Understanding Information Overload. Infogineering. Retrieved 2/28/2016 from http://www.infogineering.net/understanding-information-overload.htm.

Quoting from this article:


 * Information Overload is an increasing problem both in the workplace, and in life in general. Those that learn to deal with it effectively will have a major advantage in the next few years.


 * Information Overload is when you are trying to deal with more information than you are able to process to make sensible decisions. The result is either that you either delay making decisions, or that you make the wrong decisions.


 * It is now commonplace to be getting too many e-mails, reports and incoming messages to deal with them effectively.


 * The Information Overload Age


 * The first recorded use of the phrase “information overload” was used by the futurologist Alvin Toffler in 1970, when he predicted that the rapidly increasing amounts of information being produced would eventually cause people problems.


 * Although people talk about “living in the information age,” written information has been used for thousands of years. The invention of the Printing Press a few hundred years ago made it possible to distribute written information to large amounts of people. However, it is only with the advent of modern computers that the ability to create, duplicate and access vast amounts of information has created Information Overload amongst the general population.


 * The root of the problem is that, although computer processing and memory is increasing all the time, the humans that must use the information are not getting any faster. Effectively, the human mind acts as a bottleneck in the process.

One way to help students to approach the problem of information overload is to teach Big Ideas (essential concepts) and teach for transfer of learning. Teaching and learning Big Ideas lies at the heart of the Common Core State Standards. This is discussed in the following article:


 * McTighe, J., and Wiggins, G. (12/4/2012). Common Core Big Idea Series 1: A New Blueprint. Edutopia. Retrieved 2/28/2016 from http://www.edutopia.org/blog/common-core-new-emphases-jay-mctighe-grant-wiggins.

Jay McTighe and Grant Wiggins have long been world-class leaders in education. McTighe stresses teaching Big Ideas. Wiggins stresses the role and importance of authentic assessment. They have coauthored a number of books and articles.

Information Appropriate Load
Consider an Information scale with one end labeled Overload and the other end labeled Underload. Let's label the middle of the scale Appropriate Load. This reminds me of the Goldilocks story in which the bowls of porridge were too hot, too cold, and just right; the beds were too hard, too soft, and just right.

For a particular person faced with a particular information need, there might be too much information available, too little information available, or an appropriate (for the person) amount of information available. Moreover, the information might be written at a level that is way over the person's head, way under the person's reading and understanding levels, or appropriate to the person.

Educators are familiar with this situation. A dictionary with a quite limited number of words, and containing illustrative pictures and simple definitions, is appropriate to a student just learning to read. Books for students at different grade levels have readability levels appropriate to average students at those grade levels.

Now, think about this situation from the point of view of the creators and writers of the Wikipedia. They envisioned a collection of articles (information) that would eventually exceed that in any current print encyclopedia and that articles would be authored by many different writers. What readability level should it have? What background knowledge of the subject of a particular article should the writer of the article assume the readers will have?

Every writer faces the challenge faced by authors of Wikipedia entries. Every educational system faces the challenge of meeting the information needs of a very wide range of students with different interests, background knowledge, and reading/viewing skills.

Retrieving Just the Information One Needs
As noted before, part of the information underload problem is that often one cannot find the needed information, even in cases where it exists.

The various Web search companies and many other groups are working on this problem. What is emerging is a three-pronged approach.


 * 1. Increase the breadth and depth of information available of the Web. Develop more intelligent search engines that order the search results in a manner that better fits the needs of people using the Web. Currently Google is the most widely used of such search engines.


 * 2. Develop answer engines that are designed specifically to produce answers to problems (including questions) posed by users. Two important examples of this  are:


 * * Wolfram Alpha, a computational knowledge engine. This is an online service that answers factual questions. It comes in a limited free version and a (not free) professional version.


 * * IBM's Watson. Watson gained fame by defeating two past champions in the TV question and answer program Jeopardy in 2011.


 * 3. User-specific search, answer, and teaching engines. Such a system would "know" a great deal about what the question poser knows and would provide information and answers that are individualized to that person. Moreover, such a system views a question as a "teachable moment" and uses it to provide not only the desired information but also appropriate instruction to help the question poser learn more about the topic area.

Here are some general ideas that support #3 above. When a student asks a teacher or parent a question, the teacher or parent takes into consideration the student's current knowledge and understanding in providing an answer.

Our schooling system is designed around the idea of providing "age appropriate" and "developmentally appropriate" instruction. Of course, in a large class instruction environment, it is not possible to provide a high level of individualization of instruction.

On the other hand, a parent can provide a high level of individualization of instruction, but may not be particularly knowledgeable in the content (and appropriate pedagogy) of the topic being asked about.

An individual tutor who knows both tutee and the subject area the tutee is working on can often do better than either a teacher dealing with a class or a parent dealing with a child.

Thus, the "gold standard" in information retrieval is a system that combines search engines and answer engines, incorporates specific knowledge of the person posing a question, and can carry on an interactive conversation with that person.

Progress is occurring in the development of computer-based versions of such information retrieval systems. However, we are a long way from having computer systems that can outperform a human tutor who knows the tutee and the subject area, and who has access to the current search engines and answer engines.

Information Overload in Teaching and Learning
Each academic discipline supports the work of a number of professionals who do research and publication in the discipline. Each discipline has scholarly journals that help in the preservation and dissemination of the steadily growing accumulation of data, information, knowledge, and wisdom.

Moreover, each academic discipline has teachers and researchers who accumulate pedagogical content knowledge and share it through a variety of means. There is a large and steadily growing collection of such knowledge that an overlap between the content of a particular discipline and general methodologies for teaching. My 2/27/2016 Google search of the expression pedagogical content knowledge produced over a half-million results.

Thus, each discipline has both information content overload and a pedagogical content overload. A teacher in any discipline has to deal with both of these types of overloads.

A Math Education Example
Let's examine the discipline of mathematics to help illustrate the above ideas and the teaching/learning difficulties faced by the discipline.

First, the totality of math information and knowledge grows steadily. Feel free to skip the paragraph quoted below—it is just a bunch of large numbers. How much mathematics is there? Quoting from the linked article:


 * … the American Mathematical Society’s MathSciNet database, which has a slightly wider coverage and also includes books and conference proceedings, lists 81,142 [new] items in 2012; 99,588 [new items] in 2011; 98,281 [new items] in 2010; and 98,945 [new items] in 2009. It seems that somewhere between 50,000 and 100,000 maths papers are being published every year — and according to MathSciNet, that’s about three times as many as in the early 1970s when Ulam made his estimate. Assuming that the number of theorems per paper hasn’t changed significantly, we could perhaps estimate that between 300,000 and 600,000 theorems are now published every year… Oh, and as a final set of staggering numbers, Web of Knowledge lists a total of 1,342,406 mathematics papers since 1900, while MathSciNet lists 2,888,464 entries for the same period…

We can thus see that math is a deep, broad discipline that is accumulative and is continuing rapid growth. Consider math education from an information point of view. For an individual student and a particular topic being taught, what constitutes an Appropriate Load (as distinguished from Overload and Underload) of information to present and of supplemental information to be made available? What information and skills do we want the student to commit to long-term memory? What information and skills do we want the student to be facile in use of—and dependent on—calculators, computers, the Internet, and the Web?

I like to make use of the following Expertise Scale as I think about and discuss these types of questions.



For a particular student—with a particular current level of: math knowledge and skills; math-related cognitive abilities; and interest in math—what should be the instructional content goals, pedagogy, and assessment?

These are difficult questions and different groups of people come up with different answers. For example, is the United States (or the entire world) producing enough doctorates in mathematics who will go on to be productive math researchers? Is the United States (or the entire world) preparing its graduates of K-12 schools to deal adequately with the math they will routinely encounter in their everyday lives as responsible adult citizens? (This last question is not meant to address the "higher level" math requirements in many different jobs and professions.)

At any particular math coursework level, be it kindergarten or graduate school, there are the issues such as: breadth versus depth; memorization versus understanding; and the emphasis to be placed on teaching and learning for use in disciplines outside the discipline of math (transfer of learning).

The solutions that our math education programs are currently implementing provide a good example of information overload. In the U.S., this approach to math education is often criticized as making use of a math curriculum that is a mile wide and an inch deep.

Here is a simple example. The concept of measurement of area is important both in "pure" math and in applications of math in many different disciplines. There is considerable agreement that elementary school students need to develop an understanding of the concept of area in a plane—area of a rectangle (including the special case of a square), and area of a right triangle. Of course, students can learn the names or memorize formulas for calculating the area of other triangles as well as for parallelograms (and the special case of a rhombus), trapezoids, kites, regular polygons, circles, and so on.


 * Here is an aside. I smile to myself as I consider the need for an ordinary adult to be able to name and distinguish between two plane figures, one being a parallelogram and one being a trapezoid. A similar smile results as I think about the issue of identifying right, equilateral, isosceles, and scalene triangles. How often does an ordinary person need to be able to name such figures, and calculate their perimeters and areas? And, what about the area or circumference of a circle? We have the Big Idea that various types of figures in a plane have areas and perimeters. These figures have been studied for more than two thousand years.

Math is a discipline in which many of the problems people need to solve today can be solved by computers or by computerized equipment. Our math education system now acknowledges this situation. However, our our traditional and still continuing approach to math education continues to be one in which students "cover" a lot of math, but they do not gain the long-term retention, knowledge, and understanding to effectively deal with many of the math-related problems in their lives.

We especially see this in a widespread inability to deal with finances (borrowing, saving for the future, interest rates and compound interest, and other aspects of dealing with money), and statistics. We see this in the number of our students who learn to hate math and/or claim that they just can't do math. We see this in the inability of many of our students to learn math by reading math, and to take advantage to the readily available aids to learning and doing math such as the Web, calculators, and computers.

A History Education Example</Center>
The totality of accumulated historical data, information, and knowledge is steadily growing. Many people have memorized the quotation:


 * "Those who cannot remember the past are condemned to repeat it." (George Santayana; 1863–1952.)

Pause for a minute, and think about what this statement means to you. Does it mean that memorizing names, dates, and places will help you to avoid problems that you are likely to encounter in your life? Or, do you view the study and understanding of history in a deeper fashion?

Many students think of history education as the process of memorizing dates, names, places, and so on. Historians, however, think in terms of big ideas and connections such as causality, legacy, responsibility, and investigation.  The memorization approach is fraught with the difficulties of information overload and considerable waste of student and instructional time. Students memorize, regurgitate, and forget. The second approach requires deeper understanding and a higher level of cognitive maturity.

Suppose that a school system is taking the first approach. To take a specific example, with the inauguration of Barack H. Obama, the United States has had 44 presidents. Is it important for a student to memorize the name and date of presidency for each of these presidents? What about the names of their spouses, children, and pets? How about the vice presidents, members of the Cabinet, and the important political figures throughout the world that each president dealt with?

For example, The Presidents website is maintained by the U.S. government, it and contains a Web page about each of the presidents. That is quite a bit of data, information, and knowledge that a person might memorize, but is a tiny drop in the bucket relative to what is actually available. Moreover, how accurate is this particular website? Does it contain incorrect information, misleading information, or disinformation? What important information has been left out?

You can see the problem. Like math, history is a very deep and broad discipline. If you live in the United States, then it seems reasonable that you should know that the United States has a form of democracy in which a president is elected every four years. But, what should you know about the various presidents? That is, what should you carry around in your head and retrieve often enough so that the data, information, and knowledge stays readily retrievable?

The issue is Big Ideas versus an accumulation of facts. Both are important, so a good history education system needs to achieve an appropriate balance between the two. What constitutes an appropriate balance will vary with the cognitive abilities and interests of the students.

There have been a number of interesting and perhaps amusing tests of successes and failures of history education. For example, a 2006 and 2007 study by the Intercollegiate Studies Institute (ISI) American Civic Literacy Program indicated:


 * OUR FADING HERITAGE: Americans Fail a Basic Test on Their History and Institutions is the third major study conducted by ISI on the kind of knowledge required for informed citizenship. In 2006 and 2007, ISI published the first ever scientific surveys of civic learning among college students. Each year, approximately 14,000 freshmen and seniors at 50 schools nationwide were given a 60-question, multiple-choice exam on basic knowledge of America’s heritage. Both years, the students failed. The average freshman scored 51.7% the first year and 51.4% the next. The average senior scored 53.2%, then 54.2%. After all the time, effort, and money spent on college, students emerge no better off in understanding the fundamental features of American self-government.
 * Earning a college degree does little to increase knowledge of America’s history, key texts, and institutions. The average score among those who ended their formal education with a bachelor’s degree is 57%, or an “F.” That is only 13 percentage points higher than the average score among those who ended their formal education with a high school diploma.
 * Earning a college degree does little to increase knowledge of America’s history, key texts, and institutions. The average score among those who ended their formal education with a bachelor’s degree is 57%, or an “F.” That is only 13 percentage points higher than the average score among those who ended their formal education with a high school diploma.

Here are the first two questions from the 33-question quiz. As you read and answer these questions, think about their importance to a "responsible" adult citizen in the United States.

1)  Which of the following are the inalienable rights referred to in the Declaration of Independence?


 * A. life, liberty, and property
 * B. honor, liberty, and peace
 * C. liberty, health, and community
 * D. life, respect, and equal protection
 * E. life, liberty, and the pursuit of happiness

2)  In 1933 Franklin Delano Roosevelt proposed a series of government programs that became known as:


 * A. the Great Society
 * B. the Square Deal
 * C. the New Deal
 * D. the New Frontier
 * E. supply-side economics

As you can see, this is a memory-based test. It is unrelated to the issue of whether the test takers have learned to approach history from the points of view of causality, legacy, responsibility, and investigation.

The same type of study can be replicated for any "slice" of American history and for any large group of students or non-students. Such a study will always produce results indicating that people don't know much history. Similar types of studies will show that people don't know much XXX, where XXX is any academic discipline.

The results of such studies tend to be used to place still more emphasis on rote memorization in our schools. Such an information overload approach cannot succeed in improving our educational system, because the totality of accumulated information is both huge and is growing rapidly in every academic discipline.

Thus, in effect, our history education system leaves most students in an information underload situation. The over emphasis on an information overload-oriented approach to history education does not leave enough time for most students to master tools needed when one has an information underload in history.

Final Remarks
Reading, writing, and arithmetic (math) became formal subjects in schools more than 5,000 years ago. Since then there has been a steady increase in the accumulated knowledge of the human race. Quoting from a 2013 article by David Schilling, the pace of this increase has been increasing rapidly:


 * Buckminster Fuller created the “Knowledge Doubling Curve”; he noticed that until 1900 human knowledge doubled approximately every century. By the end of World War II knowledge was doubling every 25 years. Today things are not as simple as different types of knowledge have different rates of growth. For example, nanotechnology knowledge is doubling every two years and clinical knowledge every 18 months. But on average human knowledge is doubling every 13 months. According to IBM, the build out of  the “internet of things” will lead to the doubling of knowledge every 12 hours.

Historically, our educational system's approach to this situation can be summarized by ideas such as:


 * 1) Increase the length of the school day, the length of the school year, and the general requirements of school attendance. (A few hundred years ago, a third grade education was a significant achievement. Now, we are unhappy that only 80% of students in the U.S. graduate from high school.) Indeed, we are pushing hard on the idea that all students should be prepared to go on to post-secondary education and that most should do so.
 * 2) Increase the certification requirements to be a teacher and increase the student requirements to graduate from grade school, middle school or junior high school, and high school.
 * 3) Improve the quality of the aids to teaching, learning, and assessment.
 * 4) Teach reading, writing, and arithmetic (math) and hope that learners will then make use of these basic skills to retrieve and/or figure out the information that they need to deal with their personal problems and tasks. Note that this "learn by reading across the curriculum" is running into serious problems because today's youth are reading far less books and other non-electronic "traditional" print materials than did the previous generation. Reading Twitter blog entries and instant text messages is much different from reading and understanding the types of materials that are stressed in a traditional education.
 * 5) Lament over the inadequacy of our schools, as the gap between what various stakeholder groups believe students should know and be able to do continually widens from what students actually know and can do. It is very easy to find examples that make schooling's "good old days" seem much more successful than today's results.

The reality is that these approaches have helped a lot. We have fought the good fight and made significant progress. However, we are losing the war! That is because both the quantity and complexity of the new data, information, knowledge, and wisdom is growing so rapidly.

We need to modify our educational system according to some of the ideas listed below:


 * 1) Help all students to understand that a major goal in education is to help them become better at recognizing, clearly stating, and solving a wide range of problems.
 * 2) Help all students to gain a good understanding of decision making under uncertainty, and to recognize a variety of sources of uncertainty relevant to the decisions they are making.
 * 3) Help all students learn to make use of multiple sources of information (triangulation) as one way to help decrease the uncertainty resulting from incorrect, misleading, biased, and other poor quality data, information, knowledge, and wisdom.
 * 4) Help all students to gain a significant level of expertise in information retrieval “across the curriculum” and in the specific discipline areas they are studying.
 * 5) Help all students to gain an appropriate level of expertise in use computer and other aids to solving problems.

Author or Authors
This website was created by David Moursund.