Improving Decision Making Skills through Business Simulation Gaming and Expert Systems
University of Nebraska, Omaha firstname.lastname@example.org
Abstract Business simulations as experimental learning tools are common, but they usually train specific predetermined aspects. Research on artificial intelligence among business simulations is rare, and therefore, featured in this paper. The purpose of this research is to explore the use of business simulations games as an experimental learning tool through a contemporary, web-based application featuring artificial intelligence and mobile support. An expert system guides and advises the players, while they manage their virtual business in a competitive market against other participants. The core element is the design process of an artifact, based on the Design Science methodology. The training and learning effects on the participants are observed via the artifact itself in a series of experiments and an additional survey. Twenty-six students in Austria were chosen as the sample group to reveal and measure the improvements in decision making, experimental learning capabilities and the biasing ability of the artificial intelligence. 1. Introduction
Today the decision-making process within organizations is increasingly complex. All decision makers in businesses require basic understanding of organizational structure and how business elements influence each other. In universities effective work is done by providing students with the necessary knowledge about business concepts like production optimization, marketing, strategies, human resource management, and so on. But the theoretical knowledge is rarely put to practice. Avramenko  finds that the educational process in business schools fails to equip students with employability skills.
Business simulation games encourage teamwork and decision-making, in a risk-free environment . Players develop a holistic view of the business, they learn that sometimes alternatives have to be considered and that losses in an early stage might
lead to higher profit in a later stage. Business games and simulations became popular over the last 20 years; and they differ in complexity, focus, settings or intentions. They are web or application-based and can include random elements.
This research aims to design such a business simulation, which allows multiple players to train their management skills in a competitive environment. No perfect utilization can be reached only by the player’s actions; other players are influencing the participant’s outcome as well.
Another core element of this research was to provide a setting where an expert system can take a substantial and useful part in such a simulation game. The idea was to develop a virtual “mentor”, which acts as an advisor and biases the human player in his or her decisions. Therefore, the primary research objectives are:
How can a business simulation game be constructed, in which… � …human players can improve their strategic
management skills through decision-making in a competitive environment.
� …an intelligent agent (IA) acts as an advisor to improve the learning effectiveness of the players’ skill improvement.
A goal was to prove that participants can improve decision-making through interacting with a business simulation with other human participants (competitors). In order to achieve these objectives, a scope and balance for the business model had to be found. The simulation must feature enough complexity to completely cover all major elements of a manufacturing company but can’t be too complex, otherwise the learning effect can no longer be measured effectively.
A manufacturing company was chosen as business environment because the basic value proposition is more straightforward than for example, the business process of a service provider. In its very basic form, the value creation process of a manufacturing company is to purchase resources, produce goods and then sell them.
It was also essential to study the effects of integrating an intelligent agent that would act as an advisor to the participants.
2016 49th Hawaii International Conference on System Sciences
1530-1605/16 $31.00 © 2016 IEEE DOI 10.1109/HICSS.2016.107
To what extend can an artificial advisor bias the decision making of the participants?
As a part of the Design Science methodology, an artifact is created through which knowledge is generated based on the Information Systems Research Framework described by Henver et.al. Identifying the environment of this artifact is necessary; otherwise, it might lead to an inappropriate design and undesirable side-effects . 2. Literature Review
Simulation gaming has been used as a tool by researchers and industry for more than 50 years. Many simulations for educational purposes are without doubt accepted as useful learning tools; however, qualitative simulation games are hardly developed with a specific scientific purpose. Design Science provides an ideal supplement to align such simulations to the scientific community. 2.1. Design Science Methodology
To study and analyze behavioral decision making in a business environment Design Science has proven the most appropriate approach for this research. It can be described as a problem-solving paradigms which seeks to create an innovative construct in order to generate knowledge about a phenomenon .
While research is an activity that contributes to the understanding of a phenomenon, in Design Science the phenomenon can be created artificially, and need not occur naturally . In Design Science, the reality is replaced with an artificial construct that contributes to answer the research problem. This is especially useful, when the reality is not suitable or too costly to be studied directly.
Using this approach allows for a second significant advantage: part of this research is to learn about human perception of an intelligent agent. This agent is already an artificial construct and can therefore be directly integrated into the artifact. To ensure Design Science is science and not only design, it is necessary to prioritize the production of useful knowledge ahead of the production of the artifact.
Based on the Information Systems Research Framework by Henver et.al. the business environment is served by IS research through developed instruments. In order to add value to the knowledge base of the scientific community the designed artifact(s) has to be justified and evaluated .
In this case, the organizational strategies are the area of interest in the business environment and they are highly influenced by external factors. Market, suppliers, legal regulations and competence availability all affect a business and are constantly interacting across the borders of the business. The question becomes which business environment components are effectively relevant for improving the learning process when considering a holistic view of such a manufacturing business model, and how such a learning process can become effective through a scientific artifact like a business simulation.
Creating this artifact also has an impact on Information Technology. Modern web tools fortunately provide an ideal framework to create an artifact like that required for this research. Graff states that computerized hypertext provides an explicit structure to the material being learned, which is advantageous to learners and encourages them to engage and move around . Using web technologies also allows use of extensive, open-source frameworks, bringing functionality like dynamic charts, sortable tables, animations, form elements and more. To ensure a contemporary instantiation, the simulation has to fully support the majority of modern smartphones.
Researchers in Design Science have to be careful to support their research goals through an unbiased design of the artifact. Lainema described this problem as a challenge of providing acceptable scientific research to the academic community rather than just presenting own opinions . It can be difficult for a simulation designer to distinguish between failure and success due to the fact that no one designs a simulation to fail . 2.2. Approaches to Simulation Gaming
Thavikulwat defined a simulation as an exercise of real activities in an artificial environment and a game as an experience, featuring competition and rules . The most discussed topics in business simulation education and learning in recent years are : � Experience accumulation � Strategy aspects � Decision-making experience accumulation � Learning outcomes � Teamwork experience
Simulations follow different patterns, and they can be classified according to their characteristics. Thavikulwat classified simulations using computers
based on who takes control and how interaction takes place .
Basically, two different types of simulations can be distinguished: continuous and discrete simulations . The gaming industry has a more straightforward terminology: real-time and round-based games.
Specifically, concerning the learning aspect, much research has been performed. Simulation-based training is inherently more engaging than other training methods. Knowledge can be gathered more quickly and efficiently, and simulations are simpler to operate but provide a more complex, realistic and manageable learning environment .
Besides all positive advantages, simulation research is often criticized for measuring the affective and not the cognitive learning, resulting in challenged legitimacy and a lack of firm conclusions .
Business games are often accidently considered the same as management games. It is assumed that this equation is only true when the game actually features the management of e.g. a firm, organization, portfolio, teamwork or other factors in the business area . Therefore, business games are games with a business environment leading to the training of players in business skills or the evaluation of players’ performances.
The first known use of games for educational purposes dates back to about 3000 BC in China [2, 7]. The acknowledged beginning of business games in the modern sense started in Europe with Mary Birshstein in 1932, who had the idea of adapting the concept of war games to the modern business environment .
The artifact in this study builds on a competitive, interactive, deterministic total enterprise mechanism, in which manufacturing acts as the theme. It is played by individuals against each other, supported through the computer. The simulated time frame is split in turns or ‘periods’.
Business games have established a reputation as serious alternative methods for teaching managerial skills. Global organizations are the industry drivers and act as meeting points for researchers and practitioners . Despite proof that simulations are valid teaching instruments, researchers agree that the full potential of business simulation games is not yet revealed [9, 12, 13]. 2.3. Expert Systems and Artificial Intelligence in Simulation Gaming
Artificial Intelligence (AI) can be described as a broad interdisciplinary field, which may cover elements from computing disciplines to mathematics, linguistics, economics, neuroscience and many others
. It is sometimes difficult to decide what really belongs under the domain. While AI is far more capable in information processing than a human being (as long as the information can be transformed into digital language), it lacks the ability to ‘think’ creatively or critically.
An expert system is computer logic, designed to gather knowledge and make decisions comparable to a human expert. Mostly artificial intelligence is used for processing data into useful knowledge.
AI adaptations in computer games are as old as computer games themselves. “Nimatron,” a machine capable of playing the game “Nim,” was one of the first computerized games, developed in 1940 . Until the 70s, AI could only be found in computerized two-player games like checkers or chess . Later in the 70s, arcade games popped up, featuring single-player gaming against computer- controlled enemies . In the 90s AI was assigned with complex tasks like dealing with incomplete information, path-finding or real-time decision making and economic planning .
Intelligent agents (IA) mark the class of AIs that are often used in simulation gaming and are therefore of special interest for this research. Tecuci developed a comprehensive definition of an intelligent agent : “an agent is a knowledge-based system that perceives its environment […] and acts upon that environment to realize a set of goals or tasks for which it has been designed.”
Summers adds, that an IA is defined by its ability to determine its own behavior . Artificial agents can be complex, combining multiple different decision-making models. The model of a knowledge- based agent shows how the agent acts with the environment, as can be seen in Figure 1:
Figure 1. Main modules of a knowledge-based agent / expert system 
Modern intelligent agents are capable of learning from the players’ behavior and switch their tactics accordingly. Also phenomena from reality are transferred into computer games. StarCraft 2 oriented their unit movement after the flow of water in a stream; if there is an obstacle in the path, the water
flows with reduced speed around the obstacle, considering the available space through the environment and other water particles.
Simulations have been the testing environment for many AI applications like neural networks or crowd simulation [19, 20]. AI in educational business simulations has mostly taken coaching roles so far, and there is a trend towards the integration of AI as decision-support and expert systems .
Based on a set of input variables, a knowledge base and a processing algorithm(s) output is generated. Knowledge-based agents may additionally support complex modules like a learning or problem solving engine.
Artificial intelligence continues to be an emerging domain in computer science. Given the capabilities of AI and intelligent agents, much progress in simulation-based gaming and learning is expected. Contemporarily, AI research benefits from advancements coming from the highly profitable gaming industry . 3. The Artifact – Architecture and Design
According to the guidelines for design science in IS research , the artifact has to be innovative and purposeful. The web-based business simulation suggested in this research has allow students to improve their decision-making skills in an intuitive way. To demonstrate the utility, quality and efficacy of the simulation, methods have to be implemented to allow for a serious evaluation. This is done by measuring the effective improvements of participants (feedback through simulation) and their perceived improvements (feedback through participants).
The Business Simulation for this research is designed for desktop and mobile browsers and provides a simple interface to give participants control over various aspects of their business (Figure 2). A simulation game consists of four participants competing in a closed market. The ultimate goal is to finish with the highest accumulated revenue over the period of ten timed rounds in which participants can decide on eight different corporate strategies.
Figure 2. The Business Simulation in Action
The simulation provides the participants with a lot of information on their and the competitors’ businesses, mostly derived from charts and tables. Besides that, the artificial intelligence tries to provide feedback on advantages and disadvantages of recognized strategies. In each round (period), the same decisions are available, but an ideal strategy is impossible to predict or achieve since the users have to deal with incomplete information, and the competitors’ actions for the current turn can’t be predicted. After the simulation, an extensive analysis is provided, showing all the decisions made along with additional charts and tables. 3.1. Decision-Making
Decision-making in the Business Simulation is designed with the goal to increase usability to a maximum through a simple, intuitive interface. Learning should happen by understanding the impacts of decisions on the business, not through dealing with numbers and complex mechanisms. Typical organizational strategies build the core of the simulation and they usually feature two extremes which are mutually exclusive. For example managers can focus entirely on product A or B or they have to split available resources among them. Through a slider, users can declare if they fully support strategy A, strategy B or if they are indifferent. Figure 3 shows the mechanism of such a slider:
Figure 3. Decision-making Through Sliders
Every slider has five stages (increments), and there are a total of seven such strategic decisions that have to be made. The simulation is designed to allow advantages and disadvantages for each position on the spectrum. It is possible that a specific decision is favorable at an early stage in the game, but the opposite strategy is beneficial at a later stage. Additionally, players can toggle on/off a competitor analysis as the eighth decision. The artifact was designed to provide a holistic set of realistic decisions, and every decision features advantages and disadvantages. A major challenge in designing the artifact was balancing out the strategies, to give each strategy validity at some point, and to give them
serious consideration in specific situations. Once a decision is altered, it is immediately updated in the database. Since the simulation is purely discrete and there are no random elements, only the decisions have to be stored in the database. Calculations on outcomes like revenue can be done on demand, which simplifies the data structure and increases the performance of the app. 3.2. Timing and Tutorials
The business simulation is time controlled. Users who are done with their decisions before the time has run out have the opportunity to end their turn prematurely. If all four participants finish early, the simulation immediately processes to the next round.
Each of the ten rounds is limited to two minutes, after which the game processes to the next round automatically. The first round makes an exception, for which the time limit is five minutes to allow new players more time to get familiar with the simulation elements. This feature has been aligned with the tutorial, which is only displayed in the first round. Instead of the charts, which wouldn’t be of much use in the first round anyway, the players experience tutorial-like text information. This information aims to explain what can be done in each section and what is important. 3.3. Artificial Intelligence (“Eddie”)
The AI is implemented as an expert system using deductive rules to observe the market and decisions of the player as well as critical variables resulting from decisions in previous rounds. It is integrated as an ‘advisor’ and it has a face and a name (“Eddie”).
Figure 4. Charts, Tables and Decision Feedback
As knowledge base all variables associated to the player decisions from the current and last round are taken in consideration. This includes:
� Player Decisions � directly calculated outcomes (e.g. costs of
employees based on amount, salary and amount of extra hours)
� Indirectly calculated / implied outcomes (e.g. a resulting market share cumulated over all previous rounds)
The most appropriate feedback is determined when the engine is loading all the game variables at the beginning of a new round. An inference engine then generates the outcome in form of subjective feedback to the player. The feedback is a brief verbal statement, either a warning, suggestion or information on current issues the player might look into. The engine compares and evaluates the relevance of a total of 30 different options. During several pre-experiments these 30 feedbacks were identified as most fitting and helpful based on strategic mistakes the players made. The inference engine works by determining and assigning a priority value for the relevant feedback statements independently from other statements. High priority results are then compared by including the relevant conditions and variables from the other high-priority statements with a reduced impact and a new priority value is generated. The statement which has the highest priority by the end of this second step wins, and is displayed to the player. This feedback is selected by best-fit to the current situation, and it is ensured that Eddie never gives the same advice twice in the simulation, should this mathematically happen, Eddie suggests the next best (not already displayed) feedback. He appears to have a personality and may be described as provocative, bold, funny or pushing. His behavior and capability to bias participants is one of the primary research areas in this study. Feedback Examples (11 out of 30): � Pushing up your marketing expenses at the end
of the product life cycle is lost money! � Rapid expansion in the beginning might result
in huge personal costs and overproduction! � Focusing on one product is dangerous. Only do
it, when you see potential in this market! � Your moral is dangerously low. The
productivity in your firm is suffering! � You are producing more than you are selling.
Try do reduce production to save costs! � If you don’t start expending soon, you will get
� We are losing grip on the market. We need more sales!
� Only risk extra hours, if you need them and you can pay for them!
� We can’t compete with the prices of your competitors. Why not trying out some product development?
� Your quality is already amazing. Maybe it is time to increase the production and lower costs!
� You have a good moral! It might be save to ask for a little more of your employees.
3.4. Screens / Pages
The simulation is split into five tabs, each focusing on one department of the business. A lobby (before the game starts), a final score page (when the game ends), and a brief description of the implementation of the survey are also included to give a holistic view. 3.4.1. Lobby. The lobby serves as a starting point to direct the participants into the simulation. On this first page instructions and a chat are accessible. Header and Footer provide compact information: � Language Switch (German and English) � Timer � Actual Round / Total Rounds � Navigation Tabs
3.4.2. Progress / AI. This initial screen displays a selection of charts that shows, the actual progress in terms of revenue, turnover and cost. A simplified balance sheet is available, and it is also the place where Eddie, the AI, can be found.
3.4.3. Production. On this screen, the player can choose a production focus and the amount of product development. The company produces two different types of goods, products A and B. They have different resource demands, production times and selling prices. High product development increases the quality of the product and, therefore, the price for which it can be sold, but comes at the cost of a reduced production capacity. 3.4.4. Marketing. In this section, the user is able to influence the whole market. All players start with 10 shares (25% market share each). They can increase their amount of shares between 0 and 4 per round, depending on how much they invested into
advertisement and marketing. Additionally, to the variable individual increase, the market also has a natural lifecycle. The market demand increases with the total amount of shares and the market share defines how many products of this global demand customers are ready to purchase from the players. Should the player produce less than he or she can sell on the market, the remaining demand can be exploited by his or her competitors. The second decision the player can make here is to undertake a competitor analysis. This enabled additional information on competitors on various charts for the price of some variable costs. 3.4.5. Personnel. On this screen, players can handle two strategies concerning human resources. They can set a salary level and order extra hours to increase productivity. Both decisions influence the morale (effectiveness) of the workers. A high salary has the disadvantage of causing additional personnel costs, but is necessary if the morale drops low in the company. The morale is a general indicator for the productivity of each worker. Extra hours increased the production capacity but decreased the morale and caused additional costs. 3.4.6. Strategy. On this screen, players have the opportunity to make strategic corporate decisions. The first choice is between a rapid expansion over improving the business. Expansion leads to more employees, and therefore, to a higher production capacity. Too rapid expansion results in unaffordable personnel costs. Improving the business, on the other hand, simulates improvements in internal processes and reduces the time needed to produce products.
Users can also decide on a price or a quality focus. A price strategy (cost leadership) leads to discounts, making the purchase of resources more affordable, while a quality focus improves the quality of the products resulting in an alternative method to increase selling prices.
4. Evaluation and Verification
Referring back to the research problem, the first
four issues were dealt with by developing an artifact that is capable of training human players in decision- making in a competitive environment. The artifact was accessible from any device with internet access, and German and English were implemented as languages. An intelligent agent increased the learning potential further by advising the player on important decisions based on developments during the simulation. These four issues were addressed by
collecting feedback from participants in controlled experiments performed in two stages.
In the first stage, two business intelligence university courses with a total of 26 students, held at the Management Center Innsbruck in Austria, served as an environment for the experiments. The simulation was first explained to the students who then performed one game. The students were then led to a survey that included qualitative and quantitative feedback related to the simulation and AI. After the first simulation and the survey, the students were then randomly mixed and prepared for a second game. The purpose was to see if the overall performance in the second game increased through learning and adaptation of the game mechanics. Also, potentially interesting behavioral patterns were identified. 4.1 Survey
The survey served two main purposes: first, players were asked about good strategies for the early, middle and late game. The survey tested the artifact against the issues identified in the research problem. The responses were compared to the observed behavioral changes between the first and second round of experiments. This was done to verify if the players had understood and adapted the concepts of the business simulation. Table 1 shows the number of participants who evaluated the chosen strategy as strong during the early, mid or late game.
For example only one person felt that using a strong quality-focus (5) over price-leadership was a beneficial strategy in the early game. The market research strategy is excluded from the Table since it only had two states (on and off). Table 1. Efficient* Game Strategies
Decision Strategy / # participants Early Game (Round 1-3) 1 2 3 4 5 Product Development 1 3 3 11 8 Marketing 1 5 2 7 11 Salaries 7 11 5 2 1 Extra hours 10 6 5 5 0 S1: Improve / Expand 9 5 4 5 2 S2: Price / Quality 5 7 2 10 1 Midgame (Round 4-7) 1 2 3 4 5 Product Development 0 4 9 5 7 Marketing 0 5 11 8 2 Salaries 0 5 14 7 0 Extra hours 3 9 7 6 1 S1: Improve / Expand 2 8 4 8 2 S2: Price / Quality 2 6 6 9 2
Endgame (Round 8-10) 1 2 3 4 5 Product Development 10 5 3 5 3 Marketing 10 6 3 3 4 Salaries 5 2 10 8 1 Extra hours 7 7 5 4 2 S1: Improve / Expand 5 9 4 4 3 S2: Price / Quality 6 5 1 8 5 n=26; (*) participants evaluated with 4 or 5
Second, data about previous experiences and knowledge of business management, along with qualitative feedback related to aspects of the simulation was collected. The last screen included two questions about artificial intelligence to find out about its place in business intelligence and management. The survey asked the students for one positive and three negative observations they made during the experiments. The results were grouped and counted based on similarity. Table 2 shows the amount of individual feedback for the derived group:
Table 2. Qualitative Feedback on Simulation
Positive Feedback Count Artificial Intelligence 3 Charts 3 Competition 6 Design & Structure 7 General / Idea 4 Learning effects 4 Total (26 expected) 27 Negative Feedback Count Artificial Intelligence 1 Bugs in the Simulation 5 Calculations / Mechanics 4 Design & Structure 15 Explanations 13 Limited Time 7 Total (78 expected) 45 n=26; categories based on similarities in feedback
4.2 Simulation Decision Results
The survey and the simulation were both designed
to supplement each other. Since the survey asked the students for the perceived best strategies in early, mid and late game, the same aspect was analyzed from the actual performance of the participants.
Decisions from the first and second round of experiments were separated to find the behavioral changes influenced by adaptive learning and the impulse through the survey. Evaluating all the decisions turned out to be challenging. The limited
number of options for each decision helped to reduce complexity in the process. Clear changes in decision behavior were identified, both between the different stages in the simulation, as well as between the two
rounds of experiments. Table 3 shows these behavioral changes in terms of absolute participants, who have chosen the specific strategy:
Table 3. Simulation Data: Decision Strategies
Decision Experiments (First Round) Experiments (Second Round) Early Game (Round 1-3) 1 2 3 4 5 1 2 3 4 5 Competitor Analysis* 5 64 6 60 Production Focus 10 22 15 21 1 13 12 28 5 8 Product Development 10 21 19 17 2 13 14 11 13 15 Marketing 4 17 22 17 9 10 6 8 19 23 Salaries 0 35 32 2 0 9 29 16 7 5 Extra hours 21 38 10 0 0 27 25 12 2 0 S1: Improve / Expand 10 24 19 13 3 15 17 15 13 6 S2: Price / Quality 4 13 16 25 11 6 22 11 13 14 Midgame (Round 4-7) 1 2 3 4 5 1 2 3 4 5 Competitor Analysis* 22 70 6 82 Production Focus 11 21 31 20 9 14 15 40 13 6 Product Development 21 25 22 13 11 16 18 17 21 16 Marketing 16 19 17 17 23 17 17 13 25 16 Salaries 1 34 38 15 4 8 33 34 11 2 Extra hours 32 30 27 3 0 29 32 18 6 3 S1: Improve / Expand 11 26 26 21 8 17 18 29 18 6 S2: Price / Quality 5 27 23 28 9 10 29 30 8 11 Endgame (Round 8-10) 1 2 3 4 5 1 2 3 4 5 Competitor Analysis* 21 48 11 55 Production Focus 12 18 25 10 4 24 10 19 7 6 Product Development 19 10 20 14 6 18 13 7 10 18 Marketing 25 23 7 2 12 19 16 12 15 4 Salaries 4 24 31 7 3 18 12 24 7 5 Extra hours 30 19 11 9 0 18 20 17 7 4 S1: Improve / Expand 14 18 16 13 8 19 11 10 9 17 S2: Price / Quality 12 19 16 18 4 22 19 11 6 8 This table represents how players decided (1…5) in each stage of the game. The values are representing the absolute amount of occurrences for each strategy chosen. (*) Competitor analysis has only has two values, true (2) and false (1)
The competitor analysis was a feature used consistently by most players throughout all simulations. The decisions for production focus were more diversified. Producing Product A rather than B was noticeable at all stages.
A clear learning effect can be observed in the product development strategy. Players realized correctly that this strategy improved the total revenue, especially in the early and midgame, and changed their behavior in the second simulation. The marketing strategy showed an even superior learning effect. While players focused on an average marketing investment strategy during the first round of experiments, they reduced their efforts in the late game. An analysis of the salary showed that most decisions concerning the salary were set statically.
Using Extra Hours is a feature designed to generally have most effect in later stages of the game. This was recognized by the players. The final two strategies were more challenging to analyze. There was no ‘ideal’ way to handle these strategies. Players had strong and very different opinions on the usefulness of all strategies; the only thing they agreed on was that taking no preference is a disadvantage. Improving the business was considered a better alternative in the early game rather than expanding it. This changed in the midgame; in the endgame, the results were equal to those in the early game.
A similar result was identified in the second business strategy (Price / Quality). According to the survey, players believed either in the success of a price focus or a quality focus in the early game. In