SciELO - Scientific Electronic Library Online

vol.24 issue3 author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand



Related links

  • On index processCited by Google
  • On index processSimilars in Google


South African Journal of Industrial Engineering

On-line version ISSN 2224-7890
Print version ISSN 1012-277X

S. Afr. J. Ind. Eng. vol.24 n.3 Pretoria Nov. 2013




Use of simulation to solve outpatient clinic problems: A review of the literature



Tang Sai HongI, 1; Pah Pin ShangII, 2; Manohar ArumugamIII; Rosnah Mohd YusuffIV

IDepartment of Mechanical and Manufacturing Engineering. Faculty of Engineering. Universiti Putra Malaysia, Malaysia.
IIDepartment of Mechanical and Manufacturing Engineering. Faculty of Engineering. Universiti Putra Malaysia, Malaysia.
IIIDepartment of Orthopaedic. Faculty of Medicine and Health Sciences. Universiti Putra Malaysia, Malaysia.
IVDepartment of Mechanical and Manufacturing Engineering. Faculty of Engineering. Universiti Putra Malaysia,




The increasing demand for outpatient services has led to overcrowded clinics, long waiting times for patients, and extended staff working hours in outpatient clinics. Simulation tools have been used to ameliorate deficiencies in the appointment system, resource allocation, and patient flow management that are the root causes of these problems. Integrated studies that considered these three factors together produced better results than attempts to resolve individual causes. While simulation has proved to be an effective problem-solving tool for outpatient clinic management, there is still room for improvement. This paper reviews studies over the past 50 years that have applied management simulation to resolve outpatient clinic problems.


Die toenemende vraag na buitepasiëntdienste het gelei tot oorlaaide klinieke, lang wagtye vir pasiënte en verlengde werksure vir personeel in buitepasiëntklinieke. Simulasie sagteware is gebruik om tekortkominge in die afspraakstelsel, hulpbron toekenning en pasiëntvloeibestuur (die hoof oorsake van hierdie probleme) te versag. Geïntegreerde studies, wat hierdie drie faktore gesamentlik in ag neem, het beter resultate gelewer as studies wat individuele gevalle probeer aanspreek het. Alhoewel simulasie 'n effektiewe probleemoplostegniek vir buitepasiëntkliniekbestuur is, is daar steeds ruimte vir verbetering. Hierdie artikel hersien navorsing (wat strek oor die laaste 50 jaar) wat bestuursimulasie toegepas het om buitepasiëntkliniek probleme aan te spreek.




The global human population officially reached seven billion in 2013. This is the fastest billionth increase; the human population rose from six billion to seven billion in just 12 years [1]. The population boom, coupled with an increasing life span, poses great challenges to health care systems. And with the increase in non-communicable diseases in the older age group, the health care sector faces a daunting task to provide for those who need its services. The constant demand for cheaper service and yet improved health care service quality has become more pressing in present day society [2]. Funds provided for health care services become scarce, especially at times when the economy performs poorly, exacting an ever-increasing toll on health care services [3].

To reduce the cost of health care services effectively, an efficient health care management system needs to be implemented. The World Health Organization lists service delivery as a key component for a well-functioning health system [4]. Service deliveries in health care include treatment by professional health care personnel, timely access to the health service, and preventive care. Evaluation of service delivery is directly related to clients' satisfaction with health care services, which in turn is reflected in the service quality [5]. For example, excessive patient waiting time at clinics leads to poor client satisfaction [6]. This is, in fact, fast becoming a global problem [7]. It is therefore crucial that the time spent on health care for each patient is efficiently used to ensure patient satisfaction and effective cost reduction in health care services.

In the past ten years there have been two extensive reviews of simulation in health care [8,9]. However, the reviews are general and not specific to outpatient clinics. Sometimes the reader needs to focus not on the whole hospital, but on the outpatient clinic alone. Besides, the literature has increased over the past 10 years. Thus this review fills the gap and provides the reader with information about simulation specific to outpatient clinics.



Health care, as defined in the 1978 Declaration of Alma-Ata, is "essential health care based on practical, scientifically sound and socially acceptable methods and technology made universally accessible to individuals and families in the community through their full participation and at a cost that the community and country can afford to maintain at every stage of their development in the spirit of self-reliance and self-determination" [10]. In lay terms, patients receiving health care services are provided with treatment, diagnosis, and preventive care of sickness, physical injuries, or other ailments at a reasonable cost. Generally there are three levels of health care services; primary, secondary, and tertiary [11].

Primary health care is provided by community clinics around the country. It is an entry level health care service that offers initial consultation for patients. Primary health care generally includes family physicians and their associated staff. Where further screening and follow-up is required, patients are referred to secondary health care, which is provided by district hospitals. These hospitals usually have outpatient, inpatient, emergency, radiological, laboratory, and specialist services. Tertiary health care is usually provided by general hospitals, including some that are highly specialised. Patients are usually referred to these hospitals by secondary health care providers. In many hospitals there are modern facilities and sophisticated equipment for treating illnesses such as cancer and autoimmune diseases.

In this review, the focus is on the outpatient department in hospitals.



Whereas inpatient services are facilities that accommodate patients requiring hospital admission, the outpatient clinic serves patients who do not require overnight hospitalisation. While outpatient clinics or departments acted mainly as diagnosis and treatment centers some two to three hundred years ago [12], they are currently used for many other functions as well. Patients admitted for inpatient services are also screened in outpatient clinics; and patients who are discharged receive follow-up treatment in the outpatient clinic. These clinics also serve as teaching facilities for medical students, nursing students, and residents [13], and for health care research.

The outpatient department or clinic often acts as a specialist clinic in a general hospital. Specialist outpatient clinics include those dedicated to orthopaedics, surgery, paediatrics, ophthalmology, general medicine, and obstetrics & gynaecology. Patients with different problems consult specialists in the respective departments.



An aging population stimulates the demand for outpatient clinics [14] as the risk of non-communicable diseases increases with age [15]. Thus outpatient clinics become crowded, and supply cannot cope with demand. A congested clinic increases the work load of its staff and the chances of the clinic overrunning its usual sessions. Long working hours also demotivate the clinical staff, resulting in inefficient use of resources and time [16]. Besides, congested clinics result in longer waiting times - a major cause of patient dissatisfaction.

According to recent studies, there are at least five major causes of patient dissatisfaction with outpatient clinic services. Among them are dissatisfaction with the physician and the management, receptionist behaviour, and lengthy waiting times [17]. Of these, the patient waiting time is a major reason for patient dissatisfaction [18,19]. In a study involving a student health care centre, waiting time was the item that rated the lowest satisfaction for scheduled appointments and for walk-in clinics [20]. Patients who have to wait more than 15 minutes are consistently dissatisfied with the waiting time [18,21,22]. The longer they wait, the greater is their dissatisfaction [23,24]. In fact, when Manaf and Phang [25] tested the relationship between patients' satisfaction about the waiting time and overall outpatient satisfaction, they found the relationship to be highly correlated. Reduced waiting time would lead to overall satisfaction with outpatient services.

In short, the major outpatient clinic problems are the long patient waiting time and long staff working hours that result from a poor appointment system, poor resource allocation, and poor patient flow management in outpatient clinics.



Since the attempt of Bailey [26] and Lindley [27] 50 years ago to resolve outpatient clinic dissatisfaction, many operation research techniques have been developed to understand the underlying problems better. These techniques include the queuing theory, mathematical programming, modelling, and simulation [2,28].

In simulation and modelling, the queueing theory is used to understand and improve operations such as waiting time or queueing. Different queueing systems can be represented by queueing models such as M/M/s. Through derivation and analysis of queueing models, the performance of a system, such as the mean waiting time, can be inferred [29].

Besides improving the waiting time, the optimisation of problems in the outpatient clinic can be achieved through mathematical modeling [28]. Three principal mathematical programming methods are used to mitigate outpatient scheduling problems [2]: a) dynamic programming, b) nonlinear programming, and c) stochastic linear programming. (Refer to Cayirli and Veral [2] for a full review.)

Modelling is a structural method used to study real world issues by developing a simplified version of the problem [30]. This research method is used in health care to understand problems such as resource shortage and congestion in clinics. Modelling includes qualitative modelling, mathematical modelling, statistical analysis, statistical modelling, simulation, spatial/Geographic Information System (GIS) modelling, and the unified modelling language and integrated definition method [30].

Last but not least, simulation is a highly versatile modelling method. It can be performed manually or by computer to imitate the operation of a real world process or system over time [31]. A civil engineer or an architect uses physical simulations to imitate real structures or movement of a system [32]. However, this method is not used by hospital management, as it is almost impossible to simulate the physical model of the outpatient clinic because such modeling interrupts the health service's work. The computer-based simulation method uses computer software to imitate different real world systems or processes and to evaluate the predicted outcomes [32,31]. Discrete event simulation, system dynamic simulation, and Monte Carlo simulation are the three most common computer simulations used in health care [30]. Of these, the discrete event simulation is the most popular method.

Among the methods discussed, simulation and modelling using the queueing theory and mathematical programming are useful only for simple system studies. In reality, they often suffer because they make unrealistic assumptions and over-simplify the system [28]. As a result, the variables and uncertainties of the real world are not fully accommodated, leading to an inaccurate representation of the actual complexities of the outpatient clinic [28].



Among the approaches used to resolve problems in the outpatient clinic, simulation has the capacity to model complex outpatient clinic situations. Simulation has become more popular recently [30,33] because of its inherent advantages. For instance, simulation can describe a complex real-world system while accurately representing stochastic elements [34]. Simulation can accurately portray the operation of an outpatient clinic or other hospital department. Nevertheless, a deterministic operation research technique can only simplify the system using assumptions to represent the complexity of the outpatient clinics [28].

Simulation enables the management of hospitals and clinics to predict the outcome of a research query or an option to improve the health care delivery system [35,8]. Such accurate predictions in multiple situations enable hospital administrators to experiment with different management policies. This is very helpful, as real experiments are difficult to implement in reality without interfering with the normal running of the clinic [36]. Simulation can act as a forecasting tool where the performance of an existing system with changes in operating conditions can be evaluated. These changes include patient flow, human resources, and physical capacity. The complex relationship between the different model variables such as patient arrival rate or patient service rate can also be investigated [8,35].

End users of the simulation model are normally hospital administrators who are not operation researchers or industry engineers. Visual interactive simulations [37,38] enable the user to see the subject's movement on the computer, and help the user to understand, develop, and validate the system better [39]. Besides, a visual interactive simulation can be adopted easily by end users as a planning and management tool after test-running different variations of a project.

As summarised by Van Sambeek [32], simulation can present a good account of variability, with high end-user validation of the real situation mimic model. The result of this is a positive and dependable prediction of the true situation.



Previous studies have effectively adapted computer simulation to solve outpatient clinic problems and improve their service delivery quality. High quality service delivery must have an efficient and effective patient flow. This means that an acceptable medical staff utilisation rate and shorter idle time for doctors must be maintained while accommodating a high patient throughput, shorter patient waiting time, and low clinic staff overtime [8,35]. To obtain that target, past studies addressed outpatient clinic problems in three categories: scheduling, patient flow, and resources allocation [32].

7.1 Simulating outpatient scheduling

As the first step towards high quality service delivery, it is critical to have effective scheduling. Outpatient scheduling deals mainly with the patient appointment system that is used to manage patients' access to health care services [40]. An efficient appointment system ensures that patients receive timely access to health care services, [40] in addition to an effective use of medical staff and equipment [2]. Nevertheless, the performance of the appointment system is invariably affected by factors such as patient arrival time, service time, cancellations on the appointment date, patient or staff preferences about the appointment time, scheduler experiences, and information technology available in the hospital [40].

One of the earliest attempts to solve outpatient scheduling using computer simulation was by Fetter and Thompson [41]. They designed an outpatient simulation model to examine variables that affected the relationship between patient waiting time and doctors' idle time. These variables were appointment interval, service rate, patients' arrival pattern, number of patient absences, number of walk-in patients, and unpunctuality or interruption of the doctor. Not surprisingly, they found that a doctor's late arrival would increase the patients' waiting time. If a doctor was an hour late, the waiting time per patient with an appointment would increase by 27 minutes, and for a walk-in patient it would be 152 minutes. Moreover, an increase of 50 per cent to 90 per cent on a doctor's work load could reduce the doctor's idle time from 20 per cent to 3 per cent. However, an increase in the doctor's workload would cause patients' waiting time to increase by 10.5 minutes per patient with appointment, and 60.2 minutes per walk-in patient. Their results showed that the doctor's punctuality and a realistic clinical load are critical in outpatient clinics.

White and Pike [42] found that patient unpunctuality was not the main reason for not having appointment systems in outpatient clinics. They suggested that an appointment system should have appointment intervals of one tenth the mean consultation time. Also, they proposed that a constant number of patients be called in for each appointment time while the doctor should always be punctual (not more than five minutes late for the first appointment). Nevertheless, such an appointment system would not be applicable to small clinics or clinics with a high variability of consultation time, where sorted patient or combined doctor sessions should apply.

Different patient arrival patterns can have different effects on patient waiting time in outpatient clinics, as shown by several studies. For instance, Smith and Warner [43] compared highly irregular arrival and constant arrival patterns of patients. They concluded that regular arrival patterns could reduce the average time of patients going through the clinics from 40.6 minutes to 24.0 minutes. Besides, they found that the increase in the number of work centres had minimal effect on the waiting time in the constant arrival pattern.

Rising et al. [44] smoothed out the arrival patterns by scheduling more appointments during hours with low walk-in demand. By doing this, a doctor could diagnose 13.4 per cent more patients and still maintain the average patient waiting time. This results in fewer overtime working hours and higher doctor morale.

Other studies in this area include that of Kachhal et al. [45] who distributed the work load of an audiologist uniformly in an ear, nose, and throat outpatient clinic by dividing the non-ear patients and ear patients between two doctors. They managed to reduce the average waiting time by 44.7 per cent in this study. Harper and Gamlin [28] developed an algorithm that distributed appointments over the whole clinic session. Their simulated result shows a 10-minute reduction in waiting time for each patient. Zhu et al. [16] found that evenly distributed appointment slots could also reduce the average waiting time and clinic overtime in specialist outpatient clinics.

Some researchers suggest that patients should be scheduled into unevenly distributed appointments or blocks rather than into evenly distributed appointments. Bailey [26,46] was the first to suggest scheduling two patients at the beginning of the session. Other patients were scheduled uniformly at an interval equal to the average consultation time. White and Pike [42] recommended the same scheduling rule so that doctors would not waste time if the first patient did not show up. Ho and Lau [47] studied various scheduling rules in order to minimise the weighted sum of idle-time cost of the medical personnel, ω, and the patient, π. The scheduling rule is recommended in the situations shown in Table 1.



However, in a later evaluation of different appointment rules by Klassen and Rohleder [48], performance of this scheduling rule was found to be acceptable only in certain scenarios, and was in fact regarded as one of the worst rules. They suggested that the combination of this rule with the scheduling of patients that had low standard deviation in patient service time at the beginning of the session might make it feasible in more scenarios.

In fact, Klassen and Rohleder [48] found that scheduling patients with low service time variance to early clinic sessions produced good results in most cases. This was consistent with a later study by Cayirli et al. [49] who found that this scheduling rule worked best where the reduction of patient waiting time was the main concern. This finding was also confirmed by White et al. [50], who found that such a policy reduced the greatest amount of patient waiting time and clinic overtime without compromising the doctor utilisation rate, given sufficient resources.

Compared with the allocation of a single patient to one appointment slot, patients could also be called in multiple-blocks. Soriano [51] compared a two-at-a-time appointment system with the individual appointment system. Two patients per block could better utilise the doctors' idle time and reduce patient waiting time. Cox et al. [52] found that calling multiple patients in an appointment slot worked best in two of the clinics that they modelled. However, this scheme was rated as inferior by Ho and Lau [47] because it fared poorly in time-cost efficiency for patient waiting time and doctors' idle time.

Instead of a fixed number of appointments in a block, Fries and Marathe [53] suggested variable appointments per block. Liu and Liu [54] mentioned that outpatient clinics were multi-server systems where doctors' arrival times tended to be random. They developed a variable block appointment system with a fixed interval time for an outpatient clinic. To obtain the optimal number of patients for each block as well as the optimal block length, a simulation study was first performed to identify the properties of the best appointment rules in different operating environments. With these properties at hand, a simulation-based appointment system was developed to identify the optimal schedule for an outpatient clinic.

All the appointment rules discussed above have a fixed appointment interval. However, the appointment interval can be variable too. Ho and Lau [47] introduced the variable interval for the single appointment block rule. They claimed that variable intervals could reduce the unfairness to patients with earlier appointments, as they would have a shorter waiting time compared with patients with later appointments. One of their rules was to schedule shorter appointment intervals at the beginning of the clinic session, and then to increase the appointment interval toward the end of the session. Later, Ho and Lau [46, 55] and Ho et al. [55] modified the rule by introducing K, a threshold value that separated clinic sessions into two parts in which different scheduling mechanisms applied. The appointment intervals were smaller than the mean before the Kth patient and larger than the mean after the Kth patient. Simulation experiments were used to determine the K value for the required situation. In an extensive evaluation of 50 appointment rules by Ho and Lau [56], the variable interval rule worked best when the patient waiting time was a major concern in the clinic. Similarly, Chew [57] developed a simulation-based heuristic algorithm to find the appointment interval of each block that consumed the lowest total cost for patient waiting time, doctors' idle time, and overtime requirements. He demonstrated near optimal appointment intervals for schedules based on two patients per block and three patients per block.

Yang et al. found that most of the appointment rules mentioned in the literature worked well only in specific service environments [58]. They proposed a new appointment rule that characterised service environments in a mathematical function. Service environments could be characterised by four parameters: the coefficient of variation of the service time (cv); percentage of no show (p); number of appointments per session (N); and the cost ratio between doctors' idle time and patient waiting time (ω/π). Yang et al. showed that the performance of their rule was as good as, or better than, the rules suggested by Ho and Lau [47]. It also worked well under most service conditions.

Huang et al. [59] used the 'wait' ratio to replace conventional cost ratio in their model to design a general appointment system. They believed medical staff tended to overestimate the cost of their time over patient time. By using the 'wait' ratio, bias against the patient waiting time could be removed. The authors tested this appointment system in three outpatient clinics, and were able to reduce patient waiting time by up to 56 per cent.

Urgent appointments are occasionally added into pre-scheduled outpatient clinic sessions. The placing of urgent appointment slots is crucial, as urgent slots would affect the normal clinic operation. Klassen and Rohleder [48] studied the placing of urgent appointment slots in clinic sessions, but could not find the best position for the urgent slot. They mentioned instead that the waiting time for patients would be shorter if more urgent appointment slots were placed in the earlier part of the session, although less urgent patients would be served eventually.

Murray and Berwick [60] categorised the appointment system that dealt with urgent patients into three models: the traditional model, the carve-out model, and the advanced access model. In the traditional model, urgent patients were treated first, ahead of nonurgent patients. In the carve-out model, urgent appointment slots were reserved for urgent patients, and the regular patients were scheduled using the traditional model. In the advanced access model, patients were treated similarly, regardless of urgency, and all patients were offered an appointment slot on a particular day.

Dobson et al. [61] studied the urgent appointment scheduling system that included the carve out system and the advanced access system. Two quality measures, the un-seen urgent patients and the queue length of routine patients, were examined. The optimal solution in a non-overloaded clinic depended on both the cost incurred on urgent patients (H), and the arrival dynamics of urgent and routine patients (p). Table 2 shows their relationship.



In overloaded clinics, the optimal reservation level of urgent slots did not depend on arrival dynamics. Improving either of the quality measures would have a negative effect on the other. So the authors suggested a tradeoff between the two quality measures to arrive at the optimal reservation schedule.

In essence, medical staff utilisation and patient waiting time could be improved through various appointment rules. Simulation would help to determine which rules were the most suitable in a particular environment by taking into account the trade-off between staff utilisation and patient waiting time [8]. Table 3 summarises the various studies on simulating outpatient scheduling.



7.2 Patient flow in the outpatient clinic

The outpatient clinic is a complex facility where different patients go through various processes in the clinic before they leave. This is also known as the patient flow. It has four attributes: (a) the entrance, (b) the exit, (c) a connection route between the entrance and the exit, and (d) uncertainties of the health care component [62]. Different patients will have different patient flows, and some can be very complex. The complexity of patient flow in clinics makes simulation a better analysis tool than other operation research methods. By using simulation, we are able not only to model complex patient flow, but also to model the effect of changing patient routes on variables such as patient waiting time and staff utilisation rates that would be difficult to perform in reality.

One such study to support the above argument was undertaken by Zhu et al. [16]. With a discrete simulation model, they analysed the factors that caused long patient waiting time and clinic overtime in an outpatient facility. They found three causative factors: overloaded session, late start of a session, and unused session time. Unevenly distributed slots and irregular calling sequences were other reasons that caused long patient waiting time. Four improvement settings were suggested to overcome the detected causative factors. The improvement settings involved evenly distributed appointment slots, punctuality in starting the clinic, elimination of unused session time, and removal of irregular calling sequences. Simulation and implementation of the results showed a significant reduction in waiting time and in clinic overtime.

Another patient flow improvement at an outpatient clinic was carried out by Chand et al. [63]. They mapped the patient flow in the clinic, and identified sources of variability and factors for improvement. A simulation model based on the process map was developed and used to test the improvement factors. Scattered arrivals of patients, centralised phone calls, removal of batching on patient records, and consolidation of the patient arrival queues into one queue were factors that were deemed to improve patient flow. Such improvements could significantly reduce patient waiting time and increase the physicians' utilisation time.

Rohleder et al. [64] used simulation modelling to support process improvements at a congested orthopaedic outpatient clinic. They used simulation modelling as a tool to identify improvement alternatives and to quantify potential improvements in performance. Encouraged by the modelling results, three improvement settings - optimised staffing levels, an improved patient appointment system, and an emphasis on staff punctuality -were implemented. The total patient clinic time was reduced by 33 per cent after implementation.

Another study by Edward et al. [65] examined patient flow in two different clinics that used different flow management systems: the serial process and the quasi-parallel process. In the serial processing clinic, patients waited in a single queue, whereas in the quasi-parallel processing clinic, patients were directed to the shortest queue. The authors concluded that a 25 to 29 per cent reduction in the waiting time would be achieved if the clinic changed from serial processing to quasi-parallel processing flow management.

A change of facility layout also impacts patient flow. Sepúlveda et al. [66] simulated a cancer treatment centre with the objective of improving the patient flow process and increasing the capacity of the facility. They showed that a 100 per cent increase in chair capacity was possible with a new layout. And simulation of an alternative scheduling showed 20 per cent more patients seen per day. The researchers modelled a new clinic with 100 per cent more capacity than the existing facility. Using the new clinic model to identify key bottlenecks, it was observed that the limited accommodation of the waiting room was responsible for inefficiency in patient flow.

Wilson Memorial Regional Medical Center, Broome County, New York, USA implemented a digital image archiving system in its radiology department. With this implementation, a simulation was conducted by Ramakrishnan et al. [67] to identify the change in the patient flow, to maximise the patient throughput, and to minimise report generation time. Based on these findings, a 20 per cent increase in patient throughput and a 30 per cent reduction in report generation time was attained.

Just as when a new system is implemented, the relocation of a clinic department also affects patient flow. Groothuis et al. [68] used a systematic computer simulation approach to study the effect of relocating a phlebotomy department in a hospital. Simulation experiments were carried out to identify the impact of the new layout and the procedure change on the average patient turn-around time. A reduction of 33 per cent in patient turnaround time (from 12 to eight minutes) was possible by relocating the department.

In short, a change to a clinic procedure or its location can have substantial impact on patient flow. Simulation serves as an analysis tool to predict the impact of the changes on patient flow. Such investigations enable bottlenecks in patient flow to be detected and rectified before actual implementation of the improved system. Table 4 summarises the studies of patient flow with simulation.



7.3 Outpatient clinic resource allocation

In general, the demand for and cost of better health care increases at a faster rate than the augmentation of clinic resources such as health care staff and space. To make matters worse, funding for health care is often reduced whenever there is a downturn in the economy. Health care services frequently find it difficult to acquire more resources to cope with rising costs. For this reason, resource allocation in health care planning is crucial to maintain quality health services.

To plan resource allocation effectively, the health care community has used simulation modelling as an analytical tool [8]. Resource allocation and planning in outpatient clinics is divided into treatment or consultation room planning and health care staff planning. The treatment or consultation room is where patients receive treatment or a consultation from medical staff. Proper planning of the number of rooms during the clinic's setup is important, as it is generally easier to hire medical staff than to add space later on. Careful health care staff planning enables optimum medical staff utilisation for economical operation.

Outpatient clinics play an increasingly important role as a greater proportion of health care delivery shifts from inpatient facilities to outpatient facilities. The increased demand by patients for outpatient services is putting pressure on clinic administrators to expand the outpatient clinic capacity to include more services or new facilities [35]. Simulation helps hospital administrators to decide on the capacity of new facilities before they are set up. Better decisions can be made with reduced financial risks. Amladi [69] used simulation to help a hospital expand its outpatient service through sizing and planning a new outpatient surgical facility. Patient waiting time (quality) and facility size (price) were factors optimised in that model.

Similarly, Iskander and Carter [70] used simulation to check whether an outpatient clinic under construction had enough space and facilities in a new ambulatory care centre for its projected patient demand. Their result suggested the need to move some services to the main hospital and to modify the design for outpatient facilities. After the facility was fully functional, the predicted result of simulation was verified by the good patient flow experienced.

Hospital administrators are able to check the effect of integrating different services or facilities using simulation. A plan to combine an obstetric clinic with a gynaecology clinic was cancelled after Mahachek and Knabe [71] found that current clinic space was not sufficient for the proposed integration. Using historical data in another study, Levy et al. [72] developed a simulation model of a new outpatient services centre in Anderson Memorial Hospital that integrated existing outpatient services within the hospital with offsite services at the outpatient diagnostic centre. Analysis of facility utilisation rate, patient waiting time, time spent in the clinic, and the number of queues in the clinic was simulated using the model. From this, a minimum facility design specification was determined, based on the expected demand.

Besides its application in planning the capacity of new facilities, simulation is often also used to plan the capacity of new services. Romero et al. [73] used simulation to evaluate the implementation of a one-stop-shop in a skin cancer treatment outpatient clinic. Different scenarios of new services were analysed in the simulation model; in the best case scenario, a more than 90 per cent reduction in new patient throughput times for treatments was achievable, even at the same resource level.

Simulation is also used to plan staff allocation in outpatient clinics, since the optimum staff number needs to match the number of patients. Hashimoto and Bell [74] developed a simulation model of an outpatient clinic to improve outpatient clinic staffing. They found that the idle time of all services in the clinic could be reduced by increasing the number of doctors. However, the time a patient spent in the clinic would still increase, even if the number of scheduled patients remained the same. This was because patient load on other services was larger compared with doctor consultation. They therefore suggested that the clinic should limit the number of doctors to four (or occasionally five) and dischargers to two during clinic sessions, and retrain the medical staff to increase efficiency. As a result, the time that patients spent in the clinic was reduced by 24 per cent.

Wilt and Goddin [75] used simulation to study a proposed outpatient diagnostic centre that focused on staff requirements and work flow. They simulated the staff levels needed for the outpatient clinic in a master schedule and recommended modifications to the facility. Weng and Houshmand [76] used simulation to determine the optimum number of doctors and medical assistants required for an outpatient clinic. This simulation-based modelling successfully maximised patient throughput, reduced patient time in the clinic, and minimised operating costs. Swisher and Jacobson [77] used Visual Simulation Environment to develop an object-oriented discrete event simulation model of a family practice health care clinic. The model was used to determine optimum staffing and physical resources in the clinic. Clinic profitability, patient satisfaction, and medical staff satisfaction were the three objectives of this clinical effectiveness evaluation. They ran simulation experiments to derive suggestions for optimum staffing and resource levels of two family practice health care clinics.

Simulation is used also to show advantages in the pooling of resources for better resource allocation. Rohleder et al. [78] found that by consolidating smaller facilities into fewer but larger units in a patient service centre network, a positive resource pooling effect would be created. It would reduce demand variability and improve resource utilisation, thus reducing patient waiting time.

Along similar lines, Santibáñez et al. [79] showed that pooling resources could reduce 22.2 per cent of examination room space in a multi-provider outpatient clinic without significantly affecting patient waiting time, clinic end time, doctors' idle time, and the resource utilisation level. Surplus examination rooms could then be used for other duties. Berg et al. [80] developed a simulation model of a colonoscopy suite to evaluate the operational performance for colonoscopy screening. They found that resource pooling of procedure rooms among endoscopists could increase the rate at which the doctors were occupied. However, patient throughput reduced when the ratio of endoscopists to the number of procedure rooms increased due to the capacity constraints of the procedure rooms.

In essence, simulation helps outpatient clinic management to make better decisions about resource allocation. Better resource allocation improves not only financial efficiency but also patient and staff satisfaction. As a result, overall outpatient clinic efficiency increases. Table 5 summarises the studies on outpatient clinic resources allocation that involved simulation.



7.4 Integrating outpatient scheduling with resource allocation

Most of the studies examined outpatient scheduling and resource allocation separately. However, the research community has begun to see the merits of integrating the study of outpatient scheduling with that of resource allocation. White et al. [50] found that the interaction effect between outpatient clinic resources and appointment policy, even though small, was statistically significant. This showed that optimised results might not be achieved if only one of the two was considered. So, to obtain an optimised solution, it would often be advantageous to examine resource allocation and appointment policy together.

This strategy is consistent with the work of Santibáhez et al. [79], who found that a major reduction of waiting time occurred only when changes to clinic resources and appointment policy were implemented simultaneously. The changes were in (a) improved patient scheduling according to patient type and length of consultation, (b) dynamic and flexible allocation of examination rooms among individual clinics within the medical radiation and surgical oncology programmes, and (c) the punctual opening of clinics. Simulation results showed that 25 per cent of physical space could be saved and patient waiting time reduced by 70 per cent, with the same patient load.



This paper reviews the problems that occur in outpatient clinics and how management simulation can be used to ameliorate shortcomings. Many cases of simulation that is successfully used to solve problems of long patient waiting time and resource management can be found in the literature. However, while most of the papers showed the results of simulation, they do not always provide comprehensive solutions that are actually feasible in outpatient clinics. The following are steps that could be taken in the future to fill this knowledge gap.

First, involve the medical staff more during the development of management simulation models. Medical staff involvement can ensure that the model being developed represents the realistic situation in the clinic. Their experience in the operation of outpatient clinics would help to refine system inputs and details that are essential to model building. Besides, their involvement from the early stages would enhance their understanding of the model to be implemented, and implementation of the findings in real life settings would be easier.

Second, simulations of outpatient clinics should focus more on software that can graphically represent a model. Decision-makers in outpatient clinics usually do not have detailed knowledge of simulation and operations research. A graphical output would help them understand why and how the system should be improved in the recommended way. Besides, graphical representations are helpful in model verification and validation [28,81,79].

Third, optimisation techniques combined with simulation techniques have been used in hospital departments such as the emergency department [82,83], although this method is not yet popular in outpatient clinics. Optimisation techniques should be employed to establish the optimum appointment system, resource allocation, and capacity planning. Simulation software is often equipped with an optimisation plug-in to help the user to find an optimum solution. For example, OptQuest [84] uses metaheuristic, mathematical optimisation, and neural network components to search for optimum solutions.

Fourth, the implementation of solutions for outpatient clinic problems is rarely highlighted in the literature. Without implementation, we do not know how the suggested solutions perform in real life. The results of suggested solutions on being implemented in a clinic might be different from those obtained during simulation, possibly because patient and staff behaviour could have changed with the new system.

Fifth, hospital staff members are assumed to work in the clinic only. In reality, however, staff members (especially doctors) have other responsibilities, such as making ward rounds before they start their clinic sessions. The effect on their other responsibilities in hospital has to be taken into account when changing their schedules in the outpatient clinic, otherwise optimisation of outpatient clinics could, in turn, result in other problems.

Sixth, the effect of changing the setting in the outpatient clinic with respect to other hospital departments is rarely discussed. The different departments in the hospital are usually interrelated. Patients might not visit the outpatient clinic only: they might be treated in other departments in the hospital as well. For example, orthopaedic patients might need to visit the radiology department or rehabilitation department before keeping their appointment in the orthopaedic outpatient clinic. If the capacity of the orthopaedic clinic increases, the radiology department or rehabilitation department might become overcrowded as a result. Hence, the implications for other departments in the hospital have to be considered in the simulation model.

Finally, recent research [50,79] has shown that optimum results can only be achieved when resource allocation, scheduling policy, and patient flow patterns are analysed together. This integrated research method is less commonly employed than are separate investigations into the individual factors. Future studies should, therefore, focus on the integrated approach, especially in the management of multi-department outpatient clinics. Outpatient clinics still face many challenges as society becomes better informed on issues of health and wellbeing, and as public expectations about health care rise. In making changes and improvements to their facilities, outpatient clinic administrators need to arrive at their decisions scientifically and objectively. Management simulation is an effective technique to assist them with these crucial decisions.



[1] Haub, C. In 2011, World Population Surpasses 7 Billion. Population Reference Bureau 2011. [accessed 03 November 2011].         [ Links ]

[2] Cayirli, T. & Veral, E. 2003. Outpatient scheduling in health care: A review of literature. Production and Operations Management, 12 (4), pp 519-549.         [ Links ]

[3] Davis, S. B. & Robinson, P. J. 2010. Health care providers under pressure: Making the most of challenging times. Journal of Health Care Finance, 37 (2), pp 59-64.         [ Links ]

[4] World Health Organization. 2010. Key components of a well functioning health system. World Health Organization.         [ Links ]

[5] Boe, D. T., Riley, W. & Parsons, H. 2009. Improving service delivery in a county health department WIC clinic: An application of statistical process control techniques. American journal of public health, 99 (9), pp 1619.         [ Links ]

[6] Hart, M. 1995. Improving out-patient clinic waiting times: Methodological and substantive issues. International Journal of Health Care Quality Assurance, 8 (6), pp 14-22.         [ Links ]

[7] Buckle, R. & Stuart, T. 1996. Systematic approach reduces patient waiting times. Physician Executive, 22, pp 10-15.         [ Links ]

[8] Jun, J., Jacobson, S. & Swisher, J. 1999. Application of discrete-event simulation in health care clinics: A survey. Journal of the Operational Research Society, 50 (2), pp 109-123.         [ Links ]

[9] Fone, D., Hollinghurst, S., Temple, M., Round, A., Lester, N., Weightman, A., Roberts, K., Coyle, E., Bevan, G. & Palmer, S. 2003. Systematic review of the use and value of computer simulation modelling in population health and health care delivery. Journal of Public Health, 25 (4), pp 325-335.         [ Links ]

[10] Declaration of Alma-Ata. 1978. Paper read at International Conference on Primary Health Care, 6-12, September, 1978, at Alma-Ata, USSR.         [ Links ]

[11] Economic Planning Unit. 2000. Eighth Malaysia Plan. edited by P. M. s. D. Malaysia.         [ Links ]

[12] Yerby, A. S. 1964. The hospital out-patient department as a source of medical care. Medical Care, pp 225-227.         [ Links ]

[13] Hospital Kuala Lumpur. Department of out-patient 2008. [accessed 15 October 2011].         [ Links ]

[14] Snyder, S. & Stampfli, D. 2011. Ambulatory care offers significant opportunities for lenders. Risk Management Association, pp 22-27.         [ Links ]

[15] Bloom, D. E. 2011. 7 billion and counting. Science, 333 (6042), pp 562-569.         [ Links ]

[16] Zhu, Z., Heng, B. & Teow, K. 2010. Analysis of factors causing long patient waiting time and clinic overtime in outpatient clinics. Journal of Medical Systems, pp 1 -7.         [ Links ]

[17] DiTomasso, R. A. & Willard, M. 1991. The development of a patient satisfaction questionnaire in the ambulatory setting. Family Medicine, 23 (2), pp 127.         [ Links ]

[18] Campbell, J. 1994. General practitioner appointment systems, patient satisfaction, and use of accident and emergency services-a study in one geographical area. Family Practice, 11 (4), pp 438-445.         [ Links ]

[19] Gourdji, I., McVey, L. & Loiselle, C. 2003. Patients' satisfaction and importance ratings of quality in an outpatient oncology center. Journal of Nursing Care Quality, 18 (1), pp 43-55.         [ Links ]

[20] Eilers, G. M. 2004. Improving patient satisfaction with waiting time. Journal of American College Health, 53 (1), pp 41-48.         [ Links ]

[21] Huang, X. M. 1994. Patient attitude towards waiting in an outpatient clinic and its applications. Health Services Management Research, 7 (1), pp 2.         [ Links ]

[22] McKinnon, K., Crofts, P. D., Edwards, R., Campion, P. D. & Edwards, R. H. T. 1998. The outpatient experience: Results of a patient feedback survey. International Journal of Health Care Quality Assurance, 11 (5), pp 156-160.         [ Links ]

[23] McCarthy, K., McGee, H. M. & O'Boyle, C. A. 2000. Outpatient clinic waiting times and non-attendance as indicators of quality. Psychology, Health and Medicine, 5 (3), pp 287-293.         [ Links ]

[24] Cartwright, A. & Windsor, J. 1992. Outpatients and their doctors. London: Department of Health Institute for Social Studies in Medical Care, pp.         [ Links ]

[25] Manaf, N. H. A. & Nooi, P. S. 2009. Patient satisfaction as an indicator of service quality in malaysian public hospitals. Asian Journal on Quality, 10 (1), pp 77-87.         [ Links ]

[26] Bailey, N. T. J. 1952. A study of queues and appointment systems in hospital out-patient departments, with special reference to waiting-times. Journal of the Royal Statistical Society. Series B (Methodological), pp 185-199.         [ Links ]

[27] Lindley, D. V. 1952. The theory of queues with a single server. In Mathematical Proceedings of the Cambridge Philosophical Society: Cambridge Univ Press.         [ Links ]

[28] Harper, P. R. & Gamlin, H. M. 2003. Reduced outpatient waiting times with improved appointment scheduling: A simulation modelling approach. OR Spectrum, 25 (2), pp 207-222.         [ Links ]

[29] Hillier, F. S. & Lieberman, G. J. 2010. Introduction to operations research. New York: McGraw-Hill.         [ Links ]

[30] Brailsford, S., Harper, P., Patel, B. & Pitt, M. 2009. An analysis of the academic literature on simulation and modelling in health care. Journal of Simulation, 3 (3), pp 130-140.         [ Links ]

[31] Banks, J., Carson, J. S., Nelson, B. L. & Nicol, D. M. 2004. Discrete-event system simulation. Upper Saddle River: Pearson Prentice Hall.         [ Links ]

[32] Van Sambeek, J., Cornelissen, F., Bakker, P. & Krabbendam, J. 2010. Models as instruments for optimizing hospital processes: A systematic review. International Journal of Health Care Quality Assurance, 23 (4), pp 356-377.         [ Links ]

[33] Günal, M. & Pidd, M. 2010. Discrete event simulation for performance modelling in health care: A review of the literature. Journal of Simulation, 4 (1), pp 42-51.         [ Links ]

[34] Law, A. M. 2007. Simulation modeling and analysis. New York: McGraw-Hill.         [ Links ]

[35] Jacobson, S. H., Hall, S. N. & Swisher, J. R. 2006. Discrete-event simulation of health care systems. Patient Flow: Reducing Delay in Healthcare Delivery, pp 211-252.         [ Links ]

[36] Aharonson-Daniel, L., Paul, R. J. & Hedley, A. J. 1996. Management of queues in out-patient departments: The use of computer simulation. Journal of Management in Medicine, 10 (6), pp 50-58.         [ Links ]

[37] Hurrion, R. D. 1978. An investigation of visual interactive simulation methods using the job-shop scheduling problem. Journal of the Operational Research Society, pp 1085-1093.         [ Links ]

[38] Bell, P. C. 1985. Visual interactive modelling in operational research: Successes and opportunities. Journal of the Operational Research Society, pp 975-982.         [ Links ]

[39] Lehaney, B., Clarke, S. A., Paul, R. J. & Correspondence. 1999. A case of an intervention in an outpatients department. Journal of the Operational Research Society, 50 (9), pp 877-891.         [ Links ]

[40] Gupta, D. & Denton, B. 2008. Appointment scheduling in health care: Challenges and opportunities. IIE transactions, 40 (9), pp 800-819.         [ Links ]

[41] Fetter, R. B. & Thompson, J. D. 1966. Patients' waiting time and doctors' idle time in the outpatient setting. Health Services Research, 1 (1), pp 66.         [ Links ]

[42] White, M. J. B. & Pike, M. 1964. Appointment systems in out-patients' clinics and the effect of patients' unpunctuality. Medical Care, pp 133-145.         [ Links ]

[43] Smith, E. A. & Warner, H. R. 1971. Simulation of a multiphasic screening procedure for hospital admissions. SIMULATION, 17 (2), pp 57-64.         [ Links ]

[44] Rising, E. J., Baron, R. & Averill, B. 1973. A systems analysis of a university-health-service outpatient clinic. Operations Research, pp 1030-1047.         [ Links ]

[45] Kachhal, S., Klutke, G. A. & Daniels, E. B. 1981. Two simulation applications to outpatient clinics. Paper read at Proceedings of the 13th conference on Winter simulation - Volume 2, at Atlanta, Georgia, USA.         [ Links ]

[46] Bailey, N. T. J. 1954. Queueing for medical care. Journal of the Royal Statistical Society. Series C (Applied Statistics), 3 (3), pp 137-145.         [ Links ]

[47] Ho, C.-J. & Lau, H.-S. 1992. Minimizing total cost in scheduling outpatient appointments. Management Science, 38 (12), pp 1750-1764.         [ Links ]

[48] Klassen, K. J. & Rohleder, T. R. 1996. Scheduling outpatient appointments in a dynamic environment. Journal of Operations Management, 14 (2), pp 83-101.         [ Links ]

[49] Cayirli, T., Veral, E. & Rosen, H. 2006. Designing appointment scheduling systems for ambulatory care services. Health Care Management Science, 9 (1), pp 47-58.         [ Links ]

[50] White, D. L., Froehle, C. M. & Klassen, K. J. 2011. The effect of integrated scheduling and capacity policies on clinical efficiency. Production and Operations Management, pp.         [ Links ]

[51] Soriano, A. 1966. Comparison of two scheduling systems. Operations Research, pp 388-397.         [ Links ]

[52] Cox, T. F., Birchall, J. P. & Wong, H. 1985. Optimising the queuing system for an ear, nose and throat outpatient clinic. Journal of Applied Statistics, 12 (2), pp 113-126.         [ Links ]

[53] Fries, B. E. & Marathe, V. P. 1981. Determination of optimal variable-sized multiple-block appointment systems. Operations Research, 29 (2), pp 324-345.         [ Links ]

[54] Liu, L. & Liu, X. 1998. Block appointment systems for outpatient clinics with multiple doctors. Journal of the Operational Research Society, 49 (12), pp 1254-1259.         [ Links ]

[55] Ho, C., Lau, H. S. & Li, J. 1995. Introducing variable-interval appointment scheduling rules in service systems. International Journal of Operations & Production Management, 15 (6), pp 59-68.         [ Links ]

[56] Ho, C. J. & Lau, H. S. 1999. Evaluating the impact of operating conditions on the performance of appointment scheduling rules in service systems. European Journal of Operational Research, 112 (3), pp 542-553.         [ Links ]

[57] Chew, S. F. 2011. Outpatient appointment scheduling with variable interappointment times. Modelling and Simulation in Engineering, 2011, pp.         [ Links ]

[58] Yang, K. K., Lau, M. L. & Quek, S. A. 1998. A new appointment rule for a single-server, multiple-customer service system. Naval Research Logistics, 45 (3), pp 313-326.         [ Links ]

[59] Huang, Y. L., Hancock, W. M. & Herrin, G. D. 2012. An alternative outpatient scheduling system: Improving the outpatient experience. IIE Transactions on Healthcare Systems Engineering, 2 (2), pp 97-111.         [ Links ]

[60] Murray, M. & Berwick, D. M. 2003. Advanced access. JAMA: the Journal of the American Medical Association, 289 (8), pp 1035.         [ Links ]

[61] Dobson, G., Hasija, S. & Pinker, E. J. 2011. Reserving capacity for urgent patients in primary care. Production and Operations Management, pp.         [ Links ]

[62] Côté, M. J. 2000. Understanding patient flow. Decision Line, 31 (2), pp 8.         [ Links ]

[63] Chand, S., Moskowitz, H., Norris, J. B., Shade, S. & Willis, D. R. 2009. Improving patient flow at an outpatient clinic: Study of sources of variability and improvement factors. Health Care Management Science, 12 (3), pp 325-340.         [ Links ]

[64] Rohleder, T., Lewkonia, P., Bischak, D., Duffy, P. & Hendijani, R. 2011. Using simulation modeling to improve patient flow at an outpatient orthopedic clinic. Health Care Management Science, 14 (2), pp 135-145.         [ Links ]

[65] Edwards, R., Clague, J., Barlow, J., Clarke, M., Reed, P. & Rada, R. 1994. Operations research survey and computer simulation of waiting times in two medical outpatient clinic structures. Health Care Analysis, 2 (2), pp 164-169.         [ Links ]

[66] Sepúlveda, J. A., Thompson, W. J., Baesler, F. F., Alvarez, M. I. & Cahoon III, L. E. 1999. The use of simulation for process improvement in a cancer treatment center. Paper read at Proceedings of the 31st conference on Winter simulation: Simulation - a bridge to the future -Volume 2        [ Links ]

[67] Ramakrishnan, S., Nagarkar, K., DeGennaro, M., Srihari, M., Courtney, A. K. & Emick, F. 2004. A study of the CT scan area of a healthcare provider. Paper read at Winter Simulation Conference: Proceedings of the 36th conference on Winter simulation.

[68] Groothuis, S., Goldschmidt, H. M. J., Drupsteen, E. J., de Vries, J. C. M., Hasman, A. & van Merode, G. G. 2002. Application of computer simulation analysis to assess the effects of relocating a hospital phlebotomy department. Annals of Clinical Biochemistry, 39 (3), pp 261-272.         [ Links ]

[69] Amladi, P. 1984. Outpatient health care facility planning and sizing via computer simulation. Paper read at Proceedings of the 16th conference on Winter simulation, at Dallas, TX.         [ Links ]

[70] Iskander, W. H. & Carter, D. M. 1991. A simulation model for a same day care facility at a university hospital. Paper read at Simulation Conference, 1991. Proceedings., Winter, 8-11 Dec 1991.         [ Links ]

[71] Mahachek, A. R. & Knabe, T. L. 1984. Computer simulation of patient flow in obstetrical/gynecology clinics. SIMULATION, 43 (2), pp 95-101.         [ Links ]

[72] Levy, J., Watford, B. & Owen, V. 1989. Simulation analysis of an outpatient services facility. Journal of the Society for Health Systems, 1 (2), pp 35.         [ Links ]

[73] Romero, H., Dellaert, N., van der Geer, S., Frunt, M., Jansen-Vullers, M. & Krekels, G. A. M. 2012. Admission and capacity planning for the implementation of one-stop-shop in skin cancer treatment using simulation-based optimization. Health Care Management Science, pp 1-12.         [ Links ]

[74] Hashimoto, F. & Bell, S. 1996. Improving outpatient clinic staffing and scheduling with computer simulation. Journal of General Internal Medicine, 11 (3), pp 182-184.         [ Links ]

[75] Wilt, A. & Goddin, D. 1989. Health care case study: Simulating staffing needs and work flow in an outpatient diagnostic center. Industrial Engineering, 21 (5), pp 22-26.         [ Links ]

[76] Weng, M. L. & Houshmand, A. A. 1999. Healthcare simulation: A case study at a local clinic. Paper read at Simulation Conference Proceedings, 1999 Winter, 1999.         [ Links ]

[77] Swisher, J. R. & Jacobson, S. H. 2002. Evaluating the design of a family practice healthcare clinic using discrete-event simulation. Health Care Management Science, 5 (2), pp 75-88.         [ Links ]

[78] Rohleder, T. R., Bischak, D. P. & Baskin, L. B. 2007. Modeling patient service centers with simulation and system dynamics. Health Care Management Science, 10 (1), pp 1-12.         [ Links ]

[79] Santibáñez, P., Chow, V. S., French, J., Puterman, M. L. & Tyldesley, S. 2009. Reducing patient wait times and improving resource utilization at British Columbia Cancer Agency's ambulatory care unit through simulation. Health Care Management Science, 12 (4), pp 392-407.         [ Links ]

[80] Berg, B., Denton, B., Nelson, H., Balasubramanian, H., Rahman, A., Bailey, A. & Lindor, K. 2010. A discrete event simulation model to evaluate operational performance of a colonoscopy suite. Medical Decision Making, 30 (3), pp 380-387.         [ Links ]

[81] Sargent, R. G. 2005. Verification and validation of simulation models. Paper read at Proceedings of the 37th conference on Winter simulation, at Orlando, Florida.         [ Links ]

[82] Ahmed, M. A. & Alkhamis, T. M. 2009. Simulation optimization for an emergency department healthcare unit in Kuwait. European Journal of Operational Research, 198 (3), pp 936-942.         [ Links ]

[83] Puente, J., Gómez, A., Fernández, I. & Priore, P. 2009. Medical doctor rostering problem in a hospital emergency department by means of genetic algorithms. Computers & Industrial Engineering, 56 (4), pp 1232-1242.

[84] Systems, O. OptQuest 2012. [accessed 06-November-2011].         [ Links ]



1, 2 Corresponding and joint first authors

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License