performance support project
Giovanni Giacometti Ceroni
The Centre for Advanced
Learning Technologies INSEAD
Fontainebleau, France
- reducing the complexity or number of steps required to perform a task;
- providing the performance information an employee needs to perform a task;
- providing a decision support system that enables an employee to identify the action that is appropriate for a particular set of conditions.
2. EVOLUTION OF CONCEPTS
With the emergence of the so-called
“Knowledge Economy”, the late concepts of EPSS have evolved to support
the knowledge workers and encourage their shared learning. New definitions
of EPSS are given:
“Dynamic support systems are characterized
by the ability to change with experience, the ability to be updated and
adjusted by the performer, and by augmenting other supports found in the
performer’s community”. [3]
3. FROM USA TO EUROPE
The concept of EPSS has been spreading first throughout North America and then, to a much smaller extent, in Europe. The number of researchers and consultants working on it witnesses the success of EPSSs in the US. Leading American research centres have carried out projects to investigate the potentialities of performance support tools, from software wizards to wearable computers. Several companies have been working side-to-side with them, specializing in the development of such systems and producing a rich variety of case histories often very well documented. In Europe these applications have raised much less interest. There have been a few very interesting experiences, for instance the projects developed at the University of Helsinki [4] and at the Knowledge Media Insitute, but there has never been a real market for performance support software. This is due mainly to different industry requirements, but also to the differences in the cultural and social environment. In Europe there is a cultural (and often legal) aversion to measuring or tracking individual performance. EPSSs started raising interest only when their focus moved towards knowledge management and global performance of teams and organizations.
4. WHAT IS BOPS?
BOPS is an European Project started
in September 1998 by a consortium of eight companies distributed in France,
Italy, Germany, Greece and Luxembourg, which has developed an innovative
type of EPSS. BOPS has taken the late definitions approach of EPSS and
has enlarged and enriched it, through both a wider coverage of processes
and a careful adaptation to the European context. Within BOPS the objective
of the performance support has shifted even more towards the management
of learning and knowledge at the corporate level, so that it is probably
not correct to define BOPS as an EPSS. A more meaningful definition would
be "performance centred corporate information portal".
BOPS main objective is to offer an environment that allows workers to:
- get trained on demand on the new best practices;
- be assisted and monitored by a training staff that may recommend additional training courses;
- contribute to the development of new best practices;
It sounds like the three wishes of Gloria Gery. But to make it real, what do we need?
BOPS gives the following answer:
- A workflow management system to trigger and support the information flows;
- A corporate knowledge catalogue to store, index and retrieve documents;
- A personnel management system, to manage the data about employees skills, training needs and performance;
- A decision support system to provide the necessary aggregated information to training and division managers.
Picture 1
Picture
BOPS mediates these two approaches,
providing a framework which can be adapted relatively quickly and cheaply
to a given business environment. BOPS is not a standard package ready to
be installed, but a set of software modules, methodologies, best practices
and templates which are at the end a "recipe" to build a good customized
EPSS without reinventing the wheel.
Picture 3: Distribution of attitudes
5. SUPPORTING
PERFORMANCE: THE GOOD, THE BAD AND THE UGLY
" Many redesign efforts do not deliver the step-function improvements in performance that, by rights, they should."[11]
"Major Information Technology projects incur significant risks"[12]
This article presents the experience of three of these four pilot sites, describing the scenario, the objectives and the outcome of each one. The three pilots are:
- Caritas Luxembourg, member-organization of Caritas International, the largest Catholic charitable organization in the world;
- Siemens Information and Communication Networks SpA. Formerly Italtel Sistemi and part of the Italtel group Siemens ICN is one of the largest European manufacturers of telecommunication equipment;
- The Chamber of Commerce and Industry of Paris (CCIP), and in particular its department called "Direction des Formalités Internationales et des Services aux Entreprises" (DFISE).
Targeted processes: The business processes implemented within BOPS are:
- the request of training by an employee and the possible authorization/denial by her responsible. Communication between the employee and the responsible relies upon the BOPS workflow module. The responsible uses the BOPS catalogue server to check the employee's profile, the qualification of the requested training and its suitability. The system also supports the subscription and the final evaluation of the course by the employee;
- the validation of the data stored in the catalogue server, which holds the records with the personal and professional data of the employees and of the member-organizations;
- clients arrival at Caritas premises. After the initial contact by phone, email or meeting, the employee has to direct the clients to the organization, which might help them. The BOPS catalogue is used to understand client's needs and to identify the right service or organization to recommend.
Caritas is not interested in measuring employees’ performance; therefore the BOPS Performance Scorecard module has not been implemented. An estimate of the performance of the whole organization will be made by monitoring the system usage and on the basis of the users feedback.
Outcome:
The
installation of BOPS at Caritas was severely delayed by the data population
of the catalogue and consequently there has not been time for any serious
observation of the system usage before the end of the project. Anyway it
is interesting to analyse the reasons of this delay. It must be pointed
out that the management of Caritas was very committed to make the installation
successful, and the new system was definitely perceived as necessary and
useful. The users and the management have been well informed and motivated
by the project responsible with a series of meetings and interviews. Nevertheless
Caritas had a very basic IT culture: with the exception of a few Access
databases all the data were not in electronic format and had to be entered
manually; no central information system was used. Consequently the data
population of the catalogue server, which is the key activity in the installation
of information portals like BOPS, took much more than expected. The delay
is relevant but it is not going to compromise the final result, as the
system will be in use in a few weeks.
Targeted processes: The business processes implemented are related to the activity of the Sales Dept.:
- first presentation to a new customer. The salesperson uses the system to gather data about the customer, to select the right content for the presentation and to store the information acquired during the presentation itself together with its own comments;
- follow-up meeting. The salesperson and a business product manager use the system document catalogue to prepare a more detailed presentation for the new customer. After the meeting customer requirements and comments, evaluation of the meetings, further requirements and lacks in the documentation are entered in the system;
- proposal to customer and finalisation of the proposal. The system supports this process by providing existing documents and data and samples of past proposals. The new proposal is eventually stored in the system and possibly updated.
Siemens is also going to use BOPS to track two types of performance indicators. The department performance is measured by ratios between the number of contacted customers and the number of successes (in approaching the customer, in making the proposal and in getting the contract signed). The quality of the catalogue content is measured by the lacks in the documentation notified by the employees and by the number of meetings/presentations not fulfilling the expectations.
Status and outcome: At Siemens BOPS installation has been completed on time and after the first 2 months of testing a questionnaire [13] has been distributed to evaluate the impact on the organisation and on the employees work. The questionnaire, composed by 16 questions, addressed four different dimensions:
- Level of motivation and awareness. The first group of questions aimed at determining if the change management activities carried out at Siemens had been effective and if there was enough commitment to make the test valid. It emerged that the users had a good knowledge of the project and its objective, were moderately optimistic about its usefulness and sufficiently committed to try it out.
- Level and frequency of usage. The second part of the questionnaire showed that the 12 pilot users have been using BOPS on average once every 2 days for 5-10 minutes each time. When questioned about the overall experience of using BOPS some users have rated it "a little boring", some others "quite pleasant".
- Job enrichment. This dimension refers to the improvement in the quality and speed of the usual employees work. Already after the first 2 months most users noticed that they were able to perform certain tasks slightly faster and that in some cases the quality of their output had improved. Users have recognized in BOPS a tool to obtain training or, in the worst case, just some useful hints. These improvements have not been judged quantitatively relevant yet, but they show a positive trend and a diffuse appreciation of BOPS job support capabilities. On the other side BOPS didn't seems to boost the information flows and the communication among employees nor to have perceivable ergonomic benefits.
- Job enlargement. The last part of the questionnaire investigated the effect of BOPS on the knowledge that employees have of their company business processes and of the tasks related to their job. This is the most interesting result of the questionnaire. Also in this dimension answers show a positive trend. Most users recognized that using BOPS they acquired a slightly better knowledge and visibility of the corporate business processes, learning about tasks which are not strictly part of their job but anyway helpful.
Targeted processes: CCIP has decided to use the workflow module and the catalogue server of BOPS. The supported workflows are:
- ATA carnet drawing up. This is a simulation which helps the users to understand how to fill the carnet in a risk-free environment;
- Answering questions about the ATA carnet. The catalogue server provides documents and information to help users answering questions;
- Verification of an ATA carnet. This activity is supported by the documents stored in the BOPS catalogue server.
Outcome: At the time of the official end of the project the installation at CCIP was not finished yet, and there were not enough data to make any forecast about its outcome. There have been major delays due to customization problems and to a lack of co-ordination with the users and the CCIP project managers, but also to a basic organizational complexity, as the project management was originally entrusted to Le Preau, an IT research center affiliate to CCIP, while the system had to be deployed at DFISE. This duplicity probably made more difficult to obtain a good co-ordination and a sufficient level of commitment. As a matter of fact the system has not been successfully deployed yet and a contingency plan still has to be defined.
6. CONCLUSIONS
The outcome of the installation has been very different in the three test sites, although the technology used was exactly the same and the objectives were very similar. The successes obtained seem to confirm the validity of the BOPS concept and of its implementation from a pure technical perspective, but the failures imply the existence of other necessary success conditions, which in some cases have not been accomplished. We have identified the following key success drivers in the deployment of an "information portal" like BOPS:
- Corporate IT culture. A large diffusion of PC usage and the familiarity with a networked communication in a company obviously favour the adoption of a new IT system on the user side. In the case of Siemens, in which IT is part of the core business, a high level of IT culture also helped the developers to speed up the installation: the data population has been easier thanks to the existence of digitized documents, the server deployment has been supported by the company EDP staff, and so on. On the other hand we must say that the lack of corporate IT culture at Caritas lead to relevant delays but did not compromise a final good outcome of the project. Therefore a very basic level of IT culture is definitely a disadvantage when adopting complex information system, but does not preclude the success. Moving from typewriters to portals is possible. It just takes more time and the risk of underestimating the amount of time required is very high.
- Users motivation and involvement. The BOPS project has had a special attention for the change management. The first warning about potential failures due to organizational resistances to change had been given since the very beginning of the project; afterwards, approximately two months before the installation in the test sites, a series of change management activities, like a session to train change agents in each pilot organization and the periodical drawing up of a change management report, have been launched. Results seem to be strictly related to the intensity of these activities. At CCIP they have been neglected. At Siemens and at Caritas they helped to avoid significant resistances to change. Nevertheless they came late and were not of any use to really involve users in the system deployment, taking advantage of their point of view. BOPS is a very information-intensive system; its usefulness is proportional to the quantity of data, information and documents stored in it. Employees are not passive users of this information, but actively participate in its creation. Therefore it would have been useful to involve them in the pilot system design, addressing issues like motivation and change management much earlier, at least four months before the actual installation. Probably this would have helped not to underestimate the data population at Caritas.
- Responsibility and organizational framework. We already mentioned the fact that Le Preau was responsible for the installation of BOPS at DFISE (CCIP). As a matter of fact the two organizations are independent and people from the first were not in the best position to manage a project inside the second one, i.e. they had the responsibility but not the formal authority (though the most complete co-operation). Additionally we should consider that due to its size Le Preau did not have the organizational resources and control of its counterparts, Siemens and Caritas. The combination of a small and not powerful organization, a difficult organizational framework and a complex IT system prevented to achieve the expected results. Even in its simplest form BOPS impacts on several business processes and deeply affects the way in which organizations handle their knowledge and procedures. Those in charge of deploying it must have a proportionate control over the target organization and its active collaboration. Roles and responsibilities must be as clear as possible.
[1] Gery, G. (1997) "Granting three wishes through Performance-Centered Design". Communication of the ACM , July.
[2] Raybould, B. (1995) "Performance Support Engineering: an emerging development methodology for enabling organisational learning". Performance Improvement Quarterly, 8 (1).
[3] Laffey (1995) "Dynamism in Electronic Performance Support Systems". Performance Improvement Quarterly, 8(1).
[4] Kasvi, J.J.J. and Nieminen, M. and Pulkkis A. and Vartiainen, M. (1998) "Knowledge Management on the Shop-Floor" in "Manufacturing agility and hybrid automation" edited by W. Karwowski and R. Goonetilleke.
[5] Clark, R.C. (1993) "EPSS Look before you leap: Some Cautions about Applications of Electronic Performance Support Systems". Performance and Instruction, May.
[6] Collins, A. and Brown, J.S. and Newman, S.E (1990) "Cognitive Apprenticeship: Teaching the crafts of reading, writing, and mathematics" in L. B. Resnick (Ed.).
[7] Duffy, T. and Palmer, J. and Mehlenbacher, B. and Norwood, NJ (1992) "Online help design and evaluation". Ablex Publishing.
[8] Galagan, P. A (1994) "Think performance". Training & Development, 48(3).
[9] Gery, G. (1991) "Electronic performance support systems: How and why to remake the workplace through the strategic application of technology". Boston, MA: Weingarten Publications.
[10] Battles, B. and Mark, D. and Ryan, C. (1996) "An open letter to CEOs: How otherwise good managers spend too much on information technology ". The McKinsey Quarterly, Number 3, pp. 116—127.
[11] Heygate, R.(1993) "Immoderate redesign". The McKinsey Quarterly, Number 1, pp. 73—87.
[12] Willocks, L. and Griffiths,
C. (1994) "Predicting Risk of F
ailure in Large-Scale Information Technology Projects". Technological
Forecasting and Social Change, 47, 205-228.
[13] Butani, S.J. and Phipps, P.A. and Chun, Y.I. (1995) "Research on Establishment-Survey Questionnaire Design". Journal of Business & Economics Statistics, July, Vol. 13, No. 3.
[14] Manzoni, J.F. and Angehrn, A.A. (1998) "Understanding Orgnizational Dynamics of IT-enabled Change: a Multimedia Simulation Approach". Journal of Management Information Systems, 14, 3, 109-140.
[15] Gery, G. (1995) "Attributes and behavior of performance-centered systems". Performance Improvement Quarterly, 8(1).
[16] McGraw, K. (1994) "Performance support systems: Integrating AI, Hypermedia, and CBT to enhance user performance". Journal of Artificial Intelligence in Education, 5(1).
[17] Raybould, B. (1990) "Solving human performance problems with computers". Performance & Instruction, 29(11).
[18] Reeves, T. (1995, February 7) "What is EPSS and why should instructional designers know about them?". Paper presented at the AECT Annual Conference, Anaheim, CA.
[19] Stevens, E. and G. (1996) "The truth about EPSS". Training and Development, 60(6).
[20] Angehrn, A.A. (1993) "Computers that Criticize You: Stimulus-Based Decision Support Systems". Interfaces, 23, 3, 3-16.
[21] Silver, M.S. (1990) "Decision
Support Systems: Directed or Non-Directed Change". Information Systems
Research, 1, 1, 47-70.