Undertaking a major e-learning programme involves a large investment both in terms of money and of the resources involved. It is also a new and untested world for many organisations. If the initial projects do not deliver the expected results, there often
Undertaking a major e-learning programme involves a large investment both in terms of money and of the resources involved. It is also a new and untested world for many organisations. If the initial projects do not deliver the expected results, there often isn’t a second chance, so it is important to "get it right first time". To minimise risk and gain valuable experience, the best course of action is to start with a suitable pilot project.
Ideally there should be a shortlist of 3 – 6 candidate projects, as each will have its own strengths and weaknesses. The process below provides a framework for reviewing and comparing the candidates.
There are three key aspects to consider when selecting a candidate for a pilot project:
Relevance, Value and Deliverability
RELEVANCE
Subject
Is the subject matter ‘typical’ of the subjects that will be used in future? There is no point in picking a subject which has no relevance or interest to the company or to future project development teams. The subject also has to be one that is suitable for e-learning in the first place, or at least has a large component which is applicable to e-learning. In reality just about any subject can be covered via e-learning, but there is far more effort and creativity involved with those which do not naturally fit the mould. A pilot project will often have enough challenges in it without selecting a particularly difficult subject. The pilot will be used to sell the concept of e-learning across the organisation, so it must be a subject that is perceived by a wider audience as being acceptable, particularly by the decision-makers.
Ownership
Who owns the area proposed for the pilot? Is the department/group/leader well thought of and respected within the organisation? Is it a central/mainstream group? Do they have the capability of selling the value of e-learning to others after the pilot is complete? Learning can often be thought of as a subject of secondary importance in business, so the chosen pilot project needs a strong champion who will drive it forwards.
On the other hand, beware of a political situation, where there may be a strong advocate but with an underlying hidden agenda, which will not best serve the pilot or the longer term programme.
Scope
Is the subject one that can display a number of different aspects of e-learning in one pilot? For example, can there be some self-study work, a discussion, a group exercise, an online presentation? Does the pilot give opportunities to experiment with some new techniques or approaches, such as multimedia simulations? Can an old and tired subject be given a new slant? A pilot needs to demonstrate something new and unique, to provide experience for the company and to act as a marketing tool. At the same time, when selecting the subject matter, care must be taken to ensure that a reasonably sized self-contained segment can be extracted to create the pilot. Often the pilot scope has to be limited in order to be able to demonstrate a quality deliverable quickly, so that the bigger programme can then commence. A good measure would be to aim for something which could be developed in 6-8 weeks, was discrete, and still demonstrated a variety of features.
VALUE
Return on Investment
The deliverable from the pilot must demonstrate value to those who will judge it, and show a return on the investment. Will this pilot be perceived as adding value, despite the fact that it may take more effort, with a larger learning curve than subsequent projects? Even though it is a pilot, the aim should be to save either hard cash or demonstrate softer benefits in time saved, or skill changes. A course on a very specialist subject that will only be used once every two years by a limited audience is not a good pilot candidate. A segment of an induction programme, which normally requires a senior manager to travel for 2 days every month, has a much more obvious return.
Objectives and priorities
It should be easy to define a clear set of objectives and priorities for a well-chosen pilot. Progress against the objectives can then be measured on completion of the pilot, in order to measure success.
Marketing
Is this pilot one that can be used easily as a marketing tool for a larger e-learning programme? It needs to have credibility both within the organisation, and if applicable, to any external customers. Is it something you would feel proud to demonstrate in public at a conference?
Learning
The purpose of a pilot is not just in creating a deliverable e-learning module. Pilots test out the design, development and implementation processes. They highlight skill shortages or hidden capabilities within existing staff. They provide metrics against which future projects can be estimated. If experienced practitioners or consultants are involved in the pilot project to ensure its success, then the pilot project must be designed to extract maximum learning and experience from those people. Will the selected pilot be able to make use of the appropriate resources to maximise learning?
Repeatability
Apart from the knowledge transfer opportunities described above, the pilot must be used to determine standards and guidelines for future projects. Templates and tools should be developed and refined along the way, and reviewed once the pilot is complete, prior to subsequent projects being initiated. Standard processes and design guidelines should be documented. All these items ensure that the pilot is not a one-off project but one which is then repeatable, not just by the current team, but by new personnel. Therefore the focus for the people participating in the pilot should be on generating a more efficient and cost effective process for the future.
Scalability
The processes, people and content of a pilot must be scalable to larger projects. Although it may be possible to develop and run a pilot with one person acting in all roles from designer to developer to IT support head to online facilitator, the pilot will not deemed a success as it is not scalable to the real world of large projects. The pilot must try and be as representative as possible of standard conditions of work.
DELIVERABILITY
Resources
Can the selected pilot make use of appropriate resources? Do they have the right basic skills, attitudes and willingness to work through the pilot process and then sell the results? Are the people available in the required timeframe? A pilot project is often very intense work and 50% of a person, or one who has been allocated to work ‘in his spare time’, will not contribute to successful outcomes. The majority of participants on the pilot need to be full-time. Most e-learning projects involve people stretching or changing their roles and approaches to development, training, and support, so resources with the right mental attitudes and open minds are absolutely critical for the pilot. The project staff will normally have to take on the roles of super-users once the pilot is complete, so that new skills they have gained can be transferred to others. As in any project the people are critical, and a pilot with good people but a less suitable subject should be chosen over one with poor resources and a good subject.
The "students" or pilot testers also need to be available in sufficient numbers, at the right time, and with a suitable level of pre-course knowledge. You may need to over-recruit testers to ensure that actual numbers do not fall too low by the time the pilot is run. Often with pilots the strong views of a small body of people can sway opinion on future direction, so there needs to be a big enough body of pilot testers of the right experience to ensure that representative feedback is obtained.
Work
How much work will be required for the pilot? Is there a knowledgeable subject matter expert who can answer questions and support the design process? Is there any existing material that can be used as a foundation? How well-designed is it? Are there any materials in electronic form? Are there good exercises and activities which easily translate into being conducted online? How much work will it require to actually run this pilot once it has been developed? Is it feasible? Many pilots do require much more effort than ongoing programmes with experienced facilitators, so it makes sense to consider a project which is not going to be a time burden from the outset.
Technical feasibility
Do you have the technical infrastructure and staff to implement, manage and support the pilot? If not, can you buy the services through a hosted solution? Are there any issues, such as getting access outside the firewall which may impact the pilot? Is the pilot designed to run on the standard PC platform available to most employees? If it is not, then is the upgrade process straightforward or will it require a major project in itself? As a minimum step, examine the IT strategy and ensure that your long term plan fits into that model, even if there may be some additional work for the pilot run. Do not select a pilot which will be outside the IT roadmap. There is never an ideal time to implement any technological solution, so do not reject a pilot if it requires some technical work; that is what piloting is all about. Ensure that the pilot has a feasible IT plan running in parallel with any content development efforts, including full testing prior to the launch of the pilot.
Timescales
Do the proposed timescales for the pilot fit in with the business calendar? Will you be able to launch the pilot at an appropriate time? If the pilot is critical there is little point in starting it during a major holiday period, for example. The timing of the pilot also needs to be driven by the demand for the subject. If it is a new hire topic, it needs to be launched at a time when there are some willing new staff to test it. The launch date may also be dependent on your ability to market the solution to potential pilot testers, which may involve the development of a prototype or demonstration model to encourage participation, all of which take time.
Is the due date fixed? A pilot with a date cast in stone may be less suitable than one with equivalent qualities but more flexible timescales. Pilot development projects, especially in e-learning can stretch beyond initial delivery dates, often because of new ways of working and resource constraints.
External constraints
Are there any external constraints in a given pilot which would eliminate it from the list of possible options? For example, is there copyright on material that you want to use, for which you may not be able to gain permission? Are there any external legal constraints? Are there any Government directives which may impact either its implementation or the timing of the implementation? Are there any external bodies from which you need to gain permission before implementation?
Next steps
It is a simple task to take the topics above and use a spreadsheet to rate each of your potential pilot projects, providing both a numerical and subjective analysis of the strengths and weaknesses of each candidate. Usually one of two will appear as clear front runners and the decision-making process should then be much simpler. Go for the pilot that will be representative of future projects, deliver clear value and be feasible to implement and run.
Eleonora Panto
Dario Zucchini