The IVT Knowledge Trust

INTRODUCTION

Enhancing project performance during a "schedule crash" in a pharmaceutical environment is no straightforward task. Typically, it entails compressing the project's duration by adding more resources. This approach is high-risk as it usually leads to greater costs and diminished quality unless a solid infrastructure is put in place to mitigate the impact of changing the project plan. Project teams can create a viable infrastructure by using a combination of tried-and-tested project management practicesas well as Lean Six Sigma techniques.

In this article, we examine how to apply project management practices and Lean Six Sigma in a pharmaceutical automation environment to a schedule crash successfully. The article consists of two sections, starting with a case study that describes a successful schedule crash and followed by a brief discussion of baseline and complementary resources required for successful automation complianceand schedule crashing.

CASE STUDY

Crashing a schedule on budget and on quality

Part 1: Project definition and scope

Assess project needs and select the right tools

The subject of this case study is a Distributed Control System (DCS) validation project that was part of the automation of a new biotech pilot plant. In this project, schedule-crashing delivered a complex, compliant DCS to the plant below cost without compromising quality.

The scope of the project was DCS unit- and system-level testing, including:

• 10,000 input/output points
• 50 process cells
• 500 recipes

The project assumptions and baseline were aligned with the client's corporate policies regarding validation. The validation effort benefited from leveraging a controlled software library based on S88 standards, resulting in the elimination of a considerable amount of unnecessary testing. Once tested, the library can be used multiple times and by multiple sites.
The software was developed using a modular approach, allowing the re-use of the same code multiple times. This approach helped build a testing strategy that minimized redundancy and reduced risk. The project spanned approximately 1.3 years and required a team of approximately 15 people with an increase to about 30 during the "crashing" period.

Project Management Institute methodologies such as Earned Value reporting were used. The project involved a high number of deliverables (approximately 7,000 documents) with complex workflows (for remote and neighboring locations). It involved multiple team members and extensive documentation review and approval. The project team exercised rigorous control over the process flow and the progress of deliverables. Lean Six Sigma techniques for key metrics like cycle time measurement were implemented to ensure proper focus on success factors. Specifically, the project management team
used the variance and Earned Value approach to verify the budget and schedule. Earned Value is a forecasted variable used to predict whether the project will finish as per initial estimates.

The initial estimate of work was 11 months for 17 people. However, due to the fact that the facility was a new pilot plant, the project team encountered a series of changes that expanded the scope of the project and created rework. An updated schedule forecasted that the additional work would require an additional 6 months at the existing planned pace. Since the imminence of the launch date was the key business driver for the project, the revised duration was unacceptable. Consequently, the project team needed to determine how to manage a "crash" in the schedule while respecting two
requirements: (1) maintain quality so as not to breach company policies and procedures as well as FDA
requirements; and (2) maintain cost.

As part of the schedule-crashing, the project team was doubled in less than 3 weeks.

Part 2: The challenge

Crash the schedule from 6 to 3 months and ensure project success

As mentioned earlier, the company was faced with two courses of action: (1) maintain current project delivery speed (hence delaying product launch) or (2) crash the schedule in an effort to meet the set launch date. Given that the launch involved multiple products with high value to the business, the decision was made to crash the project schedule by doubling the team within 2 to 3 weeks.

Crashing the schedule comes with a series of problems related both to cost and quality. The dangers of exponential cost overruns during a schedule crash are very real due to challenges tied managing a larger team and the potential for unforeseen inefficiencies. Quality may also be affected by the rapid ramp-up of the project team's size with new members whose training or knowledge set may not be sufficiently aligned with the project's specific dynamics.

As part of the transition planning, additional supervision requirements were identified to ensure smooth transitioning to the higher productivity model. The new work model involved striking the proper balance between overtime and bringing in new talent so as not to impact the team with attrition or burn out. Finally, ideas to maintain morale were discussed and planned. The level of outsourcing support was evaluated so as to help manage the rapid increase required to succeed.

During the project, the team continuously analyzed the process by using Lean Six Sigma tools. The team kept focusing on critical path activities to ensure that all efforts were kept on schedule (the crashed schedule); all the while, the team prepared for the next critical path activity to ensure success at each step. It devised a process flow diagram with key input and output variables and analyzed value- and non-value-added activities.

Figure 1: Improvement Area Process Map

This process map was used to discuss improvement areas for existing activities before crashing the
schedule. Duration and effort were reviewed and assessed for each activity. Opportunities for
becoming leaner were identified and later implemented.

A fishbone diagram was prepared to break out the root-cause categories of issues that had an impact on the process. Based on these results, the next drill-down exercise was to analyze why the scripts did not match the design specifications, repeating the combination of fishbone and Pareto tools to further isolate high-impact root causes. The following root causes were identified and analyzed using a Pareto chart (refer to Figure 2).

Not Match Design: Issues occurring when the software script (i.e. documented test case) did not match the design specification. This resulted in corrections to either test script documents or design specification.

Test Case Outline: Issues occurring when the test script cannot be executed as documented. Insufficient Information: Issues occurring when the design specification did not have sufficient information to write the test script.

Coverage: Issues occurring when the structure of the test script does not cover all potential testing scenarios from the design specification.

Not used template: Issues occurring due to formatting such as footer/header errors, page break, numbering.

Not used guideline: Issues occurring due to not respecting guideline information such as phases, interlock strategy.

File Name: Issues occurring specifically with respect to improper file naming of scripts, design specifications.

Design Issues: Issues occurring in design specification such as missing information, not matching functional requirements.

The team determined that one of the main causes was that the scripts were written according to an older version of the design and had not been updated. A tighter control prior to review was implemented, and the impact of these issues was significantly reduced.

Figure 2: Review of significant issues - Pareto

This process map was used to discuss improvement areas for existing activities before crashing the schedule. Duration and effort were reviewed and assessed for each activity. Opportunities for becoming leaner were identified and later implemented.

Issues with the least impact on the process were removed from the reviewing activities, further reducing the time needed to review the scripts. (Fortunately, negligible-impact issues were few in number, which made their removal that much faster.) Some of the key process flows were also further mapped according to Lean principles focused on cycle times, value-added and non-value-added activities.

Figure 3: Test Case Development and Execution

A snapshot of the activities required to approve test scripts for execution, highlighting the potential for lean improvements in both value- and non-value-added activities.

As expected, crashing the schedule had a huge impact on project management activities. Due to its high level of risk, crashing should be attempted only once the team is convinced that it is fast-tracking the project, that is has placed all activities in overlapping or parallel states to reduce duration as much as possible.

A countermeasures matrix was built to ensure that all potential risks of crashing were mitigated. Some of the key standard risks identified involved lack of control over resources, unclear project processes, diminished quality in deliverables, and unclear training guidelines for the ramped-up team.

Dealing with issues identified in the case study

The current Work Breakdown Structure (WBS) was reviewed to identify the impact of crashing on each activity. The detailed project plan was revised for each task in light of the accelerated project timeline. Root-cause analysis guided continuous improvement, and Lean techniques ensured improved workflow, removing delays and non-value-added steps.

Figure 4: Work Breakdown Structure (WBS)

The Work Breakdown Structure (WBS) provides the number of test cases and associated tasks for each
process cell - hence, the overall scope for the project. (Note: actual numbers of units and test cases
were factored.)

Daily reporting replaced weekly reporting for further control on Earned Value (EV) enabling better monitoring and control of project progress. The key metrics were visually communicated to the entire team, who became focused at surpassing the daily goals. EV provides a tangible measure of the actual progress of work in relation to the planned value for the work. Too often, project teams measure their progress against budgeted hours without any consideration for the deliverables. As a result, the project team is not focused on completing the actual tasks in the required time. The team gets sidetracked by
preparing other deliverables that were not in the original scope, or by enhancing the level of quality beyond the planned scope (scope creep).

The required granularity and frequency are important for EV management. A simple, visual graph of EV reported monthly (refer to figure 5) combined by detailed daily deliverable progress in the form of a table enabled the team to truly focus on completing critical daily tasks. The team understood each task that needed to be completed at the test script level and executed accordingly. A "Red Flag" alarm system was implemented to rapidly assess the situation and remove roadblocks to ensure success. The leadership objectives were focused on the roadblocks as planning, process flows, and deliverables were aligned and made clear to all the players (software programmers, testers, process engineers, document controllers etc.).

As part of the project crash, a revised detailed communication plan was presented to project stakeholders - the procurement, quality, engineering, automation, and validation teams - delivering real-time progress of all critical-path activities.

Figure 5: Earned Value Testing Effort

The Earned Value (EV) graph tracks project progress after crashing the schedule. The initial budget was maintained - a key attribute in defining project success. (Note: actual hours were factored.)

Lessons learned from the case study

Applying the principles of prototyping the project, the project team sat down and analyzed the data after 3-4 process cells delivered. A series of lessons were drawn and action plans were developed and implemented:

1. Fix all problems prior to execution in all cloned process cells so as not to propagate any issues unnecessarily.
   The project benefited from replication of similar process cells that enabled accelerated coding and testing. In order to further streamline the process, it was critical that the first prototype be completely tested and all generated corrective actions considered prior to the replication process.
2. Make sure the Steam In Place (SIP) and Clean In Place (CIP) charts are updated prior to testing2. Make sure the Steam In Place (SIP) and Clean In Place (CIP) charts are updated prior to testing. SIP and CIP are processes that impact the majority of process cells. Due to their complexity, SIP and CIP are often subject to multiple changes (daily!). Communication of the modifications is critical to minimize rework. Prior to executing the scripts, the team performed an informal check to ensure that the latest versions of the charts were used.
3. No additional technical review is needed for all cloned test scripts as the work is redundant. Since an extensive assessment of the prototype was performed, including all associated documentation, the test scripts quality was assured and the team decided that the risk of removing the technical review was minimal.
4. Remove extensive comments in test summaries not required by the quality team. Quality decided that all comments put in comments section of the test scripts needed to be transferred to the test summary. A tremendous amount of effort was required to transfer, review and track the comments. The team decided to minimize the use of comments that provided little benefit to the particular test. Such a reduction, enabled savings in time and higher quality deliverables. The test summary was less likely to contain errors and was focused on the value add activities.
5. Post review of test scripts should be done just in time after the execution of the test cases so as to correct any items immediately and not lose momentum. Initially, the test scripts were accumulated, and momentum to complete was somewhat diminished as the Quality team were only able to review 3-4 weeks later due to their narrow bandwidth. The project team decided that once all test scripts were executed, a review and approval session was held with Quality. Quality augmented their capacity to accommodate the accelerated review process. All key stakeholders were summoned to a review and approve "party." All issues were immediately identified and rectified with high priority. In addition, any new quality requirements were added to the checklists of the team to ensure that other deliverables were appropriately prepared and executed.
6. All documents must be prepared prior to execution to ensure focus on execution not on documents or peripherals.
   The team realized that by having all the tools and items ready prior to the start ensured easy tracking of all test scripts and issues. Having to prepare binders or print documents during the testing sometimes caused minor distractions impacting schedule and quality.

Conclusion

Crashing should be considered only after fast-tracking (overlapping all tasks as much as possible) has been optimized. Crashing the project requires rigorous movement toward effective control of the project through instructions, statistical monitoring of performance (control of the work processes), and, when possible, mistake-proofing (see Figure 6, page 10).
It should be recognized that crashing has become a way of life for validation projects - and project teams are embracing accelerated timelines as a normal part of their projects. Solid project management understanding and know how, ensure flexibility to adjust the course of the project efficiently and effectively. The team must also be ready to cope with a high degree of stress due to heightened expectations. Hence, the leadership must motivate and communicate much more regularly.

Figure 6: Keeping control despite diminishing effort

Diminishing the effort for validation activities includes moving toward tools for more effective deployment of information and communication. The project was managed with the vision of moving toward mistake-proofing and statistical control when possible.

Due to the pharmaceutical industry's continuous focus on cost, businesses are innovatively trying to derive maximum value of each dollar invested in projects. Solid project management techniques combined with Lean Six Sigma provide the required framework to align metrics with execution for success - whether crashing or simply executing a project. This scientific methodology translates into less effort and greater control over the project during execution.

In this environment, project teams need to consider the time, dollar and quality continuums. Mitigating risks in each of these areas in a schedule crash situation is essential for meeting both project-team and stakeholder objectives.

DISCUSSION

Building an effective approach for schedule crashing

Using a baseline and selecting appropriate supporting standards

Facing multiple challenges is part of delivering a compliant automated system within a regulated environment. Management and the project team must address specific project needs to build the appropriate framework during initiation efforts. For automation projects in the pharmaceutical and biotech industries, there are four key areas that must be mastered to deliver a high-value, compliant, integrated project, namely:

• Legal regulations
• Industry standards (beyond the project baseline provided by PMBOK)
• Company policies, guidelines and procedures
• Tools and project methodologies (where Lean Six Sigma provides key tools and processes for successful project execution)

Successful automation compliance: Legal regulations

Some of the key regulations pertinent to this type of automation project are:

• Food and Drug Administration (FDA) cGMPs such as 21CFR Parts 210 and Part 211
• Major statutes under the Environmental Protection Agency (EPA)
• Respect of required legal acts as outlined by the Occupational Safety & Health Administration (OSHA)

National Electric Code (NEC)

Defining how the implementation of such a system meets the intended spirit of local, state and federal regulations is crucial. Laws are subject to interpretation, and understanding what is realistic and truly current Good Manufacturing Practice can make the difference between high cost and right cost.

Projects conducted in a regulatory environment must provide documented evidence that the product meets pre-determined specifications and quality. Extensive knowledge and experience of regulation requirements are needed to meet compliance requirements effectively. The "c" (current) in cGMPs becomes critical as interpretations of the legal requirements are in continual evolution, and team knowledge must be assessed frequently to ensure that it is current with regulations. Laws may offer much room for interpretation; therefore, interpretation is one of the greatest challenges for the industry. While it is cooperative to a degree, in general the industry is also quite conservative. Project team members may, in their quest for continuous improvement, create scope creep unnecessarily, resulting in overspending - a concern to financial stakeholders. Such outcomes, compounded over multiple projects, result in massive costs that, in the end, are transferred to patients.

Figure 7: Business Drivers for Success

In accordance with PMBOK, successful projects selectively mesh knowledge and practice components to ensure focused planning and execution. The above illustration has been customized from the PMBOK and adapted to the case study presented in this article.

The importance of company policies, guidelines and procedures

Without belaboring the point, another important prism for interpreting regulations is the company's own decision-making infrastructure, including its policies, guidelines and procedures:

• Policies: where high-level direction is provided in support of the organization's efforts to ensure legal compliance
• Guidelines: where the company's recommended practices are identified and expectations spelled out
• Procedures: where detailed, compulsory instructions are instituted so as to ensure compliance where deemed critical by the company

Alignment with the above has been shown to yield greater customer satisfaction on a consistent basis.

Getting buy-in from senior management

No project team can get its work done without persuading senior management that its approach will yield benefits in terms of time and dollars - hence the crucial importance of getting senior management on-board. Project teams can build a strong case by focusing on the following considerations:

• Use an integrated, focused project approach that combines proper guidelines, standards, and tools to complete a project successfully and to meet critical customer requirements
• Clearly identify Key Performance Indicators (KPI) and align them with business strategy (for example, it does not make sense to spend extra money to accelerate the project when time is not of the essence). KPIs need to be SMART (Specific, Measurable, Achievable, Relevant and Time Based). Post-project, KPIs must re-examined in order to extract lessons learned
• Adhere to appropriate legal requirements, which helps prevent unnecessary expenditures of time and effort on non-required areas (for example, overemphasizing testing on systems that are of low risk to the product, the customer and the business)
• Assess, understand and mitigate risk appropriately at the beginning of the project, which allows the project team to provide the right level of "insurance" at the right time in the right conditions Apply the right tools and knowledge areas at the right time in accordance with standards (for instance, to prevent team members from unnecessarily using new tools and creating scope creep)
• Ensure that the project as a whole is properly aligned for scope, cost and schedule. With a welldesigned framework in place, activities can be executed in a well-orchestrated fashion. As a result, the project will yield efficiencies in two often wasted resources: time and money
• Define clear expectations across the project team and stakeholders as well as a well-defined project process, which is measurable and controllable. This helps establish a spirit of continuous improvement with heightened awareness on how to improve the overall KPIs

STANDARDS QUICK REFERENCE

Find out more about relevant industry standards and the organizations behind them: PMI (www.pmi.org) - ISA (www.isa.org) - ISPE (www.ispe.org) - IEEE-SA (www.standards.ieee.org)

ABOUT THE AUTHORS

Mark Cupryk is the VP and General Manager of Operations for Invensys Validation Technologies. He holds a Chemical Engineering degree from McGill University and a Master in Business Administration from Concordia University. He is also a certified Project Management Professional from the Pennsylvania Project Management Institute with a Master in Project Management. He has worked in automation and validation for over 15 years and is Lean Six Sigma Black Belt. Mark can be reached by phone at (617) 899-9264, by fax at (514) 485-6617 or by e-mail at mark.cupryk@ips.invensys.com.

Doina Morusca is a Senior Project Manager for Invensys Validation Technologies. She is specialized in both business and manufacturing systems with Lean Six Sigma. She holds a Master in Business Administration from Concordia University as well as a Master in Education. She is also a certified Project Management Professional from the Pennsylvania Project Management Institute. She can be reached by phone at (508) 878-6141 or by e-mail at doina.morusca@ips.invensys.com.

Dean Takahata is a Project Manager for Amgen. He holds an Engineering degree from Miami University and has worked in automation and project management for over 22 years. He has held various positions in project operations and software product development with ABB and Invensys. He can be reached by e-mail at deant@amgen.com.

ABOUT IVT

Invensys Validation Technologies (IVT) is recognized as the industry leader in global regulatory compliance and business
systems solutions for the life science and process industries . Services encompass the entire enterprise: equipment, process
control, cleaning control, automation, manufacturing execution systems, and information technology.

IVT MISSION STATEMENT

"Together, we bring excellence"

We regard these values as essential to bring excellence;

Integrity - We are honest and true to our mission. It is the foundation of our actions.
Team Work - We endeavor to bring out the best in each other; to be more creative, productive and accountable.
Respect - We acknowledge individual's talents and differences.

We care about each other, our customers and partners. We communicate with openness and sincerity.

Improvement - We relentlessly seek ways to do better internally and with our customers.
Leadership - We believe that leadership isn't about position; it's about the attitude, skills and expertise we show every day.