“Crashing the schedule” in DCS Validation Pharma Projects

In projects where speed to market is the key business driver, a schedule crash can deliver project success on budget and on quality. In this case study, we examine how to conduct a successful schedule crash in a pharmaceutical automation environment. Implementing a schedule crash is no straightforward task as it entails compressing the project’s duration by adding more resources. To curtail the risk of higher costs and reduced quality, the project team created a viable project infrastructure using proven project management practices and Lean Six Sigma techniques.

The challenge: crash a schedule on budget and on quality

1. Define project needs and appropriate tools to meet objectives

The subject of this case study is a Distributed Control System (DCS) validation project that was part of the automation of a new biotech pilot plant. In this project, schedule-crashing delivered a complex, compliant DCS to the plant below cost without compromising quality.

The scope of the project was DCS unit- and system-level testing, including:

• ~ 10,000 input/output points
• ~ 50 process cells
• ~ 500 recipes

Project assumptions and baseline were aligned with the client’s corporate policies re validation. A controlled software library based on S88 standards was used to eliminate a considerable amount of unnecessary testing. Once tested, the library can be used multiple times and by multiple sites. The software was developed using a modular approach, allowing the re-use of the same code multiple times. This approach helped build a testing strategy that minimized redundancy and reduced risk. The project spanned approximately 1.3 years and required a team of approximately 15 people with an increase to about 30 (in under 3 weeks) during the “crashing” period.

Project Management Institute methodologies such as Earned Value reporting were used. The project involved a high number of deliverables (approximately 7,000 documents) with complex workflows (for remote and neighboring locations). It involved multiple team members and extensive documentation review and approval. The project team exercised rigorous control over the process flow and the progress of deliverables. Lean Six Sigma techniques for key metrics like cycle time measurement were implemented to ensure proper focus on success factors. Specifically, the project management team used the variance and Earned Value approach to verify the budget and schedule. Earned Value is a forecasted variable used to predict whether the project will finish as per initial estimates.

The initial estimate of work was 11 months; however, since the facility was a new pilot plant, the project team encountered changes that expanded the scope of the project and created rework. The additional work would have added 6 months. Since a prompt launch was the project’s key business driver, the revised duration was unacceptable. Consequently, the project team determined how to implement a “crash” in the schedule while: (1) maintaining quality in accordance with company policies and FDA requirements; and (2) containing cost.

2. Crash the schedule from 6 to 3 months and ensure project success

As mentioned earlier, the company was faced with two courses of action: (1) maintain current project delivery speed (hence delaying product launch) or (2) crash the schedule in an effort to meet the set launch date. Given that the launch involved multiple products with high value to the business, the decision was made to crash the project schedule by doubling the team within 2 to 3 weeks.

Crashing the schedule comes with a series of problems related both to cost and quality. The dangers of exponential cost overruns during a schedule crash are very real due to challenges tied managing a larger team and the potential for unforeseen inefficiencies. Quality may also be affected by the rapid ramp-up of the project team’s size with new members whose training or knowledge set may not be sufficiently aligned with the project’s specific dynamics.

As part of the transition planning, additional supervision requirements were identified to ensure smooth transitioning to the higher productivity model. The new work model involved striking the proper balance between overtime and bringing in new talent so as not to impact the team with attrition or burn out. Finally, ideas to maintain morale were discussed and planned. The level of out-sourcing support was evaluated so as to help manage the rapid increase required to succeed.

During the project, the team continuously analyzed the process by using Lean Six Sigma tools. The team kept focusing on critical path activities to ensure that all efforts were kept on schedule (the crashed schedule); all the while, the team prepared for the next critical path activity to ensure success at each step. It devised a process flow diagram with key input and output variables and analyzed value- and non-value-added activities.

A fishbone diagram was prepared to break out the root-cause categories of issues that had an impact on the process. Based on these results, the next drill-down exercise was to analyze why the scripts did not match the design specifications, repeating the combination of fishbone and Pareto tools to further isolate high-impact root causes.

The following root causes were identified and analyzed using a Pareto chart:

• Not Match Design: Issues occurring when the software script (i.e. documented test case) did not match the design specification. This resulted in corrections to either test script documents or design specification.
• Test Case Outline: Issues occurring when the test script cannot be executed as documented.
• Insufficient Information: Issues occurring when the design specification did not have sufficient information to write the test script.
• Coverage: Issues occurring when the structure of the test script does not cover all potential testing scenarios from the design specification.
• Not used template: Issues occurring due to formatting such as footer/header errors, page break, numbering.
• Not used guideline: Issues occurring due to not respecting guideline information such as phases, interlock strategy.
• File Name: Issues occurring specifically with respect to improper file naming of scripts, design specifications.
• Design Issues: Issues occurring in design specification such as missing information, not matching functional requirements.

The team determined that one of the main causes was that the scripts were written according to an older version of the design and had not been updated. A tighter control prior to review was implemented, and the impact of these issues was significantly reduced.

Issues with the least impact on the process were removed from the reviewing activities, further reducing the time needed to review the scripts. (Fortunately, negligible-impact issues were few in number, which made their removal that much faster.) Some of the key process flows were also further mapped according to Lean principles focused on cycle times, value-added and non-value-added activities.

As expected, crashing the schedule had a huge impact on project management activities. Due to its high level of risk, crashing should be attempted only once the team is convinced that it is fast-tracking the project and placed all activities in overlapping or parallel states.

A countermeasures matrix was built to ensure that all potential risks of crashing were mitigated. Some of the key standard risks identified involved lack of control over resources, unclear project processes, diminished quality in deliverables, and unclear training guidelines for the ramped-up team.

Dealing with issues identified in the case study

The Work Breakdown Structure (WBS) was reviewed to identify the impact of crashing on each activity. The detailed project plan was revised for each task in light of the accelerated project timeline. Root-cause analysis guided continuous improvement, and Lean techniques ensured improved workflow, removing delays and non-value-added steps.

Daily reporting replaced weekly reporting for further control on Earned Value (EV) enabling better monitoring and control of project progress. The key metrics were visually communicated to the entire team, who became focused at surpassing the daily goals. EV provides a tangible measure of the actual progress of work in relation to the planned value for the work. Too often, project teams measure their progress against budgeted hours without any consideration for the deliverables. As a result, the project team is not focused on completing the actual tasks in the required time. The team gets sidetracked by preparing other deliverables that were not in the original scope, or by enhancing the level of quality beyond the planned scope (scope creep).

The required granularity and frequency are important for EV management. A simple, visual graph of EV reported monthly combined by detailed daily deliverable progress in the form of a table enabled the team to truly focus on completing critical daily tasks. The team understood each task that needed to be completed at the test script level and executed accordingly. A “Red Flag” alarm system was implemented to rapidly assess the situation and remove roadblocks to ensure success. The leadership objectives were focused on the roadblocks as planning, process flows, and deliverables were aligned and made clear to all the players (software programmers, testers, process engineers, document controllers etc.).

As part of the project crash, a revised detailed communication plan was presented to project stakeholders – the procurement, quality, engineering, automation, and validation teams – delivering real-time progress of all critical-path activities.

Lessons learned from the case study

Applying the principles of project prototyping, the project team analyzed the data after 3-4 process cells delivered. A series of lessons were drawn and action plans were developed and implemented:

1. Fix all problems prior to execution in all cloned process cells so as not to propagate any issues unnecessarily. The project benefited from replication of similar process cells that enabled accelerated coding and testing. In order to further streamline the process, it was critical that the first prototype be completely tested and all generated corrective actions considered prior to the replication process.
2. Make sure the Steam In Place (SIP) and Clean In Place (CIP) charts are updated prior to testing. SIP and CIP are processes that impact the majority of process cells. Due to their complexity, SIP and CIP are often subject to multiple changes (daily!). Communication of the modifications is critical to minimize rework. Prior to executing the scripts, the team performed an informal check to ensure that the latest versions of the charts were used.
3. No additional technical review is needed for all cloned test scripts as the work is redundant. Since an extensive assessment of the prototype was performed, including all associated documentation, the test scripts quality was assured and the team decided that the risk of removing the technical review was minimal.
4. Remove extensive comments in test summaries not required by the quality team. Quality decided that all comments put in comments section of the test scripts needed to be transferred to the test summary. A tremendous amount of effort was required to transfer, review and track the comments. The team decided to minimize the use of comments that provided little benefit to the particular test. Such a reduction, enabled savings in time and higher quality deliverables. The test summary was less likely to contain errors and was focused on the value add activities.
5. Post review of test scripts should be done just in time after the execution of the test cases so as to correct any items immediately and not lose momentum. Initially, the test scripts were accumulated, and momentum to complete was somewhat diminished as the Quality team were only able to review 3-4 weeks later due to their narrow bandwidth. The project team decided that once all test scripts were executed, a review and approval session was held with Quality. Quality augmented their capacity to accommodate the accelerated review process. All key stakeholders were summoned to a review and approve “party.” All issues were immediately identified and rectified with high priority. In addition, any new quality requirements were added to the checklists of the team to ensure that other deliverables were appropriately prepared and executed.
6. All documents must be prepared prior to execution to ensure focus on execution not on documents or peripherals. The team realized that by having all the tools and items ready prior to the start ensured easy tracking of all test scripts and issues. Having to prepare binders or print documents during the testing sometimes caused minor distractions impacting schedule and quality.

Conclusion

Crashing is an approach that should be considered only after fast-tracking (overlapping all tasks as much as possible) has been optimized. Due to the pharmaceutical industry’s continuous focus on cost, businesses are innovatively trying to derive maximum value from each dollar invested in projects. In this environment, project teams need to consider the time, dollar and quality continuums; mitigating risks in each of these areas in a schedule crash situation is essential for meeting both project-team and stakeholder objectives. Finally, a combination of solid project management practices and Lean Six Sigma techniques will provide the required framework for aligning metrics with execution – and ensure project success.

Read the full article

For more information on this case study, please download a PDF version of the complete article – including a Discussion section and supporting graphics – as it appeared in the IVT Article document.