Research · · Verified
How Much Rework Follows a Change in a Client Source System?
A before-and-after cohort method for measuring rework after a source-system change, separating migration defects from instruction gaps.
On this page
Research question and decision supported
The question is how much rework follows a change in a client source system, and whether the rework comes from the migration, the new instructions, or the team's adaptation. The decision supported is whether to extend hypercare, add a temporary review, revise the training, or change the changeover plan. A source-system change can alter fields, labels, defaults, and report timings even when the underlying process looks the same. The method compares work before and after the change with a consistent rework definition. It does not assume that every increase is a team performance problem.
State whether the decision is about hypercare, training, the changeover plan, or the review standard. The rework rate alone does not choose among them. Pair the rate with the cause classification before recommending a response. The decision should name the owner who can make the change.
Define the change event
Record the date the new system or workflow became authoritative, not the date of the announcement or the training. If the change is phased, define each phase and the tasks affected. List the fields, screens, integrations, and reports that changed, and note which ones the offshore role uses. Identify any period when both systems were live, because dual entry can create its own errors. State the version of the instruction and the training that applied after the change. A clear event definition keeps the before-and-after comparison from mixing different transitions.
List the changed fields and screens that the offshore role actually uses. A system change that does not touch the role's work may still affect the queue through a report or a downstream handoff. Record the training date and the instruction version for each phase. The event definition should be specific enough to reproduce.
Build the before-and-after cohort
Select completed work from a stable baseline period before the change and from the period after the new system went live. Match the cohorts by task type, complexity, client, and shift. Include the same review method in both periods where possible. If the team changed the sample or the reviewer at the same time, record it as a confounder. Exclude work that was completed during the dual-run period unless the analysis is specifically about dual entry. Report the cohort size and the number of items excluded with the reason.
Show the size and the task mix of both cohorts before comparing the rates. If the mix changed, use stratification or report the comparison by task type. Exclude dual-run work unless the study is about dual entry. State how many items were excluded and why.
Classify the rework causes
Use a short taxonomy: migration defect, missing field or mapping, unclear new instruction, access or permission issue, training gap, execution error, and client change request. Ask the reviewer to choose one primary cause at the moment of return, using the new system's evidence. Do not let a general label such as system issue absorb every problem. Review a sample of each code with the system owner to confirm the classification. If most rework cites a migration defect, the fix belongs with the implementation team rather than the offshore desk. If most cites an unclear instruction, the fix belongs with the process owner.
Have the system owner review a sample of migration-defect codes. If the owner disagrees, the code may be describing an instruction gap instead. Record the disagreement and refine the definitions. The classification is more credible when the implementation team and the offshore team agree on it.
Measure the change curve
Plot the rework rate by week for the first eight to twelve weeks after the change. Show the baseline rate as a reference band rather than a single point. A spike followed by a decline suggests an adaptation or hypercare effect, while a flat increase suggests a persistent design problem. Report the median time from go-live to the first week within the baseline band. Add the volume and the number of open exceptions, because a low rework rate during a backlog can be misleading. Do not compare only the first week with the last week and call the difference a trend.
Plot the weekly rate with the volume and the number of open exceptions. A spike during a low-volume week may be less important than it looks. Mark the training dates and the system fixes on the chart. The timeline helps distinguish a learning curve from a persistent defect.
Separate migration defects from new instructions
A migration defect is a mismatch between the old and new system that the team could not have avoided, such as a dropped field or a changed default. An instruction gap is a case where the new system works but the team was not told how to use it. Read a sample of both and confirm with the system owner. Report them separately, because a migration defect should be fixed by the implementation team and an instruction gap by the process owner. If the team is blamed for both, the change review will not fix the actual cause. The split also shows whether hypercare is targeting the right problem.
Report the two categories with separate counts and examples. If a defect was fixed during the period, show the date and the rework after the fix. A category that declines after a fix is evidence that the classification is meaningful. A category that stays flat may need a different intervention.
Compare by role and task type
Some roles depend on fields that changed, while others use a stable screen. Group the rework by role, task type, and the number of changed fields the task touches. A task that uses three changed fields is more exposed than one that uses a single stable view. Compare the rate for new hires and experienced staff, because a new hire may not have an old-system habit to unlearn. Check whether one shift received training later than another. The comparison should identify where the change burden is concentrated, not just the average across the team.
Include the number of changed fields each task uses and the training date for each role. A role that was trained early may have a different curve from one trained late. Compare new hires and experienced staff separately. The pattern can show whether the issue is tool familiarity or old-habit interference.
Use the result to change one support
Choose a change that matches the dominant cause. If migration defects dominate, extend the defect log and the implementation review rather than adding more offshore training. If instruction gaps dominate, add a short job aid and a worked example for the changed fields. If access issues dominate, fix permissions before the next phase. Assign an owner and a review date, then repeat the cohort measurement with the same rework definition. Do not change the taxonomy and the training at the same time if the team needs to know which one helped. Record the change alongside the data.
Choose a support that matches the dominant cause and can be removed when the curve returns to baseline. Set an end date for the extra review so it does not become permanent. Record the cost of the support alongside the rework it prevents. A temporary control should have a planned exit.
Limitations
The method is observational and cannot separate the system change from a simultaneous client, volume, or staffing change. Rework records may be incomplete, and reviewers may classify the same issue differently. A small cohort or a short post-change window can produce an unstable rate. The baseline may not be comparable if the task mix changed. Report the confounders, the cohort sizes, and the taxonomy version. Use the result to target hypercare and instruction repair, not to assign blame for a transition that the team did not design.
The method cannot isolate the system change from concurrent client, volume, or staffing changes. Rework records may be incomplete, and reviewers may code the same issue differently. A short post-change window may miss a late increase. Report the confounders, cohort sizes, and taxonomy version, and use the result to target hypercare.