Designing research workflows people could inspect and reconcile
Researchers and librarians were not simply resisting a new interface. They needed confidence that their references, folders, and established workflows would remain intact when they moved from Legacy RefWorks to New RefWorks.
Duplicate review exposed the information researchers needed to compare possible matches before deciding which records to keep.
From expecting users to trust the new platform
to helping them verify what changed
People were more willing to move critical research work when they could inspect the result for themselves.
I led UX research and design across workflows that affected confidence in New RefWorks, including migration, deduplication, and the separation of work across projects.
Legacy RefWorks was nearing retirement, but critical workflows remained unsupported
RefWorks was moving through a multi-platform transition. Classic RefWorks had already been retired, and Legacy RefWorks was expected to follow as New RefWorks became the primary platform.
Many users had already moved to New RefWorks. The users who remained on Legacy RefWorks often relied on established workflows for systematic reviews, shared research projects, and large reference collections. For them, migration was not simply a matter of learning a new interface.
They needed to know that the references, folders, duplicate-handling practices, and project structures they depended on would continue to work as expected after the move.
My role was to understand what was preventing those users from migrating, identify the gaps between Legacy and New RefWorks, and help shape workflows that gave them greater confidence in the new platform.
Reference counts formed part of the audit trail researchers were required to maintain
In interviews and workflow observations with more than 10 librarians and research administrators supporting systematic reviews, users repeatedly recorded reference counts as work moved through each stage of the review.
They documented how many records entered the process, how many remained after duplicate removal, how many moved through screening, and how many were ultimately included or excluded. Those counts became part of the evidence used to explain how the review had been conducted.
RefWorks needed to remain consistent with the record researchers maintained outside the product
Users compared the counts shown in RefWorks with the numbers they had recorded throughout the review. As long as those values aligned, they could continue accounting for how references had moved through the process.
Problems emerged when the product count no longer matched the researchers’ record. A difference could indicate that references had moved, merged, duplicated, disappeared, or been counted differently than expected. Without enough information to explain the discrepancy, users could not confidently reconcile the next stage of the review.
Systematic reviews required teams to account for how the evidence set changed
The reference-counting behavior was not an informal workaround. It was part of how systematic-review teams documented the movement of records through search, duplicate removal, screening, inclusion, exclusion, and reporting.
At each stage, researchers needed to account for how many references entered the workflow, how many were removed, and how many continued forward. The counts maintained outside RefWorks provided a record of how the final evidence set had been formed.
RefWorks was one part of that larger documentation process. The values shown in the product needed to remain consistent with the counts users had recorded elsewhere.
When the numbers aligned, researchers could continue the review. When they did not, users had to determine whether records had been moved, merged, duplicated, removed, or counted differently than expected.
What first appeared to be resistance to migration was therefore not primarily about learning a new interface. The deeper issue was whether users could reconcile the product’s state with the record they were required to maintain.
This was not only a migration problem. It was a reconciliation problem.
The remaining users did not simply need a smoother path into New RefWorks. They needed to confirm that what the product showed still matched the record they were required to maintain throughout the systematic-review process.
That changed the design direction. The product needed to make record movement, count changes, duplicate decisions, and project context visible enough for users to identify discrepancies and reconcile the result before continuing their work.
Three high-risk workflows required different forms of visibility
The reframe created a consistent design requirement: when the product changed the state of a research record, users needed enough information to understand the result and compare it with the record they maintained outside RefWorks.
The specific evidence differed by workflow. Migration required count reconciliation, deduplication required record comparison, and project organization required clear boundaries between separate bodies of work.
Migration reconciliation
Show source and destination counts clearly enough for users to compare what left Legacy RefWorks with what arrived in New RefWorks and investigate discrepancies.
Duplicate review
Expose matching settings and record-level differences so researchers could evaluate possible duplicates before deciding which records to keep.
Project separation
Keep the active project, its references, and its permissions visible so users could distinguish one research effort from another while working from a single account.
These requirements did not produce one universal verification feature. They shaped a set of safeguards that made each workflow easier to inspect, understand, and reconcile.
Each workflow exposed the information users needed to understand what changed
The shared requirement was visibility, but the product response could not be identical across every workflow. Deduplication needed to support record comparison, projects needed to preserve clear boundaries, and migration needed to help users compare the old and new system states.
Let researchers inspect possible duplicates before deciding what to keep
Duplicate removal changed the number of references in the review. Researchers therefore needed to understand how possible matches had been identified and compare the underlying records before deciding whether they represented the same source.
Constraint: Duplicate counts could vary depending on which fields were compared and how closely those fields needed to match. A single automated result did not give researchers enough information to determine whether the outcome was appropriate for their review.
Product Response: Allow users to define the matching criteria, review possible duplicate groups, compare record metadata, and choose which reference to retain.
Tradeoff: The workflow required more review than automatic duplicate removal, but it gave researchers greater control over a decision that directly changed the evidence set.
1
2
3
4
5
6
Give each project a clearly defined workspace within one account
Folders organized references, but they did not create the separation researchers needed between distinct projects. Users working across multiple reviews needed to know which references, folders, collaborators, and actions belonged to the project they were currently viewing.
Constraint: Some researchers maintained separate accounts to keep bodies of work apart. A single undifferentiated workspace made it harder to tell which references and collaborators belonged to which project.
Product Response: Introduce project workspaces within one account and keep the active project visible in the global navigation. References, folders, sharing, and project-level actions remained associated with the selected workspace.
Tradeoff: Projects added another level of structure to RefWorks, but reduced the need to manage separate accounts and made the current work context clearer.
1
2
3
4
5
6
Help users compare Legacy RefWorks with what arrived in New RefWorks
A successful transfer message did not resolve every migration concern. Users still needed to compare reference and folder counts and determine whether the result matched the state they had recorded before migration.
Constraint: When reference or folder totals differed between Legacy and New RefWorks, users needed a way to determine whether the difference reflected expected system behavior or a migration problem.
Product Response: Make migration status, reference counts, folder structure, and known differences visible enough for users and support teams to compare the two systems and investigate discrepancies.
Tradeoff: Migration still required manual comparison and support in complex cases, but greater visibility made discrepancies easier to identify and discuss.
Document the Expected State
Capture the reference and folder counts users expected to carry forward from Legacy RefWorks.
Migrate the Collection
Transfer references and organizational structure into New RefWorks.
Reconcile the Result
Compare the migrated collection with the counts and structure documented before migration.
Resolve Discrepancies
Investigate unexpected differences before relying on the migrated collection.
Migration reconciliation process. Users compared the migrated collection with the counts and structure they had documented before leaving Legacy RefWorks.
Confidence grew when users could explain what changed
The most meaningful change was not that users stopped checking their work. Systematic-review teams still maintained records outside RefWorks and still compared those records with what the product showed.
The product became more useful when it gave them clearer information to understand count changes, inspect duplicate decisions, distinguish separate projects, and investigate migration discrepancies.
That visibility reduced the gap between what RefWorks displayed and what users were required to account for in their own research process. Over time, New RefWorks became capable of supporting the transition away from Legacy RefWorks, which was retired in June 2023 after a four-year voluntary migration period.
-
01
Migration discrepancies became easier to identify and discuss
Users and support teams had clearer reference counts, folder structures, and migration status to compare when the Legacy and New RefWorks states did not align.
-
02
Duplicate decisions remained visible to the researcher
Matching settings, possible duplicate groups, metadata differences, and primary-reference choices gave users more information before records were merged or removed.
-
03
Multiple research projects could be managed from one account
Dedicated project workspaces gave researchers clearer separation between references, collaborators, and actions associated with different bodies of work.
Legacy RefWorks was retired after a four-year voluntary migration
The retirement did not happen because users were asked to trust the replacement. It followed a longer transition in which New RefWorks continued to close important workflow gaps and support the work users still depended on.
I began by trying to reduce migration friction. I ended by protecting the information users needed to reconcile their work.
I initially understood the remaining migration challenge as an adoption problem. If New RefWorks were easier to learn and the transition required less effort, I assumed more users would be ready to leave Legacy RefWorks.
Research changed that assumption. Systematic-review teams were already responsible for documenting how their evidence set changed. Their hesitation increased when the values shown in RefWorks no longer aligned with the record they were required to maintain.
That changed how I evaluated simplification in expert systems. The question was no longer only, “What can we remove to make this workflow easier?” It became, “Which information must remain visible so users can understand what changed and reconcile the result?”