Improving Student Incident Data Quality
Problem
How do we iteratively improve the behavior product, while also delivering rapid improvements to increase user adoption and alleviate buyer concerns?
Approach
My designer developed several features with complex permissions and workflows that filled gaps in the existing product and appealed to our largest school districts.
Impact
Acquired first large school district of the year, with missing functionality no longer a detractor. In talks for state-wide contract to accelerate roadmap.
Design Approach
The Panorama for Positive Behavior product (internally called PosBee) was released as a lean MVP.
PosBee enables districts to document and analyze student behavior incidents.
I managed two designers and served as the design lead responsible for resourcing and having a voice in product strategy.
During my tenure, we invested in behavior logging with two key functional improvements:
Editing submitted incidents for major or minor incidents
Ability to configure major and minor incident severity to improve incident resolution and analysis.
Client
Panorama Education (Edtech)
Duration
4 months (Nov 2022-Feb 2023)
My Role
Design Leader
Collaborators
1 Product Designer, 2 Product Managers, 5 Engineers
Editing & Updating Submitted Incidents
Behavior incident follow-up & investigations often reveal new info that can’t easily be updated in the existing product. To unlock larger contracts and improve logging value, we added editing functionality for permissioned users.
Design Approach
Our Q1 release added key elements to existing incident management: from new permissions to new pages and workflows.
My designer needed to also balance editing accessibility and ease with technical limitations of editing components of varying complexity across our product and other products int he platform.
In addition, design worked with engineering to define the nightly exports to school information systems (SIS) for state/local reporting needs.
New permissions assigned by role to edit submitted major and minor incidents
Added Edit buttons to key list pages for easy edit access
This video provides a walkthrough of our minor incident editing functionality (voiceover included).
New edit component for tracking edit reasons and notes, while in incident edit mode
New Edit log timeline page to present a feed of when, who and why edits were made
Added navigation to improve the information architecture of individual incidents
Challenges
When was the right time to ask for additional details, so that it did not disrupt editing?
Edit logs track who and why incident records change and incidents may be updated by multiple people. But, we didn’t want to interrupt educators when they are focused on the updates themselves.
How did we mitigate?
Removed edit notes from minor incidents,
Only required the “Reason” field on both incident types, and provided a short list of options; and
Presented the edit log notes at the start of the incident form, instead of using a confirmation modal after saving.
How do we integrate and juggle system-wide changes while releasing iteratively?
During the design and development of these features, another team released a navigation redesign and component updates. This meant that the minors release would have a distinct feel from the majors release.
How did we mitigate?
Planned our minor logging release after a new navigation component launch. This would save the team from recreating an incoming component;
Design spec referenced system components, encouraging engineers not to match exact Figma files;
Connected the two engineering teams once design discovered the conflict to coordinate releases and code hygiene; and
Regular design 1:1s across teams to coordinate and review in-progress builds and designs.
How do we manage permissions and publication of this feature?
Administrators could give users permission to edit submitted major and minor incidents. At this point in the school year, we were unsure whether schools would integrate editing into their in-school processes for the current school year
How did we mitigate?
Released with permissions off, so administrators would have to opt-in to the feature,
After user complaints about the missing feature, we decided to turn on the feature for all clients for only specific roles.
Provided in-product tutorials with Pendo for district admins, and
Released code iteratively, but marketed after having a meaningful feature set.
Configuring Major & Minor Incident Severity
When logging was first released, it was assumed that teachers and administrators had shared definitions of incident severity.
In Q1, we learned through user research and usability testing of existing workflows that there were more severity designations than just “major” or “minor”.
As we increasingly leaned towards a more configurable logging form, an important first step was addressing severity.
Added severity configuration options for adding levels to majors and minors.
Future Releases
With our first launches delivered as MVPs, we planned additional functionality for a minimum lovable product (MLP) by August 2023.
Added confirmation modal for switching between major and minor distinction
Design Approach
In Q2 2023, the team defined a vision for multiple severity levels, evaluated it with users and defined the functional releases.
Added administrator-only field and section for assigning severity to individual incidents.
Modified data visualizations to capture severity levels
Challenges
Is it a “minor incident” or “teacher-managed”? How does language encourage behavior?
Most incident types are determined by a behavior matrix used within a school system. Every educator is trained in this matrix, which is unique in school district. The team wondered: should copy be configurable? Do users agree on what these phrases mean?
How did we mitigate?
Provided helper text to provide generic guidance applicable to the majority of schools — instead of adding complexity of custom copy.
Tested design concepts with administrators and teachers to assess difference in comprehension.
How can we build towards a more configurable form, without a massive change/upgrade?
The team has discussed multiple approaches for the logging form based on new user insights. The existing form depended most on the severity level, but this would not scale in the future logging form.
How did we mitigate?
Engineering lead facilitated discussions and architecture reviews to compare scope for different approaches;
Design developed visionary concept sketches for rallying team around the future — and worked backwards from that;
Product, Design and Engineering worked together to identify key epics needed to achieve the vision by the next buying cycle, and
Scope was constrained to improve experience while setting technical foundation for the future.
Who decides severity?
User research had revealed a difference between teacher and admin perception of behavior severity. Context clues and school policies provide direction, but misinterpretations still happen.
How did we mitigate?
Designed severity assignment as a single field for easy data entry,
Associated severity with a single user role, so those most knowledgable would be responsible,
Designed email communications (future releases) that automated communication; and
Created concepts for modifying severity for future releases.
Impact of All Feature Additions
As a result of this work, missing functionality was no longer a top reason for losing contracts.
We also secured our first greater than 5K customer of the year, a key demographic we'd historically had challenges acquiring.
Additionally, we earned stakeholder buy-in and shipped the first features for a large-scale architectural overhaul.
At the time of my departure, we were also in talks with a statewide school district thanks to the configuration abilities we had now included.