sm

CU Online Degrees with Coursera- RESEARCH STUDY

Summer 2023

Background and Objectives

During the summer of 2023, I led a research study on "CU Online Degrees with Coursera” under the supervision of my program manager while working for the UXRD team at the Office of Information Technology (OIT), CU Boulder. We undertook this research for the Office of Academic and Learning Innovation(ALI), CU Boulder. 

As CU Boulder worked to improve the student experience with finding, enrolling in, and completing online degree programs with Coursera, we conducted a research study to help the campus better understand the student experience with the current CU Boulder Coursera degree-granting programs. This research helped to confirm or dispel campus assumptions and inform future improvements for the CU online degrees, including the next enrollment form process.

My Role

  • Defined clear research goals -partnered closely with Academic Learning & Innovation to create the research goals of the study. 
  • Drafted a well-rounded interview script to capture a comprehensive understanding of the student experience. The script incorporated a mix of open-ended and rating scale questions to elicit qualitative and quantitative insights.
  • Developed participant criteria based on demographics and reached out to potential participants globally through recruitment emails, ensuring a diverse and comprehensive range of insights. Students were recruited from the Buff Portal Feedback Card and the Slate Post-Enrollment Surveys. 
  • Interviewed 10 participants virtually from 4 different online degree programs i.e.M.S. Data Science (MSDS), M.S. Electrical Engineering (MSEE), Master of Engineering in Engineering Management (ME-EM), and M.S. in Computer Science (MSDS). This direct engagement with the participants allowed me to capture qualitative insights that were vital to our research.
  • Analyzed post-enrollment surveys to distill patterns and insights, complementing the interview findings.
  • Summarized the findings and recommendations into a comprehensive research report.
  • Presented these insights to stakeholders, translating the research findings into actionable strategies to elevate the overall student experience for those pursuing “CU Online Degrees with Coursera.”

Methodology

We chose in-depth interviews as our research methodology. This approach allowed us to delve into the intricate nuances of student experiences, perspectives, and challenges encountered while pursuing master's degrees through CU online programs.

Research study Constraints

  • Time: In order for this research to be most helpful for CU, we needed to have a quick turnaround time.
  • Questions: The questions we asked in the research study may have driven requirements from the Office of Academic and Learning Innovation.
  • Participants: The user population we wanted to study was challenging to identify, recruit, and schedule testing sessions with because we wanted to work with an international population and include users who may not have been CU students yet.

Research Analysis

Importance of Program Aspects - Individual Ratings

Comprehension of Program Aspects

Importance of Program Aspects - Ranked

When asked to rank the program aspects by importance, the following shows the average place:

  1. Performance-Based Admissions (PBA)
  2. Non-Credit/Preview
  3. Pay as you go
  4. Session Calendar
  5. Certificate 
  6. Quick Enrollment Time 
  7. Community

Program Aspect

Performance-Based Admission (PBA)

What’s Working

  • PBA was Very Important (4.25/5) to students when rated individually and it was selected as the Most Important aspect of the program when ranked against others
  • The term “Performance-Based Admission” is clear and easily understood by students from various backgrounds and experiences
  • Students found PBA important because it didn’t require TOEFL or GRE:
    • No delay while studying for exams, taking exams, waiting for scores
    • No need to find old transcripts or reach out for recommendation letters
    • Less anxiety
  • One student appreciated the flexibility and freedom to choose from two pathway specializations (can better align education with their interests and passions)

"Because my background degree is right now from 30 years ago. So I mean, finding my old teachers to give me a recommendation letter [...] is very complicated for me. They are retired right now so finding them... it can be hard. That's one example of how having all of the requisites for admission could be difficult for me. So just to be performance-based is great."

What’s Not Working

  • One shared a concern about the long-term reputation of the program due to PBA, which might negatively affect the value of their degree in the future 
    • Student was from MEEM and expressed other reputation/value concerns about the program (see Other-Academics)

Program Aspect

Non-Credit / Preview

NON-CREDIT PREVIEW - Working

  • The ability to preview a course as non-credit was rated Very Important (4.25/5) and students ranked Previewing as the second most important program aspect
  • Upgrading from non-credit to for-credit was Very Easy (4.5/5)
  • The Terms ‘Non-credit’ and ‘For-credit’ are clear and easily understood by students
  • Students shared that “previewing” helps them:
    • Decide whether to take the course or not (5)
    • Have flexibility with their busy schedules, juggling family and work commitments (2)
    • Decide which of two courses to enroll in (1)
    • To continue coursework in spite of a Dean’s Hold stuck on account (1)

What’s Not Working

  • Some of the previews do not help to assess the level of difficulty of a course since assignments are not included
  • Students understood the reasoning but were concerned that Lab courses could not be previewed

Program Aspect

Flexible Session Start Dates

What’s Working

    • Students ranked the flexible start session dates as Very Important (4/5)
    • Students were given a scenario to enroll in a course that had already started a week ago and were asked if they could still enroll.
      • 9/9 students passed this scenario and knew they could still enroll in the course even when it started
    • Students were given another scenario where they were asked if they would consider enrolling in a course with a session ending in three weeks, provided they have ample free time and are prepared to start a new course

     

  • 9/9 students passed this scenario and knew that they could still enroll in the course with a session ending in three weeks
  • Students shared that flexible session dates were very important because:
    • With busy schedules they could miss the deadlines (2)
    • It allows them to move through the degree faster (1)
    • It allows them to enroll when they have the money to pay for a course (1)

"So my only limitation for not completing the program right now is the financial part because I cannot afford everything at once, I cannot get enrolled in everything. So for me flexibility is very important because it gives me the space to put together my financials and say, I wait 15 days and I get another paycheck and I can pay you for another course. So I get to juggle with that a bit and that helps me a lot."

If not, I would have to wait until another [session] and I don't know if my schedule will be full. So it's good to have that flexibility, I can see oh I have no activities for the weekend so I can get deep in my courses. And I have the money to spend on it. I really find that useful.

Program Aspect

Pay-as-you-go

What’s Working

  • Students like Pay as You Go tuition model as it offers a lot of flexibility to pay for their courses.
  • Students rated Pay as You Go tuition model as 3.25 (Moderately important) on average.
  • Students shared Pay as You Go is important because:
    • It helped them reduce their financial burden and it increased affordability, particularly benefiting international students.

What’s Not Working

  • 1/10 reported they were blocked from enrolling in a new class due to an unclaimed balance from a dropped course
  • 1/10 reported that Flywire does not work on VPNs
  • Additional fee charged by Flywire

Program Aspect

Certificates

What’s Working

  • 9 out of 10 students knew the university automatically issues certificates and they are not required to take any additional steps to receive them
  • Students like to receive certificates automatically along the way to their degree:
    • Some specifically reported that they might include certificates in their resume since certificates could be proof of their progress towards the degree for future employers (3)
  • Certificates were less important to other students since they have a clear end goal of the degree (3)
    • The student from Israel shared that certificates have very little meaning to employers in Israel, therefore it’s of no value to him (1)

What’s Not Working

  • 2 wished they had known earlier in the program about certificates
  • 1 MEEM student reported receiving certificate before completing the last required course

Program Aspect

Community

What’s Working

  • SLACK connects students, providing a collaborative digital platform that facilitates communication and information sharing
  • Office hours provide an opportunity for open communication and discussion between students and TA
  • Students shared that it is slightly important to feel connected with the CU Boulder community because:
    • One did not need classes to connect as they are a “pandemic learner”
    • One student acknowledged that’s to be expected since it is an online program

What’s Not Working

  • No connection to the instructor
  • No virtual events/meetups organized by the University
  • No information about classmates in cohort (students are unaware of who is seeking a degree)

Buff Portal

What’s Working

  • 10/10 reported using Buff Portal for final grades, GPA, holds, transcripts, dropping classes, seeing financial information, and accessing the library
  • Overall, students reported a positive experience in Buff Portal
  • 5 students reported noticing the August change to their portal but it was subtle and no one reported problems or confusion

What’s Not Working

  • 3 students reported seeing information that didn’t apply:
    • Enrollment verification (for international students)
    • Tax information
    • Discrimination & Harassment training for on campus students*
  • 1 student reported a CEAS Dean’s Ethics Hold that is still on their account had recently prevented enrollment in the next session

Recommendations

Performance-Based Admissions

  • Keep Performance-Based Admissions because:
    • It is a key differentiator for the CU Boulder degree
    • It is incredibly important to the learners enrolling in the program

 

  • Continue using the term “Performance-Based Admissions”

 

  • Continue to track the rigor (and perceived rigor) of the program and therefore the reputation of it, due to PBA

Preview Courses as Non-credit:

  • Continuing to offer Previewing
  • Continue using the terms Non-Credit, For-Credit and Preview.
  • Exploring if Previewing ability can be added to Lab courses.
  • Improving the consistency of the descriptions of the programs to include more assignment information.

Certificates and Stackability

  • Continue offering certificates and continue to send them out automatically
  • Improve student awareness about the awarding of certificates in promotional materials (before enrollment).
  • Possibly explore if MEEM students are receiving certificates prior to completing their last course.

Pay as You Go

  • Explore the reported issue of students with balances (from a dropped class) not being able to register – could students register with a balance? Or could they carry a balance in their accounts if they have dropped a course (student suggestion)
  • Explore the reported issue with Flywire not working behind VPN for students in China
  • Explore other payment options for international students with fewer or no additional fees

Flexible Session Start Dates/Calendar

  • Continue allowing students to enroll after the session has begun.

Community

  • The University should sponsor more online events like career fairs, other meetups with classmates.
  • Offer one live online session with the instructor each session. Could be an office hour or a more formal instructional session. Record it for people who can’t attend live. Students could submit questions ahead of time. 
  • TAs/CAs should consistently attend office hours and not end office hours early if no one has showed up since students might still plan to attend. 
  • TAs/CAs should have a strong mastery of the course material.
  • Allow students in the degree program to see a list of other students in the degree program and possibly connecting to their LinkedIn profiles to improve networking and connection.