City of Dallas User and Site Research

User and site research is an important part of running and maintaining a website and how it will affect users, especially for a government website such as this one. For this piece, my group and I researched the City of Dallas website and its users.

Project Specifics

Semester

Spring 2022 8W1

TECM 5750

Tools

Role

UX Researcher

Background

Why the City of Dallas Website?

The City of Dallas (COD) website is a federal site and a hub for Dallas residents, visitors, and other affiliates of the city to find important information and complete important tasks. It serves the purpose of effectively equipping users with accessible, relevant, and useful information.

Despite government programs in place to increase the usability and accessibility of government websites, there has not been as much change over the past couple of decades. Our group chose the COD website to complete two reports over due to its status as a government website and give insight into what can be done to improve it for its users.

Below are overviews of what our team did research on, from its user base to the website itself. Not every part of our research was included here but can be found in the Appendices of the reports themselves, which can be found here.

User Research

User Research Goals

From the beginning, we knew that we needed to get the right kind of data from users to formulate what their wants and needs were, so we created a list of research goals:

discover the types of users who use this site through research
develop at least 2 personas that represent these user groups
understand user behavior, attributes, goals, and challenges through research
present at least 4 complete user findings or pain points to address

In order to reach these goals, our methodology included a user survey and 4 user interviews.

Survey

Materials & Procedures

Using Qualtrics, we developed, designed, and distributed our survey of 23 questions which consisted of open-ended, closed-ended, and Likert scale questions. To distribute the survey, we made posts on Reddit, LinkedIn, Facebook, and shared it with people who are known to use the
website. The survey was active for a total of two weeks.

Interviews

Participant Profiles

Participants interviewed are COD users or potential users. We gathered a total of 4 interviews.

Materials & Procedures

We conducted 4 user interviews that consisted of approximately 23 questions, give or take some follow-up questions. To conduct these interviews, we either met with the participants in person or used video-conferencing software such as Zoom or Microsoft Teams. Additionally, we took notes and/or recorded the interviews using voice-recording software.

User Research Findings

Affinity Map

In a remote collaborative session using Miro, our team recorded ideas, observations, and quotes from survey and interview results using virtual sticky notes. We categorized sticky notes by themes to draw out notable insights and analyzed records for patterns as they relate to potential user archetypes.
We developed the following themes to categorize our data by:

user attributes & characteristics
user behavior
user opinion
navigation & finding information
trust & reliability
user interface

Affinity Diagram for User Research

Using our affinity diagram, we were able to come to three conclusions about the users and the expectations they had for COD website.

Most users visit the site rarely

Most users use the site only rarely, reassuring the fact that this site needs to be more efficient and usable. A total of 43% of users say they visit the site rarely (this data is only from the survey).

Users’ expectations of a government site are mostly high

Users’ expectations of a government site like the City of Dallas are mostly high because it is federally run. The information from a government website must be accurate since the intended audiences are mostly required to use it. From our survey, 43% of users say their expectations are high.

Users rated trust as the second most important factor in a government site

Users align trust with government, and believe trust is second most important in a federal site behind accurate and relevant content. Since users need to use personal information on some of the site’s features, they expect that information to be protected. Our survey showed that 64% of users voted trust as the second most important factor in a government site.

Personas

To inform subsequent site testing and provide user targeted insights, our team developed two personas, a primary persona, and a secondary persona, to represent COD’s audiences. These personas focus on users’ goals, behaviors, and pain points. These personas will be referenced throughout the remainder of our project as they strictly define and narrow the COD user scope. In order to create some details such as age and employment status for these personas, we used demographic information from our surveys.

Primary Persona, Sandra Brown

Our primary persona, Sandra Brown, is a new homeowner in Dallas and is motivated to learn more about her community and use her community’s resources. She knows the city website must fulfil any of her information needs regarding her residency. Her career and family keep
her busy, so she uses the city site with the expectation that information will be quick to find, access, use, and read.

Persona 1 - Sandra Brown

Secondary Persona, Trevor Stanley

Our secondary persona, Trevor Stanley, is a freshman at the University of North Texas and a regular visitor to Dallas. Trevor keeps busy with school, socializing, and volunteering. He wants to gain experience and connections that can boost his future career and make sure his time in North Texas is memorable.

Persona 2 - Trevor Stanley

User Journey Map

To visualize our target audiences’ interactions with the site, we developed a user journey map based on our primary persona, Sandra Brown. The map depicts a sequential view of her interactions from the “Identifies Information Need” stage to “Takes Action,” along with the complexities that would shape her user experience (mindset and feelings). The map follows the digital journey Sandra takes to file a noise complaint on her neighbors.

User Journey for Sandra Filing a Noise Complaint

Low points on user journey were IA related

One of Sandra’s low points on her journey was when she was looking for information about services by clicking on “Resident” then “Services.” Her overall journey depicts the process of her finding out how to submit a noise complaint via the city. While filing a noise complaint is a service provided by the City of Dallas, it is not something the user can get to through the global navigation menu, even under “Services.”

This is a frustration for Sandra because she came to the website with a problem, is not expecting to spend time on this issue, and ends up having to put effort into looking for this service in a different place.

Empathy Map

The map articulates what we know about our target audience by capturing knowledge about their feelings, actions, and thoughts in relation to COD. The map captures the thoughts, feelings, quotes, and actions made by Sandra while looking for family services.

Empathy Map of Sandra Looking for Family Services

Persona’s actions, feelings, and mindset affect information needs and search behavior

While browsing through the “Services” tab under “Residents,” Sandra is looking for activities, events, or resources for her children to partake in. All her thoughts, feelings, actions, and sayings are all regarding how she might take her children to an event or what she needs for them.

Some of her thoughts include finding information on resources such as bus passes, bus schedules, price of events, and library card registration for her family. As Sandra is our main persona and user type, we need to consider her wants and needs when applying this information to the website itself.

User Research Conclusion

After conducting user interviews and surveys, we compiled our data in an affinity diagram to better understand the users, their behavior, attributes, goals, and challenges. With our findings, we then created two personas, a journey map, and an empathy map.

From our affinity diagram, we found that the majority of users rarely use the City of Dallas website, but when they do, they use it to find information, pay bills, and browse important news and events. We also found that most users have high expectations for a government website like this one for trust, privacy, and security purposes.

The journey map allows us to better understand user needs and major pain points in finding information that they need. We found that one major pain point was related to the IA of the website because the user does not want to waste time looking for information that is not readily available.

Our empathy map explained a few user behaviors and needs regarding specific services and resources of the website for the website’s main user type, residential homeowners. As residential homeowners go on this website more often than other users, it’s important to consider their thoughts, feelings, and actions when they look for information.

Site Research

Site Resarch

Site Research Goals

At this point, we had a better understanding of our users. We began work on the research for the COD site itself, so we created a list of research goals:

understand various user journeys & learn about things that work well, and things that can be improved from user feedback
develop at least 2 complete, actionable, and research-driven takeaways from TAP
develop at least 2 complete, actionable, and research-driven takeaways regarding elements of expert reviews
develop at least 2 complete, actionable, and research-driven takeaways from comparative analysis
measure and report final site findings using a scale of severity ratings and usability metrics

In order to reach these goals, our methodology included usability testing, heuristic expert evaluation, and a comparative analysis.

Usability Testing

We performed a type of usability test called a Think Aloud Protocol, or “TAP.” This is a usability study in which we observed users as they attempted to use the website while thinking out loud using a set of user tasks. This helped us to identify the severity of the site – specifically its information architecture – and allow us to see inside the user’s cognitive process.

Participant Profiles

Participants were chosen on a location basis, specifically users in Texas. We recorded a total of 9 participants’ data.

Materials & Procedures

We decided to run a remote unmoderated study, in which moderators are not present with the participant. Using UserTesting.com, we developed and administered our tests to a total of 9 participants. The tests included a brief script, 6 user-tasks that were targeted at the needs and goals of our personas, and a post-task questionnaire that gauged the difficulty of each question and the users’ overall experience.

User Testing Tasks

Heuristic Evaluation

We conducted a heuristic expert evaluation of the site using a heuristics guideline from Sabre, a travel technology company. We chose this set of heuristics due to its design-specific accessibility and usability standards with severity ratings.

Additionally, we also considered the design principles from the U.S. Web Design Systems’ website. These principles correspond to the guidance of the 21st Century Integrated Digital Experience Act which increases the accessibility and usability of government websites.

Comparative Analysis

To gain a better understanding of what is currently published regarding usability in federal websites, a comparative analysis was performed. This analysis targeted 2 “competitors” – the City of Austin and the City of Houston websites – that are closely related, widely used, and well-established. The primary purpose of this analysis was to identify comparators’ solutions that excel, are lacking, or are missing critical design elements.

We established a set of 3 main heuristics to evaluate against:

information architecture
usability
content

Each of these categories was broken down even further into specific features to take a closer look into.

Spreadsheet of our Comparative Analysis

Site Research Findings and Recommendations

Images and Visuals are Enjoyable

Multiple user testing participants mentioned that they enjoyed the main image on the landing page. They found the colors and quality of the image to be appealing. This type of attention grabber is ideal for the welcome section of a city website.

This aspect of the website also adheres to the heuristic of using simple background images. This image is not distracting, is behind text, and blends in with the established color palette of the website.

Homepage of the City of Dallas Website

Content and Information are Abundant

The website has sufficient information on every page, especially on the landing page and within its navigation links, tabs, and menus. Global and secondary navigation menus drop down into subcategories to search by, and there are plenty of ways to navigate the site and find information just by being on the landing page. Many users who participated in testing mentioned how thorough the information on the website was.

Organization Systems do not Support Optimal Findability

An organization system in information architecture is how content is categorized and structured. We found that the global navigation menu, the secondary global navigation, and column navigation were unintuitively and very inconsistently grouped. Content within these areas featured a mix between audience, topic, and task-based content.

We found in our comparator analysis that compared to the City of Austin’s organization systems, the City of Austin featured a much more targeted, organized, and thought-out organization scheme. Topics were organized consistency by either audience, topic, or task, and the organization of the landing page flowed by popularity.

It was also a notable instance in our user interview. One participant points out the organization system problem on the websites landing page, specifically regarding how and why content was group in three columns for navigation purposes.

COD 3 Column Navigation Menu

Recommendation

Content to be organized/grouped in a more clear and consistent manner to support content usability and findability.

Labelling system is Inconsistent and Unintuitive

A labelling system in information architecture is how content is represented or labelled. From our comparator analysis, we noticed right away that other websites provide more context around what “311” is. On the City of Dallas website, “311” is a main tab in the global navigation, and even when clicked on, fails to provide context to the user about what it is and what it can do.

311 in the COD Top Navigation Bar

311 section of the city of austin website

311 Section of the City of Austin Home Page

Recommendation

Rename labels to be simplified for user comprehension, consistent for ease of use, and fitting toward existing user mental models.

Navigation Systems are Underdeveloped

Many users got lost while trying to complete tasks simply because they couldn’t understand how they got to a certain location. It was hard for them to retrace their steps if a certain task took multiple clicks to complete. And since the city website directs the users to external websites many times, users would have trouble finding their way back to the home page when it was time to start a new task.

Our comparator analysis revealed the lack of a sitemap in the footer of the City of Dallas website. Most websites, especially federally run, should include a sitemap so users who decide to use it can understand the websites’ structure, making it easier to navigate.

Recommendation

Include navigation breadcrumbs to help guide users through information and implementation of a sitemap.

Redundant and Unncessary Features

There’s a level of complexity added by the unnecessary navigation bar in the top-left and the chatbot in the bottom-right of the website. This secondary main navigation bar is redundant because it includes information that is already provide in the main navigation bar and other location on the home page. There is no need to offer multiple ways to achieve the same outcome. This extra navigation bar is inconsistent and only adds more routes for the user.

The chatbot rephrases the information that can already be easily found on the website itself. When users were asked about this feature, some reported that they immediately ignore chatbots unless it’s a last resort.

Recommendation

Remove or repurpose secondary navigation bar and chatbot feature. Since these elements were mostly ignored it would help reduce the clutter and overall design of the UI.

Lack of Appropriate Domain Name

The website does not have a .gov name which users use to identify qualified government organizations and programs. Other city websites in Texas such as houstontx.gov and austintexas.gov, correctly have this domain. Having the proper domain will increase the trust, searchability, security, integrity, and standard for the website’s content.

Recommendation

Implement the “.gov” domain to assure users they are accessing an official U.S. government site.

Site Research Conclusion

After conducting a heuristic evaluation, competitor analysis, and think-aloud protocols, we found several usability issues and good practices. The purpose of the City of Dallas website is to provide different types of users with a vast amount of information regarding resident services, employment, visitor interests, local government, operating a business, etc.

Participants in our usability tests mostly had difficulty with Tasks 3 and 5 which dealt with using the calendar feature and finding information on building permits. Part of the confusion came from some participants overthinking the tasks, however the issues with the navigation and labelling of the website also contributed. Reorganizing and renaming some of the content in the global navigation bar will assist users complete their goals faster. The current structure doesn’t align with the user’s expectations and causes confusion and frustration.

Our comparator analysis helped our team to recognize how this websites usability could be enhanced based off what is already out there. From both the City of Houston and City of Austin sites, we learned about how the different components of information architecture are presented on similar sites, how popular content is displayed, and how city website specific elements are used.

From the heuristic evaluation we’ve determined that the website follows the majority of the essential rules needed for websites. Only minor issues such as unvisited links not changing colors were identified.

Original User and Site Reports

Main Artifacts

Here I've included PDF files of each report for better viewing.

User Research Report

Site Research Report

Challenges and Thoughts

Challenges

One major challenge that we faced was getting enough survey responses for the user research report. The minimum amount we had to reach was 50 proper responses. As mentioned in the report, we were only able to get 28. Instead of throwing out that information, as we reviewed the results and still found rich data, we had to implement another research method which was planning interviews before our deadline for that portion of the project.

We added extra hours of filtering through interview data and notes in time for our deadline, but we were able to overcome that obstacle. Since we evenly split the interviews as best we could, I can confidently say that I feel prepared for the next time that something in a project doesn't go according to plan.

Thoughts

Overall, I'm proud of the research we were able to do for this website. I've lived in DFW my entire life and always found city websites confusing or complicated to navigate, so being able to work on research to improve this website gave me a better understanding on how city websites work in order to benefit the workers/staff and the citizens of the city.

The website itself does better than many other kinds of government websites and typically follows government website guidelines, but the minor issues the website had distracted it from the important website it is.