Archive

Posts Tagged ‘usability’

Lessons learned from an Intranet Redesign

January 9, 2012 Leave a comment

Great websites are built to serve users’ needs quickly and easily. If you build an online community but understand the tasks your users need to complete, and make those tasks easy to complete – then your website will #fail.

Intranets are similar. In March 2007 I was asked to redesign the intranet for GSA’s National Capital Region. I worked on a team with Ann Everett, NCR’s Regional Administrator, and Terry Forline.

The first task was to establish our goals. We decided to improve the layout, brighten the color palette, and reposition links to improve the functionality of the intranet for our users. We also wanted our intranet – which we call NCR InSite – to save users significant time and money. Dell estimates that its intranet creates annual productivity gains of $36 million. Government intranets are different and it’s hard to see what is done behind their firewalls – but we found Australia has some excellent oldie but goody intranet best practices online.

Our second task was to learn what tasks GSA staff wanted to accomplish on NCR InSite, and what they hated the most. We sent out an email to over 1,200 employees to ask for their help to make NCR InSite better. 1 We got back over 135 responses – a response rate better than 10% which is stellar.

Within a month our team had made most of the changes pinpointed in the survey. We made significant headway towards implementing most of the changes people had asked for. And the best outcome was that our employees were pleased with the results and proud of their InSite, bragging that it was one of the best within GSA. We couldn’t make all of the changes that everyone asked for, but we did at least ask people what they wanted. Listening to users can be very powerful.

==========================================================

Email to NCR InSite users sent March 2007

Help us improve NCR InSite

We are committed to improving NCR InSite to ensure that it meets your needs – provide basic information and tools that you need to do your job effectively, information about jobs, pay, benefits, and services; and help employees learn more about GSA’s businesses, policies, and programs.

Recently, work has begun behind the scenes to expand and improve NCR’s Insite. For example, events have been added to the NCR calendar. Users can more easily NCR’s shuttle schedule, cafeteria menu, and remote email address, as the icons on the horizontal navigation bar are now identified with text reflecting their purpose. The cafeteria menu is up-to-date.

However, we need your help to ensure that the revisions reflect your needs. We would like your opinion and feedback on your visit to our website so that we might redesign it in a manner that is more user friendly and is more efficient for you and for others. We hope you take the time to help us by completing a brief survey. All information collected from the survey will be kept confidential and only will be used in support of improving NCR’s InSite.

Based on the feedback we receive, we will improve NCR’s InSite to make it easier to use. We expect to be implementing these changes over the next few months and will inform you in advance of the changes so as to minimize any disruptions or inconvience.

We are looking forward to the opportunity to hearing from you, and helping you have a more productive experience using NCR InSite.

Sincerely,

The NCR InSite Team

=================================================================

NCR InSite Survey Results

Survey conducted from 3/21 – 3/30/07

1. What tools and features do you currently use most on InSite?

Not at All

Sometimes

Very Often

Top News

55

108

30

Calendar of Events

69

93

27

NCR EmployeeLocator

6

48

135

Regional Links to weather, traffic

38

33

7

Self-Assessment Guides (APPAS)

75

96

16

Human Resources Information

28

112

60

NCR BrandWizard

102

60

12

IT EquipmentRequest/ComputerMove Form

84

86

21

IT Security TipsAnd Info

84

80

19

Other: Shuttle Times
Thanks program, archived news and Ask Ann
Shuttle Schedule, access GSA InSite and home page
Shuttle Schedule
Lotus email link to check email through vpn from home
I use the web access Citrix to access my VPN Account while telecommuting
Cafeteria Menu Link
Reading the Ask Ann items
Chris, Thanks Program
Employee HR & Web Applications
PBS, Financial Resources Links
E2 Travel, Thanks!
Building Locator, Webforms
Web applications
Building matrix, division website, project website
Project information portal, citirix
PBS Resources, Asset Building Plans, Rent Roll, Pricing Desk Guide
GSA Directory Locator
Launch Web based applications such as FMIS
Acquisition website for model specs and procurement bulletins
Acquisition page

 

 

2. What is the primary method you are using to find information on InSite?
List of Topics (on the homepage and in the left navigation column of sub-pages)

88

Top Navigation Bar

32

Bookmarks/Favorites

26

Search Engine

28

Other: PBS Resources
PBS Resources
None – these are not helpful – especially the search engine
Alphabetical listing
The search engine doesn’t give relevant hits
none
none

 

 

3. What is the most important content you would like to see developed for future inclusion on the NCR InSite?

Search

1. Better Search Engine that is effective for valuable info retrieval.

2. A reasonable and simple search system – maybe modeled after google.

3. Not content, but a good search engine for the site

4. Search features and the documents they produce need to be more relavent

5. A search engine that works better would help.

6. A better search engine

7. Better search results

8. Better Search String

9. Search feature needs improvement, old pages need to be updated or removed

10. Search effectiveness is poor.

11. Word search database

12. There needs to be a better search function for locating topics within InSite

13. Better search function for finding regulations and related memo(s) and interpretations of GSA/NCR relevant policies particularly for the financial/budget and real estate communities.

14. I would like my searches to require no more than 3 clicks. I also think that it will help the user to know what information is included in the words shown in the screen. The best websites allow you to run the cursor over a “title” and see in a summary of what it refers to. A final suggestion is to make the GSA Form database user friendly.

Organization charts

1. NCR Directory with photographs and Office

2. Org chart related to personnel locator with a link to the PBS Portal

3. A more detailed organization chart

4. Organization guide to show all employees by organizational unit

5. Provide organizational charts for the NCR that provide links to information on services and contacts in respected organizations. example, in the PBS area, Potomac Service center. What customers and buildings are they servicing-a map of their area using GUI may be useful, what are the key projects with points of contact, being worked on or are in the planning phase. Overall find our Web site to be useful and easy to access. Also, linkage across all our business lines may make it easier to locate key contacts on projects. Hope this information is useful.

6. There needs to be a quick way to find Organization Charts along with names of key players. A photo of each party would be even better. I saw this on another regions InSite and found the Org. Charts invaluable.

Events and News

1. Homeland Security and COOP advisories should be featured in a Breaking News format – maybe consider text messaging.

2. More NCR internal news.

3. Put all event announcements in right side following other news announcements.

4. Regional Happenings

5. Cafeteria and Lunch time specials.

6. Employee news.

7. Upcoming events, with background information.

8. Need to make the calendar of events more visible and usable.

9. Future developments – please feature this.

10. Different manner of presenting “what’s going on in NCR.” Current look is old and boring

11. Stop lights on how we as a region are doing with performance goals.

Human Resources

1. This may have been corrected, but the HR tab within NCR Insite are not user friendly and information should be updated

2. HR Guidance for Supervisors

3. This is not very important. But I would like to have the calendar that shows the shows the holidays and pay dates for the year.

4. GSA Jobs website – Ready & available for GSA associates

5. I would like to see more job information on the quick hire process.

6. HR info should be current and complete. Associates should be compelled to fill in complete details and update them promptly when there is a revision. Announce relocations of persons like contracting officers and project managers.

7. Would be nice to see Performance Review / dates of APPAS actions somewhere – maybe on calendar – so employees and managers could plan ahead of time for this process. A simple how to guide, with a phone number of someone in HR concerning APPAS issues, would be wonderful.

8. Create a section for the new employees so we know who they are when we see them and they feel welcomed. Also create a section for retiring employees so we are aware of this and can make plans ahead of time.

 

 

Conference Rooms/Hotels/Restaurants/Meeting Resources

1. Prop Dev. Conference Rooms and other support

2. Listing of conference rooms.

3. A list of nearby restaurants, cafes, eateries, etc. would help out employees, contractors and visitors.

4. List of hotels, conference rooms, and meeting facilities for planning future meetings.

Employee Locator

1. Phone book could be more user friendly. You have to have the correct spelling or it will not find items.

2. I recommend that a cleanup be made of the Employee Locator database. There are multiple problems with hyphenated names, middle names, initials added to the first name, as well as nicknames.

Credit Union

1. GSA Credit Union Information

Navigation/Links (specific)

1. Phone numbers for the building manager’s office and the IT help desk should be MUCH easier to find (should be one click away from the home page). Link to Current Clips would be helpful. Occasional graphic, pictorial and-would liven things up a bit. It’s a helpful, well-run site, though. Thanks!

2. Links to other GSA Region’s intranet sites

3. Need better layout, make easier to find things (example – PBS customer list is hidden in PBS organization)

4. I like the fact that you can now see the icons. Please add the Motor Pool to the icons.

5. Make the overseas travel in E2 work.

6. Link to Central Office pages with contacts or organizational structure

7. Direct links to other regional offices

8. When titling resources, please ask the users to assist in the actual naming. Too often, names are dreamed up that no one knows what it might refer to. Experienced persons such as our contractor, Mark, can give some insights in this regard.

9. The links to policy documents needs to be clearer. There should be links to GSA Insite for National Policy on the same topics, while you looking at the regional information.

10. Please underline links! It is so annoying that you have to mouse over stuff to see if there is a link in it.

Social Networks/Communities of Practice/Knowledge Management

1. I believe each office should have a Questions Aswered and Comments staff to share information that may not trickled down to staff from management and for those employees who are not invited to attend informative meetings to be aware of recent policy, and procedural changes.

2. Communities of Practice – Social Networks

3. Isn’t there a Toastmaster’s club that meets regularly in the NCR ROB cafeteria?

4. It might be nice to know who/where/ are the residential experts on a variety of topics. . Such as “need information on” hoteling, where can a notary be found, carpooling, and other information that our colleagues might have a need to know

5. ‘Virtual library’ with form and other info, better-organized links (categorized)

InSite Design/Graphics/Images/Updates

1. The homepage needs to be more creative. Suggestions: along with the GSA symbol and flag at the top, make the main page a little more colorful, maybe changing according to the seasons (cherry blossoms for the Spring).

2. More images (very boring looking now)

3. Home page needs to be more attractive and colorful with graphics

4. I think the site is outdated and it appears from the surface (front page) to offer very little real value. I am sure there is a lot of useful information on InSite, but the site appears to static and hard to believe that it’s being updated at all.

5. Employees who use the tools and features need to be on the main page in bold. Easier to read, no searching, and the important features are on the front page instead of listing all tools on the side, they become very confusing.

PBS Specific

1. Samples of A&E Solicitation using the 2 step process

2. Being able to search by building address on the Building Locator search page would be nice. Thank you!

3. A continuously updated list of all the building managers and their assistants in all the GSA-operated buildings.

4. As an engineer I would like to see more topics on being more energy efficient.

5. Is it possible to access the database for lease files through Insite?

6. The site should be easier to navigate, the colors are boring and provide users with links to more in-depth information about and for the different PBS divisions

Miscellaneous

1. IT

2. Can’t think of anything at this time

3. More timely content – and less broken links.

4. None

5. Don’t have any suggestion.

6. The site is very well developed

7. Better organization of site.

8. Better organized – – terrible way to locate anything the site is confusing

9. Maybe an icon you can click on that give a short description of all of the different locations on the website so that it would be easy to see what is available.

10. I have none in mind.

11. At this time I can not think of anything that will improve InSite, it is very useful in its current format but taking this survey has encouraged me to take the time to surf through more topics to learn to better for navigating purpose.

12. Hot Links

Usability vs. UX: analysis of case studies

April 12, 2011 Leave a comment
Title of Study Results Tools Notes
1. Sullivan, Patricia. “Beyond a Narrow Conception of Usability Testing,” IEEE Transactions on Professional Communication, 32, 4, (December 1989):256 – 264 Suggests new frameworks for viewing usability studies methods and interpreting the validity of their results. Postulates that “a growing number of psychologists, engineers, and technical
communicators want to make the user more integral to the
whole development process.”
An analysis of other’s methods Questions Plain language movement probably has some influence on her too although not cited.
Title of Study
Results
2. Hassenzahl, M. and Tractinsky, “User Experience – a Research Agenda.” Behaviour and Information Technology, 25, 2, (March-April 2006): 91-97 Suggest a new theory of UX where designers exert control to ensure that a positive experience becomes certain. UX is about contributing to our quality of life by designing for pleasure rather than the absence of pain.
Conducted a literature review of proposals received.
One can sense the rhetorician at work who works to craft the pleasing experience and downplay any lack of quality. iPhone antenna problems for example.
Title of Study
Results Notes
3. Nielson, Jacob. Writing for the Web. http://www.useit.com/papers/webwriting/ Suggests many best practices to follow and also suggests further study of papers and books – and then finally recommends that one enroll in his courses.
Years of usability studies and analysis drawn from that body of work. A website’s rhetoric will be less effective if users find it difficult to read. Notable in that a brief space many salient points of how people read – and how writers should take this into account when creating online communications.
Title of Study
Results Notes
4. Obrist M., Roto V., and Väänänen-Vainio-Mattila K. “User experience evaluation: do you know which method
to use?” CHI 2009, April 4 – 9, 2009, Boston, Massachusetts, USA. Extended Abstracts 2009: 2763-2766.
Unknown – this was an abstract. However the questions were particularly illuminating. Contributions from conference attendees on current known methods. Creation of a Special Interest Group (SIG) that will identify and gather
people interested in UX evaluation in different application
areas and contexts. results.
Can we ever really know how the user feels? Do they even know? Or can we only influence positive feelings and minimize negative ones?
Title of Study
Results Notes
5. Bevan, Nigel. “What is the difference between the purpose of
usability and user experience evaluation methods?” Internet paper, http://www.nigelbevan.com/
Bevan notes a weakness in the methods – no metrics or requirements. He states that “user experience
seems to . . . .focus on evaluation [which] has preceded a concern with establishing
criteria for what would be acceptable results of evaluation. That comment was useful as I, too, wondered where the UX standards were.
Rigorous analysis of the UX methods and creation of a categorization of usability measures reported. He then compares and contrasts each method as to how it measures UX or usability. usable as roadmap of what one is measuring and how to do it better
Title of Study Results Tools Notes
6. Rodden et al, “Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications”, Proceedings of CHI 2010.http://www.rodden.org/kerry/heart Creation of a UX framework – HEART: (Happiness, Engagement, Adoption, Retention, Task success). This was used to measure user satisfaction for a major redesign for iGoogle. They reported an initial decline in their user satisfaction metric (measured on a 7-point bipolar scale). However, this metric recovered over time, indicating that change aversion was probably the cause, and that once users got used to the new design, they liked it. With this information, the team was able to make a more confident decision to keep the new design. Happinesswas measured via a weekly survey on a 7-point bipolar scale).Engagement% of active users who visited 5 or 5+ days of the last week.Adoption how many new users? (i.e. # of accounts created in a week).

Retention how many users are still present (i.e. % of 7-day active users in a given week still active 3 months later).

Task success efficiency (e.g. time to complete a task), effectiveness (e.g. % of tasks complete), and error rates.

It makes sense to add a scale to UX measurements. Couldn’t it go to 11? Is it wrong to apply usability metrics to UX?
Title of Study
Results Notes
7. Large organizations need to track and compare their online sales, customers and trends such as shopping cart abandonment. http://blog.goecart.com/index.php/proven-website-conversion-tips/ Creation of overall framework to measure several factors to better identify causality. PULSE metrics: Page views, Uptime, Latency, Seven-day active users (i.e. the number of unique users who used the product at least once in the last week), and Earnings. Most of this data is proprietary and unavailable. Large ecommerce firms (Amazon, Ebay, Facebook) do have inhouse models and ongoing studies but this data is not shared nor publicly available.
Title of Study
Results Notes
8. How can Blackboard, Inc. better capture feedback and improve the UX on its web pages and software products?Presented at UX BarCamp DC in Jan. 2011http://www.slideshare.net/bbuiax/design-for-the-rudes Blackboard created a framework for capturing user feedback. RUDES: Reliable, Useful, Delightful, Engaging, Simple. Users rate each experience as the RUDES and is asked if each component exceeds, meets, or misses.Unknown – appears to be a work in process. Unknown. Blackboard staff stated that scaling factors were necessary to make better design decisions. They did not disclose how this data would be collected, analyzed or used. Worth noting that the desired answer is positioned first. How good is a survey if one tries to influence it so strongly?
Title of Study
Results Notes
9. Fornell, Claes. (2011) “Citizen Satisfaction with Federal Government Services Plummets While Satisfaction With Government Websites Remains Strong”. News release and commentary.http://www.theacsi.org/index.php?option=com_content&task=view&id=236&Itemid=259 Nonsensical – agencies mission’s vary so widely that to compare satisfaction rates means nothing. Can one compare NASA to IRS? TSA to DOI? The popup survey ACSI reports scores on a scale at the national level for more than 225 companies, and over 200 federal or local government services. causes and consequences of customer satisfaction. The surveys vary among websites so comparing one federal agency’s score to anothers is not comparable – yet it is widely done.

Give great web

March 25, 2011 Leave a comment

Your web page is the front door to the public and to your customers. It’s the first thing that most people see. Your web pages creates impressions and brands your agency. I agree with Candi Harrison that we must strive to create a positive image and provide great customer service.

To create a powerful, positive brand as a caring, proactive customer oriented agency we must create government web pages that follow these rules:

1. Audience – specify your audience – in the first sentence if possible:
a. Citizens/Industry/Agencies/Contracting Officers/Attorneys/Industry can find . . .
b. If its unclear – use “Are you looking for . . . ?”

Boring pages start with xxx agency provides aaaa services to bbbb. Defining your audience puts your customers first – where they belong.

2. Task – identify the tasks someone can complete. Make easy for your users to register for an event, sign up for a newsletter, buy your products/services, contact you, or fill out a form. Easy means just that – a user unfamiliar with your page should be able to figure out who its for and what they can do on your page within ten seconds. Usability testing is available from GSA’s First Fridays. Also refer to usability.gov for guidelines your pages should follow.

3. Readability – your pages should be readable at the 8th grade level – use the SMOG index to check them. Do this now as all federal web pages will have to be written in Plain Language by Oct 2011.

4. Brevity – are your web pages concise? Short so users don’t have to scroll? Be considerate of your busy user’s time.

5. Current – are your web pages up to date? Check to be sure that announcements within news releases, publications, powerpoint presentations, speeches and/or blog posts are also included on your web pages. This is also an OMB requirement.

If your web pages meet all of the above – congratulations! If not, don’t worry about it – do something about it. Take steps to address these problems. Most web pages are constantly evolving. Call it continuous improvement, agile, six sigma, or whatever you like – but do keep working on your web pages to ensure that your customers get the very best that your agency and you can give.

And yes – federal web policies, best practices and guidelines – apply to all web pages including yours. Everyone appreciates an easy to read page. No one has ever read a web page that gave guidance on an important issue and thought – “Wow, I wish that was harder and more complicated.”

We all search for information, stumble upon pages and need to quickly decide if we are following the right information path. Help your customers, and help your agency make a better impression on how well it delivers great customer service.

Usability Testing: Getting surprised by our users

February 25, 2011 Leave a comment

Since Oct. 2010, my team has conducted four usability tests on our government website. What we have found astounded us. I’m now convinced that usability testing for all government websites should be required because:

1. Our users did not read our homepage to navigate or follow links. Instead, they just searched. We’ve spent so much of our resources on crafting navigation paths and menus. Our users want only quality search results.

2.  When our users searched, the search results were not optimal. We need to reallocate our resources to ensure our search results help our users complete their tasks.  Now that we know how important search is – we need immediate resources dedicated to ensuring search engine optimization.

3.  Users scanned our pages, looking for keywords that made sense to them. Their keywords were not our keywords. We assumed that users used our jargon but users searched using simple words and not our program names. If the information was at the bottom of the page the users questioned why it had been put there – it seemed that they thought important information (to them) belonged in only one place – at the top of the page.

4.  Users sometimes didn’t trust the links on our pages even when it seemed that it was the correct one for them to select to complete the task. They would look at us and ask if they should click it. Why wouldn’t they just follow the link? Why were they hesitant? We don’t know the answer to this question.

So we’ve learned a lot from our users, and need to put the information together in a presentation for our program managers. I’ll write more about that in my next blog.

Eyetracking, heatmapping and your website

July 2, 2007 1 comment

What is eyetracking? Heatmapping? Why should you care?

For website creators its close to the gold standard for learning how users actually use your website. I attended an Eyetracking Demonstration led by Dr. Kathryn Summers, University of Baltimore & Michael Summmers, Summers Consulting, Inc. in collaboration with Nick Boswell, Tobii Technology and GSA’s Web Manager University, which produced the July 2007 event.

Eyetracking follows a user’s eyes as they try to accomplish a specific task on your website. During the presentation, Michael Summers explained that measuring exactly where user’s eyeballs look on a page – as well as where they do not look – can be vital for website managers. Web managers need to know, for example, to not put anything in the upper right corner – it won’t be seen. We’ve become conditioned to expect advertising there and our eyes avoid looking at that area.

We watched users look for the Federal Emergency Management Agency (FEMA) phone number on USA.gov. We could see from eyetracking data – represented as red lines and dots – that users were scanning furiously, choosing many paths throughout the website, making false starts, but finally completing the task. Their difficulties indicated that users needed more help to be able to accomplish this task easily. People who are in a disaster and looking for FEMA’s phone number are likely to be impatient and need to find this information as quickly as possible. One can see the logic in performing these tests as you can literally see the information through another’s eyes.

Heatmaps were created at the end of each user’s eyetracking tests. Generally heatmaps demonstrate what areas of the webpage are viewed most, and where user’s eyes lingered the longest (indicating areas of most interest). Heatmaps can generated in other ways – by clickthroughs on the links within a page – but this blog post is not about those types of heatmaps.

Rules of thumb that I learned from this demo:

I asked if there was eyetracking data that tracks shopping cart abandonment. Well – no. This research is proprietary and in high demand. Most ecommerce based organizations are highly interested in what factors lead to shopping cart abandonment, and conversely, conversion rates – how to convert users into buyers. Jakob Hencke, of Tobii Technology, said that the  Tobii forum is a good place to research that information. Google’s website optimizer forum may be a good resource too.