Great websites are built to serve users’ needs quickly and easily. If you build an online community but understand the tasks your users need to complete, and make those tasks easy to complete – then your website will #fail.
The first task was to establish our goals. We decided to improve the layout, brighten the color palette, and reposition links to improve the functionality of the intranet for our users. We also wanted our intranet – which we call NCR InSite – to save users significant time and money. Dell estimates that its intranet creates annual productivity gains of $36 million. Government intranets are different and it’s hard to see what is done behind their firewalls – but we found Australia has some excellent oldie but goody intranet best practices online.
Our second task was to learn what tasks GSA staff wanted to accomplish on NCR InSite, and what they hated the most. We sent out an email to over 1,200 employees to ask for their help to make NCR InSite better. 1 We got back over 135 responses – a response rate better than 10% which is stellar.
Within a month our team had made most of the changes pinpointed in the survey. We made significant headway towards implementing most of the changes people had asked for. And the best outcome was that our employees were pleased with the results and proud of their InSite, bragging that it was one of the best within GSA. We couldn’t make all of the changes that everyone asked for, but we did at least ask people what they wanted. Listening to users can be very powerful.
Email to NCR InSite users sent March 2007
Help us improve NCR InSite
We are committed to improving NCR InSite to ensure that it meets your needs – provide basic information and tools that you need to do your job effectively, information about jobs, pay, benefits, and services; and help employees learn more about GSA’s businesses, policies, and programs.
Recently, work has begun behind the scenes to expand and improve NCR’s Insite. For example, events have been added to the NCR calendar. Users can more easily NCR’s shuttle schedule, cafeteria menu, and remote email address, as the icons on the horizontal navigation bar are now identified with text reflecting their purpose. The cafeteria menu is up-to-date.
However, we need your help to ensure that the revisions reflect your needs. We would like your opinion and feedback on your visit to our website so that we might redesign it in a manner that is more user friendly and is more efficient for you and for others. We hope you take the time to help us by completing a brief survey. All information collected from the survey will be kept confidential and only will be used in support of improving NCR’s InSite.
Based on the feedback we receive, we will improve NCR’s InSite to make it easier to use. We expect to be implementing these changes over the next few months and will inform you in advance of the changes so as to minimize any disruptions or inconvience.
We are looking forward to the opportunity to hearing from you, and helping you have a more productive experience using NCR InSite.
The NCR InSite Team
NCR InSite Survey Results
Survey conducted from 3/21 – 3/30/07
|1. What tools and features do you currently use most on InSite?|
Not at All
|Calendar of Events||
|Regional Links to weather, traffic||
|Self-Assessment Guides (APPAS)||
|Human Resources Information||
|IT EquipmentRequest/ComputerMove Form||
|IT Security TipsAnd Info||
|Thanks program, archived news and Ask Ann|
|Shuttle Schedule, access GSA InSite and home page|
|Lotus email link to check email through vpn from home|
|I use the web access Citrix to access my VPN Account while telecommuting|
|Cafeteria Menu Link|
|Reading the Ask Ann items|
|Chris, Thanks Program|
|Employee HR & Web Applications|
|PBS, Financial Resources Links|
|E2 Travel, Thanks!|
|Building Locator, Webforms|
|Building matrix, division website, project website|
|Project information portal, citirix|
|PBS Resources, Asset Building Plans, Rent Roll, Pricing Desk Guide|
|GSA Directory Locator|
|Launch Web based applications such as FMIS|
|Acquisition website for model specs and procurement bulletins|
|2. What is the primary method you are using to find information on InSite?|
|List of Topics (on the homepage and in the left navigation column of sub-pages)||
|Top Navigation Bar||
|None – these are not helpful – especially the search engine|
|The search engine doesn’t give relevant hits|
3. What is the most important content you would like to see developed for future inclusion on the NCR InSite?
1. Better Search Engine that is effective for valuable info retrieval.
2. A reasonable and simple search system – maybe modeled after google.
3. Not content, but a good search engine for the site
4. Search features and the documents they produce need to be more relavent
5. A search engine that works better would help.
6. A better search engine
7. Better search results
8. Better Search String
9. Search feature needs improvement, old pages need to be updated or removed
10. Search effectiveness is poor.
11. Word search database
12. There needs to be a better search function for locating topics within InSite
13. Better search function for finding regulations and related memo(s) and interpretations of GSA/NCR relevant policies particularly for the financial/budget and real estate communities.
14. I would like my searches to require no more than 3 clicks. I also think that it will help the user to know what information is included in the words shown in the screen. The best websites allow you to run the cursor over a “title” and see in a summary of what it refers to. A final suggestion is to make the GSA Form database user friendly.
1. NCR Directory with photographs and Office
2. Org chart related to personnel locator with a link to the PBS Portal
3. A more detailed organization chart
4. Organization guide to show all employees by organizational unit
5. Provide organizational charts for the NCR that provide links to information on services and contacts in respected organizations. example, in the PBS area, Potomac Service center. What customers and buildings are they servicing-a map of their area using GUI may be useful, what are the key projects with points of contact, being worked on or are in the planning phase. Overall find our Web site to be useful and easy to access. Also, linkage across all our business lines may make it easier to locate key contacts on projects. Hope this information is useful.
6. There needs to be a quick way to find Organization Charts along with names of key players. A photo of each party would be even better. I saw this on another regions InSite and found the Org. Charts invaluable.
Events and News
1. Homeland Security and COOP advisories should be featured in a Breaking News format – maybe consider text messaging.
2. More NCR internal news.
3. Put all event announcements in right side following other news announcements.
4. Regional Happenings
5. Cafeteria and Lunch time specials.
6. Employee news.
7. Upcoming events, with background information.
8. Need to make the calendar of events more visible and usable.
9. Future developments – please feature this.
10. Different manner of presenting “what’s going on in NCR.” Current look is old and boring
11. Stop lights on how we as a region are doing with performance goals.
1. This may have been corrected, but the HR tab within NCR Insite are not user friendly and information should be updated
2. HR Guidance for Supervisors
3. This is not very important. But I would like to have the calendar that shows the shows the holidays and pay dates for the year.
4. GSA Jobs website – Ready & available for GSA associates
5. I would like to see more job information on the quick hire process.
6. HR info should be current and complete. Associates should be compelled to fill in complete details and update them promptly when there is a revision. Announce relocations of persons like contracting officers and project managers.
7. Would be nice to see Performance Review / dates of APPAS actions somewhere – maybe on calendar – so employees and managers could plan ahead of time for this process. A simple how to guide, with a phone number of someone in HR concerning APPAS issues, would be wonderful.
8. Create a section for the new employees so we know who they are when we see them and they feel welcomed. Also create a section for retiring employees so we are aware of this and can make plans ahead of time.
Conference Rooms/Hotels/Restaurants/Meeting Resources
1. Prop Dev. Conference Rooms and other support
2. Listing of conference rooms.
3. A list of nearby restaurants, cafes, eateries, etc. would help out employees, contractors and visitors.
4. List of hotels, conference rooms, and meeting facilities for planning future meetings.
1. Phone book could be more user friendly. You have to have the correct spelling or it will not find items.
2. I recommend that a cleanup be made of the Employee Locator database. There are multiple problems with hyphenated names, middle names, initials added to the first name, as well as nicknames.
1. GSA Credit Union Information
1. Phone numbers for the building manager’s office and the IT help desk should be MUCH easier to find (should be one click away from the home page). Link to Current Clips would be helpful. Occasional graphic, pictorial and-would liven things up a bit. It’s a helpful, well-run site, though. Thanks!
2. Links to other GSA Region’s intranet sites
3. Need better layout, make easier to find things (example – PBS customer list is hidden in PBS organization)
4. I like the fact that you can now see the icons. Please add the Motor Pool to the icons.
5. Make the overseas travel in E2 work.
6. Link to Central Office pages with contacts or organizational structure
7. Direct links to other regional offices
8. When titling resources, please ask the users to assist in the actual naming. Too often, names are dreamed up that no one knows what it might refer to. Experienced persons such as our contractor, Mark, can give some insights in this regard.
9. The links to policy documents needs to be clearer. There should be links to GSA Insite for National Policy on the same topics, while you looking at the regional information.
10. Please underline links! It is so annoying that you have to mouse over stuff to see if there is a link in it.
Social Networks/Communities of Practice/Knowledge Management
1. I believe each office should have a Questions Aswered and Comments staff to share information that may not trickled down to staff from management and for those employees who are not invited to attend informative meetings to be aware of recent policy, and procedural changes.
2. Communities of Practice – Social Networks
3. Isn’t there a Toastmaster’s club that meets regularly in the NCR ROB cafeteria?
4. It might be nice to know who/where/ are the residential experts on a variety of topics. . Such as “need information on” hoteling, where can a notary be found, carpooling, and other information that our colleagues might have a need to know
5. ‘Virtual library’ with form and other info, better-organized links (categorized)
1. The homepage needs to be more creative. Suggestions: along with the GSA symbol and flag at the top, make the main page a little more colorful, maybe changing according to the seasons (cherry blossoms for the Spring).
2. More images (very boring looking now)
3. Home page needs to be more attractive and colorful with graphics
4. I think the site is outdated and it appears from the surface (front page) to offer very little real value. I am sure there is a lot of useful information on InSite, but the site appears to static and hard to believe that it’s being updated at all.
5. Employees who use the tools and features need to be on the main page in bold. Easier to read, no searching, and the important features are on the front page instead of listing all tools on the side, they become very confusing.
1. Samples of A&E Solicitation using the 2 step process
2. Being able to search by building address on the Building Locator search page would be nice. Thank you!
3. A continuously updated list of all the building managers and their assistants in all the GSA-operated buildings.
4. As an engineer I would like to see more topics on being more energy efficient.
5. Is it possible to access the database for lease files through Insite?
6. The site should be easier to navigate, the colors are boring and provide users with links to more in-depth information about and for the different PBS divisions
2. Can’t think of anything at this time
3. More timely content – and less broken links.
5. Don’t have any suggestion.
6. The site is very well developed
7. Better organization of site.
8. Better organized – – terrible way to locate anything the site is confusing
9. Maybe an icon you can click on that give a short description of all of the different locations on the website so that it would be easy to see what is available.
10. I have none in mind.
11. At this time I can not think of anything that will improve InSite, it is very useful in its current format but taking this survey has encouraged me to take the time to surf through more topics to learn to better for navigating purpose.
12. Hot Links
My recent trip to the Plain 2011 conference in Stockholm, Sweden was very worthwhile. And that’s saying a lot – as I covered my costs for this three day international conference. I attended because the USA now has a Plain Writing Act which requires all federal agencies’ public facing published content, especially online, to be written in Plain Language as of October 2011. I wanted to learn what other countries have learned from their experience with Plain Writing laws. The sessions I enjoyed the most were:
Annette gave an insightful and amusing history of Plain Language and an overview of how a small group of people decided to make a federal law and how they did it. That alone was worth the trip across the Atlantic. I asked how will agencies be measured if their websites are written in Plain Language? Maybe the ACSI survey could ask users to rate the simplicity of the writing? Maybe it will
Helena Haapio (Finland) – Communicating Contracts – when text alone is not enough.
Because so many of my agency’s staff are contracting officers, I was eager to learn more. Helen’s handout had many tips on using tables, illustrations, and examples of succinct and simply written phrases. She also talked about the well known example of The Comma That Costs 1 Million Dollars (Canadian). Rogers Communications and Aliant had a contract dispute over a contract containing a sentence with 45 words, with a comma that a court interpreted as meaning that Aliant could renegotiate it’s five year contract after only one year. My contracting officer coworkers have found it useful and intesting.
Thomas Mueller (Germany), of Siegel + Gale, the cost of complexity
This presentation stated that brands who provide customers with simple communications and processes have great ROI. Also available is the 2010 Global Brand Simplicity Index: United States. An alternative title could be – if you aren’t writing in plain language and providing excellent usability to your customers you could be leaving money on the table. The lowest ranking brands for simplicity were utilities and financial institutions. If you invested your own money in companies rated highest in providing simplicity in their communications and interactions, your investment would outperform all others. Siegel+Gale has a blog with entries on simplicity – worth reading.
Sue Owen (Australia) , Using plain language in emergency warnings.
Sue spoke of how – tragically – many people did not understand that they should evacuate due to bushfires and floods in the Victoria province and subsequently lost their lives. A hearing determined that the warnings were written in dense, unclear, and ambiguous terms. Afterwards new guidance was issued requiring disaster warnings issued in Victoria to use clear language, avoid euphemisms, and contain specific information in relation to:
• the severity, location, predicted direction and likely time of impact of bushfires on specific communities and locations; and
• the predicted severity of impact of the bushfire and whether a specific fire poses a threat to human life.
The warnings must address and answer these questions:
- where is the threat?
- when will it be here?
- how bad will it be?
- what they can/should do
Cathy Baskerfield (Australia) – A synopsis of research when writing for people with limited literacy skills. Her presentation covered research on how to better reach the population with low literacy. Much of any country in the world has at least 20% of its population coping with low literacy (immigrants, disabled, the elderly, indigeneous) . She postulates that Plain Language and information design is inherently interdisciplinary. Slides 26 – 33 show that combining good graphics with text:
• Can improve comprehension
• Support relationship among ideas
• Show spatial relationship
• Change adherence to information, for example, taking prescription tablets
|Title of Study||Results||Tools||Notes|
|1. Sullivan, Patricia. “Beyond a Narrow Conception of Usability Testing,” IEEE Transactions on Professional Communication, 32, 4, (December 1989):256 – 264||Suggests new frameworks for viewing usability studies methods and interpreting the validity of their results. Postulates that “a growing number of psychologists, engineers, and technical
communicators want to make the user more integral to the
whole development process.”
|An analysis of other’s methods Questions||Plain language movement probably has some influence on her too although not cited.
|Title of Study
|2. Hassenzahl, M. and Tractinsky, “User Experience – a Research Agenda.” Behaviour and Information Technology, 25, 2, (March-April 2006): 91-97||Suggest a new theory of UX where designers exert control to ensure that a positive experience becomes certain. UX is about contributing to our quality of life by designing for pleasure rather than the absence of pain.
||Conducted a literature review of proposals received.
||One can sense the rhetorician at work who works to craft the pleasing experience and downplay any lack of quality. iPhone antenna problems for example.
|Title of Study
|3. Nielson, Jacob. Writing for the Web. http://www.useit.com/papers/webwriting/||Suggests many best practices to follow and also suggests further study of papers and books – and then finally recommends that one enroll in his courses.
||Years of usability studies and analysis drawn from that body of work.||A website’s rhetoric will be less effective if users find it difficult to read. Notable in that a brief space many salient points of how people read – and how writers should take this into account when creating online communications.|
|Title of Study
|4. Obrist M., Roto V., and Väänänen-Vainio-Mattila K. “User experience evaluation: do you know which method
to use?” CHI 2009, April 4 – 9, 2009, Boston, Massachusetts, USA. Extended Abstracts 2009: 2763-2766.
|Unknown – this was an abstract. However the questions were particularly illuminating.||Contributions from conference attendees on current known methods. Creation of a Special Interest Group (SIG) that will identify and gather
people interested in UX evaluation in different application
areas and contexts. results.
|Can we ever really know how the user feels? Do they even know? Or can we only influence positive feelings and minimize negative ones?|
|Title of Study
|5. Bevan, Nigel. “What is the difference between the purpose of
usability and user experience evaluation methods?” Internet paper, http://www.nigelbevan.com/
|Bevan notes a weakness in the methods – no metrics or requirements. He states that “user experience
seems to . . . .focus on evaluation [which] has preceded a concern with establishing
criteria for what would be acceptable results of evaluation. That comment was useful as I, too, wondered where the UX standards were.
|Rigorous analysis of the UX methods and creation of a categorization of usability measures reported. He then compares and contrasts each method as to how it measures UX or usability.||usable as roadmap of what one is measuring and how to do it better
|Title of Study||Results||Tools||Notes|
|6. Rodden et al, “Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications”, Proceedings of CHI 2010.http://www.rodden.org/kerry/heart||Creation of a UX framework – HEART: (Happiness, Engagement, Adoption, Retention, Task success). This was used to measure user satisfaction for a major redesign for iGoogle. They reported an initial decline in their user satisfaction metric (measured on a 7-point bipolar scale). However, this metric recovered over time, indicating that change aversion was probably the cause, and that once users got used to the new design, they liked it. With this information, the team was able to make a more confident decision to keep the new design.||Happinesswas measured via a weekly survey on a 7-point bipolar scale).Engagement% of active users who visited 5 or 5+ days of the last week.Adoption how many new users? (i.e. # of accounts created in a week).
Retention how many users are still present (i.e. % of 7-day active users in a given week still active 3 months later).
Task success efficiency (e.g. time to complete a task), effectiveness (e.g. % of tasks complete), and error rates.
|It makes sense to add a scale to UX measurements. Couldn’t it go to 11? Is it wrong to apply usability metrics to UX?
|Title of Study
|7. Large organizations need to track and compare their online sales, customers and trends such as shopping cart abandonment. http://blog.goecart.com/index.php/proven-website-conversion-tips/||Creation of overall framework to measure several factors to better identify causality. PULSE metrics: Page views, Uptime, Latency, Seven-day active users (i.e. the number of unique users who used the product at least once in the last week), and Earnings.||Most of this data is proprietary and unavailable. Large ecommerce firms (Amazon, Ebay, Facebook) do have inhouse models and ongoing studies but this data is not shared nor publicly available.|
|Title of Study
|8. How can Blackboard, Inc. better capture feedback and improve the UX on its web pages and software products?Presented at UX BarCamp DC in Jan. 2011http://www.slideshare.net/bbuiax/design-for-the-rudes||Blackboard created a framework for capturing user feedback. RUDES: Reliable, Useful, Delightful, Engaging, Simple. Users rate each experience as the RUDES and is asked if each component exceeds, meets, or misses.Unknown – appears to be a work in process.||Unknown. Blackboard staff stated that scaling factors were necessary to make better design decisions. They did not disclose how this data would be collected, analyzed or used.||Worth noting that the desired answer is positioned first. How good is a survey if one tries to influence it so strongly?|
|Title of Study
|9. Fornell, Claes. (2011) “Citizen Satisfaction with Federal Government Services Plummets While Satisfaction With Government Websites Remains Strong”. News release and commentary.http://www.theacsi.org/index.php?option=com_content&task=view&id=236&Itemid=259||Nonsensical – agencies mission’s vary so widely that to compare satisfaction rates means nothing. Can one compare NASA to IRS? TSA to DOI?||The popup survey ACSI reports scores on a scale at the national level for more than 225 companies, and over 200 federal or local government services. causes and consequences of customer satisfaction.||The surveys vary among websites so comparing one federal agency’s score to anothers is not comparable – yet it is widely done.|
Your web page is the front door to the public and to your customers. It’s the first thing that most people see. Your web pages creates impressions and brands your agency. I agree with Candi Harrison that we must strive to create a positive image and provide great customer service.
To create a powerful, positive brand as a caring, proactive customer oriented agency we must create government web pages that follow these rules:
1. Audience - specify your audience – in the first sentence if possible:
a. Citizens/Industry/Agencies/Contracting Officers/Attorneys/Industry can find . . .
b. If its unclear – use “Are you looking for . . . ?”
Boring pages start with xxx agency provides aaaa services to bbbb. Defining your audience puts your customers first – where they belong.
2. Task - identify the tasks someone can complete. Make easy for your users to register for an event, sign up for a newsletter, buy your products/services, contact you, or fill out a form. Easy means just that – a user unfamiliar with your page should be able to figure out who its for and what they can do on your page within ten seconds. Usability testing is available from GSA’s First Fridays. Also refer to usability.gov for guidelines your pages should follow.
3. Readability – your pages should be readable at the 8th grade level – use the SMOG index to check them. Do this now as all federal web pages will have to be written in Plain Language by Oct 2011.
4. Brevity – are your web pages concise? Short so users don’t have to scroll? Be considerate of your busy user’s time.
5. Current – are your web pages up to date? Check to be sure that announcements within news releases, publications, powerpoint presentations, speeches and/or blog posts are also included on your web pages. This is also an OMB requirement.
If your web pages meet all of the above – congratulations! If not, don’t worry about it – do something about it. Take steps to address these problems. Most web pages are constantly evolving. Call it continuous improvement, agile, six sigma, or whatever you like – but do keep working on your web pages to ensure that your customers get the very best that your agency and you can give.
And yes – federal web policies, best practices and guidelines – apply to all web pages including yours. Everyone appreciates an easy to read page. No one has ever read a web page that gave guidance on an important issue and thought – “Wow, I wish that was harder and more complicated.”
We all search for information, stumble upon pages and need to quickly decide if we are following the right information path. Help your customers, and help your agency make a better impression on how well it delivers great customer service.
Since Oct. 2010, my team has conducted four usability tests on our government website. What we have found astounded us. I’m now convinced that usability testing for all government websites should be required because:
1. Our users did not read our homepage to navigate or follow links. Instead, they just searched. We’ve spent so much of our resources on crafting navigation paths and menus. Our users want only quality search results.
2. When our users searched, the search results were not optimal. We need to reallocate our resources to ensure our search results help our users complete their tasks. Now that we know how important search is – we need immediate resources dedicated to ensuring search engine optimization.
3. Users scanned our pages, looking for keywords that made sense to them. Their keywords were not our keywords. We assumed that users used our jargon but users searched using simple words and not our program names. If the information was at the bottom of the page the users questioned why it had been put there – it seemed that they thought important information (to them) belonged in only one place – at the top of the page.
4. Users sometimes didn’t trust the links on our pages even when it seemed that it was the correct one for them to select to complete the task. They would look at us and ask if they should click it. Why wouldn’t they just follow the link? Why were they hesitant? We don’t know the answer to this question.
So we’ve learned a lot from our users, and need to put the information together in a presentation for our program managers. I’ll write more about that in my next blog.