eGovMoNet: measurement of user satisfaction of e-Government solutions
Under the supervision of Agder University’s Mikael Snaprud, the second meeting of eGovMoNet (the European network focusing on strategies to measure user satisfaction of e-Government solutions) took place in Copenhagen on October 24-25, 2008. Zea Partners is involved in this European thematic network as representative of the PloneGov project, an international open source software sharing initiative.
Meeting report: Copenhagen Businesses School, DK - 24-25 Oct 2008
The second initiative was conceived to take one step forward the results of the 29-30 May 2008 Geneva kick-off meeting of eGovMonet, and in particular to:
- Present developments since the kick-off meeting.
- Introduce the first thoughts on how to describe the methods to compare them.
- Present methodologies in use according to the first measurement template.
- Discuss similarities and differences of the methodologies.
- Start to elaborate a set of key indicators for eGovernment impact and user satisfaction.
- Introduce a project handbook and hints on how to use the project work site.
- Offer a summary of project status and an update project progress plans and review status of project indicators.
As usual, there was plenty of presentations. In the practical impossibility of letting our readers have a detailed report, we strongly suggest anybody interested to download the full presentation texts at the following URL: http://www.egovmonet.eu/2nd-meeting
Herein under, as in the case of the kick off meeting, we propose some of the highlights of this second step towards eGovMoNet maturity: it will be very evident that confidence continues to build as the network hones its tools and its ability to harmonize them.
- The [email protected] InfoAccessibility Observatory, co-funded by ONCE Foundation (Spain's National Foundation for the rights of Blind People and of the people with perceptive or motion deficiencies) and the European Social Fund, related about their technical assessment and user satisfaction testing methods, applied to key sectorial eGovernment websites and services, and including self-administered user testing and questionnaires
- Stephen Jenner talked about The UK Central Government perspective about eGovernment and Customer Satisfaction measurement, illustrating the service transformation programme that - starting from the vision (let the service meet the needs of people rather then the needs of government), defining the strategic themes (including benchlearning, rationalizing, grouping services in a rational way, linking local and central government) and defining the measures that will let the government engage in customer satisfaction tracking mechanisms. The resulting "cycle of customer insight and service improvement", vaguely resembling a PDCA wheel, and key areas of improvement were also a subject of this talk.
- Chiara Mancini from Emilia-Romagna Region (Italy) illustrated the co-design methodology, a strategy to design on line public services. This approach is based on two macro-phases: one, profiling users by expected benefits and satisfaction on e-government services (by preparing and executing a Computer-Assisted Telephone Interviewing survey on 3000 citizens in Emilia-Romagna on expected benefits and user satisfaction of e-government services). Two, “measuring” best practices, by observation sessions with both service users and non-users, and administering an evaluation questionnaire. Batteries of indicators included: transparency, e-democracy, multilingual approach, user oriented content organization, availability of special virtual offices, accessibility, usability, multichannel delivery
- Eleni Vergi from the Observatory for the Greek Information Society talked about the Mistery User Methodology (MUM), a form of participative evaluation in which citizens, as users of the services, are brought in as active players of the evaluation process: their feedback assists in making e-services more citizen/ business centric. A structured, five step (Locate service / Login to service / Locate form / Complete form / Submit) questionnaire was administered to a sample of users of an online tax submission services, also requested to express an overall evaluation on all the procedure. results were discussed.
- Uros Pivk presentation was about Measuring e-Government user Satisfaction in Slovenia. Among the many interesting points of this case, the gap between expressed interest from potential users and the actual use of e-government information and services. Factors influencing that gap have not yet been explain and understood. The three factors: added value, public awareness and trust are equally important for the success of any eGovernment initiative.
- Charlie Wallin (Kommunforbundet Vasternorrland) talked about E-governance in Sweden presented the main e-government initiatives and the role of VERVA, Swedish Administrative Development Agency, promoting e-government services and producing Swedish National Guidelines for Public Sector Websites (www.verva.se). The involvement of accredited consultants in usability tests is mandatory.
- Miriam Brašková and Anton Lavrin (Technical University of Košice, Slovakia) let the audience know about "Highlights from Measurement Practice in Slovakia". After an overview of the state of art of eGovernment initiatives, the conclusions stressed the lack of methodology design applicable to all levels of eGovernment services, and observed that - as long as customers use the "contact" section of any website just to solve specific problems - the use of "FAQ" and "Contact" sections as sources of data for Customer satisfaction analyses may be problematic. Country specific problems delaying the adoption of the national programme for information society were also addressed.
- In its Interactive Session entitled "Stakeholder Perspectives" Professor Jenny Rowley (Manchester Metropolitan University, UK) promoted and facilitated the discussion among eGovMoNet participants. After an introduction two plenary sessions took place: "Group activity: Exploring Stakeholders and their Perspectives" and "Towards an Understanding of Stakeholder Interests". In the frame of a typology of stakeholders groups/roles (internal / customer / influence / referral / employee / supplier) the experience of putting oneself in some specific stakeholder's shoes may be intriguing and challenging, but was also an useful mind-opener for all the participants.
- In her complex and insightful presentation, Christine Mahieu, Business Analyst FEDICT, related about the Belgian practice about user satisfaction and impact measurement. After providing some historical background about "eGov Measurement Framework Design" implemented in 2008, Mahieu described the methodology based on quantitative and qualitative survey, stressing the importance of defining the aims (status assessment, impact) and the intended users of measurement results (policy makers, researchers, public). All raw data will be available on ePractice, already hosting a public paper in 4 languages with aggregated data about the results. methodology cost was about 125.000 euros for 3 waves of survey and 8 focus groups. Experience and lessons learned from using the methodology were also illustrated.
- Peter Röthig (www.wibe.de), after his kick-off talk, illustrated the connection of the WiBe Economic Efficiency Assessment concept to ISO/IEC 9126 - Software Engineering Product Quality standards. After delving in some depth into both the standards, showing how WiBe 4.x takes into account both the monetary and non-monetary sense of economic efficiency, Peter Röthig illustrated the harmonization of ISO and WiBe approaches along the life cycle of the product, gradually shifting from the hands of "virtual users" (the planning phase) to 100% real users (the application phase). Along the time axis, the need for user feedback grows. The need of policy maker decision tables and of three user group specific satisfaction indexes (customer, satisfaction, co-administrations) was covered in the last part of the talk.
- After the kick off talk, Deloitte contributed again to the speeches with a talk by Patrick Wauters, about the preliminary results of a "STUDY ON USER SATISFACTION AND IMPACT IN EU27". The standardized methodology was illustrated in great detail: the states involved included Belgium, Netherlands, France, UK, Sweden, Austria, Italy, Spain, Poland, Germany. The survey was relative to 1000 citizens – 400 companies per Member State (a grand total of 10.000 citizens, representative of "the internet population" and 4000 representatives of companies, a mix of SME and large companies). The resulting data were analyzed in depth and in a multi-faceted approach. The differences in socio-demographics in the 10 selected Member States apparently did not influence the overall picture, illustrated in great detail in several tables. eGovernment services are overall used by 76% of the sample, with a peak for "looking for a job" and for "declaring income taxes" services. The great deal of data and lessons learned cannot be summarized here, but surely the exposition was effective and interesting.
eGovMoNet is very lively, and the future of the network seems promising, as long as the scientific and informative quality of the network is already going beyond the most optimistic expectations.