The Which forum
There’s little doubt that the forum proved wildly successful in its early days but soon proved extremely problematic for the CA. For the first time, members were able to directly question Which? senior management. Some, like Deputy Director Kim Lavely, or Directors Dave Roberts and Alan Stevens or Customer Services Manager Fergus Smyth or Head of Online Paul Kitchen were able to handle this with deft skill, frankness and openness, but others felt under siege and unable to deal with the increasingly perceptive and often testing questions emanating from the online membership.
When Peter Vicary-Smith was appointed his initial approach appeared to be about dismantling the structure loyal to the previous administration. He also seemed to have as his main priority the financial stability of the institution. Regrettably, he refused to use the forum in any way, which suggests he clearly saw it - if not as a threat - something which had to go.
New executives moving in and getting rid of dead wood is nothing new, but this was not an ordinary organisation. It was a charity-led institution, that espoused fairness, equality and even-handedness, but it’s hard to see how those qualities figured in Vicary-Smith’s ‘reorganisation’.
The forum’s lonely path - the Great Experiment
At this point it’s probably a good idea to examine the history of the forum.
The original Which Online Forum System
Thanks to JB for copious notes
The Consumers' Association started Which Online around November 1996. It had various components: an ISP; on-line versions of reports and books; and the forums, which started in late 1996.
These forums contained areas devoted to the consumer related topics (to be expected) and also an area - Armadillo - for conversations of general interest, serious or frivolous. One important factor was the extent to which subscribers contributed in ways that might ordinarily have been expected of the providers. Possibly half the expertise on the forums came from subscribers, and there was considerable goodwill, apparently on both sides, and a great willingness on the part of the Consumers' Association staff and management to share information.
This sharing is crucial. Obviously the internet can be used as a medium to present Customer Advice on web pages and to communicate by e-mail, but in this it is not distinct from other media.
Forums, however, enable a form of open dialogue that is hard to achieve by traditional means. And this open interaction is very much in tune with the vision of the founder of the CA (Michael Young, Lord Young of Dartington) as a co-operative and collective endeavour.
The forums used the WebX software, and ran satisfactorily for some time. But a series of flaws began to emerge. The search function became less reliable; unpredictable delays occurred in completing the load of pages; and the function of "subscribing" and "unsubscribing" to areas and forums failed for some people in unaccountable and unpredictable ways. There are numerous possible causes for this. Some have attributed it to less skilled hands overseeing the operation, others - less generously - to a deliberate policy on the part of the top management.
Rumours and Forums
As time went on, there were continuing attempts to review and improve the forums. They ran well and effectively from their inception until the millennium. Towards the end of 2001 the first mention of a replacement for the WebX software appeared in the "weekly diary" of Paul Kitchen (then the manager of Which Online). It should be noted that this diary was often the most informative source available. Sadly it was later discontinued.
Nothing came of this rumour, but in December 2002 another topic was created by John Winter (who turned out to be a Which Online employee) to seek feedback for changes to the forum system.
There was a great deal of debate in this topic. However, the effect of its introduction was considerably diluted by a sudden change in the general visual style of the existing forum, which not only attracted a lot of criticism, but which was not, in fact,the proposed major change to the forum. That change, however, served to deflect criticism from the more significant changes that were starting to take shape.
Even so, there were many suggestions, and it began to emerge that a completely new forum system was to be developed. It also became clear that the amount of feedback required from the subscribers was very limited, and also that there would not be much reporting of progress. Indeed the process of development was conducted in total secrecy.
It can be assumed that at some point Sift were chosen and engaged to supply the software. We have been told that a specification for the forum software was written; this specification was and remains secret. There is also some evidence that a few Which Online subscribers were consulted (again in secret), but that many recommendations were not adopted.
A Trial is Born
In November 2003 a trial version of the new Which Online forum system, christened Newwol by the forum users, was launched. It was intended to become the new Which Online forum system within a few weeks. It was accompanied by a PDF document which was purported to be a user's guide, but which was actually mostly taken up with tables intended to show the superiority of the software over WebX and propounding a (flawed) concept of "community". Incredibly, for an organisation specialising in appraising the way big companies deal with their customers, the WOL staff seemed blissfully unaware of the numerous faux pas they were making which included dictating what the new forum would be like, refusing to consider new suggestions, behaving aggressively and abusively towards the users (their paying members) and seeking to foist onto the current group of users a product which was neither suitable for many reasons nor desirable, especially given that the existing forum software worked well if maintained properly.
Reactions and Protests
It was immediately obvious that the system was not ready, and the date for its full launch was steadily put back. There was a great deal of unhappiness at the quality and function of Newwol, coupled with a feeling that the disappearance of the existing forums - Oldwol - would not be justified by this as a replacement. It is fair to say that the goodwill built up since 1997 had, by this time, largely evaporated.
Towards the end of February 2004 it was announced that the new system was about to become the only system. There were considerable protests. Letters and e-mails were sent to Kim Lavely (then Acting Director of the Consumers' Association) and to members of the Council of the CA.
Intervention
Kim Lavely intervened to delay the launch, saying:
"As promised, I've thoroughly reviewed the forum situation, by reading postings, using the trial site, talking to some staff, talking to some WOL users, and generally reviewing the evidence we have about forums. I really do appreciate the strength of feeling here and hope that my personal intervention shows how seriously your comments are being taken.
I have evaluated the current status of the Sift-based forum, and I have agreed with Paul that we will refrain from launching this until many of the problems identified have been addressed. There is no fixed timescale on this, but it is likely to be measured in weeks. I am not promising that the new forums will be perfect when we launch. We may well launch knowing that there are still outstanding issues, and it might be more sensible to deal with these with the system up and running. But it will be improved before we go live.
Post-launch, things will be monitored very closely to identify any latent problems. I'd like to stress that these new forums are an ongoing development, not an end in themselves, and that we'll continue to learn and develop and improve them as time goes by. I'd also like to make it clear that the forums (and their participants) are important to us at CA. But we think the forums should have a lot more active participants and should be closely connected to the thing that makes WOL really special -- great Which? content/information.
We've made quite a leap going to Sift. Having done so, we are now in a position where we can get good quality support when it's needed. Our motives for supplying a system with forum group "message centres" rather than the single catch-all offered in WebX was principally to enable our staff to monitor more easily their own areas of responsibility. This was very difficult in WebX and the evidence from the trial suggests that staff will find it much easier to get around and find the questions they need to answer.
I do appreciate that many of you are actually chipping in with good advice and expertise in areas that perhaps you may not normally have in your list of favourites, and we certainly welcome the helpful expert contributions from all of you. I realise that some of you like to monitor all new postings, and that "message centre" enables this, so with Paul I'm looking again at the decision not to provide this facility. In the meantime, please trust us to try and resolve these issues. I'll be back when I have more news, as will Paul (after his well earned week off)."
Kim Lavely’s post was interesting for several reasons. Her top level intervention suggested a lack of awareness as to what had been taking place. But more tellingly, and something which goes to the root of the internet as an information tool, one of the major complaints about Newwol had been the lack of ‘Message Centre’.
Message Centre was nothing special; it was simply a place where the most recent messages in any topics to which a member had subscribed would be placed, so - at a glance - any member who had chosen to subscribe to Topics a,b and c would instantly see what recent messages had been left in those topics. Interestingly, it’s exactly what almost all social networking sites now include. Kim’s choice of words was interesting: she had said “I'm looking again at the decision not to provide this facility.” which clearly indicates a ‘message centre’ set up was possible, but that the Which intermediate management had conspired to prevent it. Further confirmation had been forthcoming when a request that it be developed had been refused on the pretext of it being too difficult. One user had even reverse-engineered sufficient of the NewWol forum system to compensate by synthesising a script to achieve almost the same result. As soon as this was posted for other users’ benefit, the request came from WOL management to remove it, with the ostensible reason that its presence would derail the trial.
Thus, not only was the introduction of Newwol beset with lies from the outset, the management had attempted to emasculate the forum and its members by preventing the free flow of information.
The new system remained as a trial - and the original (Oldwol) was not suspended. For over three weeks, there was no further communication from Which Online staff about the forum systems, new or old.
On 24th March Kim Lavely posted again:
"Hello again, and sorry for the delay in getting back to you.
Following a useful meeting with Roger (a forum user who had been elected to the Council) we're making further progress with our new community area. We've identified a series of improvements that we want to make before going live. The development work required for these will take around 10 days, so we hope to be able to launch with them in place in a month from now. Most of these improvements are about increasing the number of posts that can be seen, easier ways to post and clarifying navigation issues.
The improvements are intended to address some, though not all, of the points raised in various ways by current users. We would then aim to run the new system for a period of time, say a couple of months, before introducing a second series of improvements based on a combination of user feedback of the area and some of the issues already identified by trialists.
After launch we also want to establish a User Panel to ensure that future developments continue to involve users. We'll have more about that later.
Time now for me to hand back to the Development Team of John, Paul and the people at Sift to make things happen. Rest assured that I'll continue to watch with interest."
Kim's first post was received well; the second one evoked a cynical response which was of some concern. One phrase that Kim Lavely used, that did seem odd, was "User Panel". Why establish a special group of some kind when the very creation of a successful forum system enables an active conversation between users and suppliers? And it's fair to say that the Which Online staff had not given much indication that they were receptive to advice from their users. Indeed there had been occasions where they have seemed actively hostile. Was a "User Panel" a way to limit the number of people whose feedback was to be considered? And would the participants be bound by some agreement of confidentiality so that they might not freely discuss their input?
The End
But, finally, and unexpectedly, it was announced by Malcolm Coles (then editor of Which and of Which Online) that Newwol was to be abandoned. He said:
"Hi everyone. And thank you for your patience while we considered what to do with the proposed new Which? Online forums. Our review was, of course, prompted by the feedback from those of you who've been using them. However, we've also been considering our likely future requirements, which have evolved since the new forums were first specified, and how the new forums would integrate with those.As a result, we have decided that, on balance, the proposed new approach does not do enough to meet your or our needs. Therefore, we've decided to stick with the forums in their current form (ie what you're reading here), for now. We're still sure that we need to integrate content and discussions more. And there are a number of other things we want to get right, such as single log-in access. We might have got there with the new software but, on reflection, and especially in the light of members' feedback, we think it's more appropriate to take a step back while we resolve a number of other technical infrastructure areas. We won't be looking at the forums again until we have made progress on those, probably early next year.
In the meantime, we will continue to support these original forums and look at ways to make them more robust. And when we return to the issue, we will, of course, seek members' views.
As for the new forums, they will be switched off on Monday 10 May (an email will be sent to participants confirming this). All details submitted so far to the new forums, including all posts, will be deleted for data protection reasons.
Thanks again for your patience while we resolved this. And also for the valuable feedback from those of you who took part in the trials."
Newwol became inaccessible on Monday 10th May 2004. No e-mail was sent.
Were we able to see it at the time, the Powers That Be in Which? Online (PTB) had decided to allow the existing form to wither and die.
John Bennett, the ardent chronicler of the forum tribulations, wrote
“My suspicion is that the forums will now have a severely limited budget, that little change will happen, and that some of the real outstanding problems with Oldwol will not be addressed in a hurry.”
In fact, he was more accurate than he could ever have believed. In what was a stunningly deceitful approach to their membership, and a thoroughly two-faced approach to the forum users, little by little changes began to occur which all but guaranteed the demise of one of the most forward-thinking and ambitious experiments of any institution. At the same time it became clear that the management of WOL were barely stopping short of committing outright fraud on their paying members. For many the forum had become a valuable and almost indispensable aspect of the WOL offering.
When they changed the mere style of the old forums, there was a lot of complaint from experienced users. The complaints were met with flat denials, which, to say the least, annoyed people who were supportive of Which Online. The substance of the complaints was never addressed, or in some cases met belatedly and without enthusiasm on the part of the Which Online team.
And when it came to the development of a replacement for the existing system (by convention the terms "Oldwol" and "Newwol" had become current) it seems that the actual users should have been consulted and provided with information at every stage, and that to fail to do so for a radical change of the forum system was inviting disaster. The long silences and the blank refusal to answer some questions were, to put it charitably, puzzling. In a few unhappy cases paying subscribers were addressed by employees of the Consumers' Association in ways that would be rude and contemptuous in any context.
But even in less blatant incidents there was often a dismissive tone.
John Bennett wrote:
"Interactions where one side reads like managers addressing employees are unlikely to appeal if the other side expects to hear suppliers addressing customers."
There is firm evidence that people consulted about the development of Newwol were ignored, and that they were denied the opportunity to express their reservations on the grounds of an imposed confidentiality.
When Paul Kitchen called for called for "active feedback and consultation about the development of the forum system" his reply to the feedbackreply was interesting:
We took feedback from W?OL forum regulars, customer services and others. It has not been a process that we have rushed into since we want to make the new online community truly excellent. I have given you feedback this morning, but this is not finalised. The danger of me telling you what will happen is that it might not happen. I am reluctant to tell you too much before we know whether the service can be provided to the desired requirements.
We invited feedback from different areas, we have taken note of all the feedback and tried to develop an idea that is the best for W?OL. Needless to say, given the different and conflicting ideas that we received on some issues, it is inevitable that some people will be pleased with some aspects of the new system, and some will not. That's a fact of life. I do not think that a big debate on the intricacies of a new system would be helpful at this point."
What was interesting about the tone of the response was the line
“given the different and conflicting ideas that we received on some issues, it is inevitable that some people will be pleased with some aspects of the new system, and some will not. That's a fact of life.”
But there were few points of concern that did not involve either the deliberate stifling of information flow or an excessive control apparently deemed necessary by the Which? team.
Towards the end of the existence of Newwol there was a sustained and significant silence, broken only by the announcement of the end of the experiment. Shortly thereafter, Which Online established the policy of communications being entirely the responsibility of the Customer Services group, thus effectively denying the original aspiration for the forum and curtailing any involvement by the non-customer services staff.
The original invitation "The power of the Which? Online community lies with its members. You have the knowledge and experience from which others can benefit.” now rang laughingly hollow, and any pretensions the once proud Consumers’ Association had had to want its membership to become involved in its ventures was now abandoned.
Community
It is certainly true that the proportion of subscribers using the Which Online forums was relatively small, although the number of subscribers logged in but failed to post was never published. But the management took things a step further. To hasten the demise of the forum they made the entrance almost impossible to find for new subscribers.
In effect they denied the rights of their subscribers to access valuable material.
It was claimed that Newwol was structured in a way that would increase the proportion of psters to 'lurkers'. Much was made of the idea of "community". It is both sad and ironic that Oldwol had a discernible community, largely generated by transient material, but also by forums like a writing competition, which contains both transient and FAQ material (and so was parallel to formal discussion). The question that might be asked now is to what extent have the CA looked for, are looking for, any current research on the phenomenon of on-line communities?
Since the closure of the forum, the Which? team have experimented with pale shades of ideas used in other places. Blogs are penned - sometimes by WOL staffers, others by apparently unknown people, often penning their thoughts, however ill-thought out, and subscribers are invited to add ‘comments’, much like most newspapers. Interestingly, these comments are often appended to reports on 'Best Buys' and the discrepancies between what the Which? researchers find and what the consumers find is often stark. In a recent Best Buy article, which found a washer-dryer to be 'outstanding' and made the reccomendation to 'buy', out of the 23 comments, 10 scored the machine with only one star out of five, which suggests something is wrong in the way Which? is doing its research.
Other ideas have included little ‘think tanks’ where people can pen thoughts and ideas, but which then inexplicably vanish. They also experimented with a focus-group enterprise, where a group was assembled and asked to give feedback on ideas and propositions, but these all bear the signs of similar strategies. They are about control, first and foremost, and any hint or suggestion that members can now deal directly with the senior managers is gone. More worryingly, that are all about restricting the flow of information. Why would that be? And what a great shame that the vision of Lord Young and the efforts of Kim Lavely and Sheila McKechnie should be so shabbily regarded by the present incumbent.
Home - Who we are - Things you might not know about Which? - the Forum - The Great Experiment - A history of the CA forum - Information Control - The present - Links