Interesting when considering how plone is designed
What I've noticed we tend to do is ask ourselves what features we want, at open gardens and other such forums. The post says that even asking actual end users what they want is often not useful and can result in a UX that needs to be changed a lot when user testing happens. In our case, us, core developers/power users/integrators aren't the typical users of plone or rather we only one kind of user, where Plone is a complex product that has to suit both editors and integrators. I'm not sure how many core developers consider themselves integrators. So asking "us", for user research and roadmaps doesn't seem like a good idea.
While I agree with you that Plone is often designed with little to none validation from the end users, the article itself is very approximative and misleading regarding the value of user research in general, which wrongly seems to lie only in testing how well a prototype works. There's a lot more, both before and after the user testing, that can be done to know more about who the users are (their needs and expectations) and how they will use Plone.
I agree with the bottomline that user research is overrated. See the Plone toolbar...as far as I know it's design and functionality is based on user research. I could not find any person so far that find the Plone toolbar usable or an improvement over the green bar in Plone 4.
I'm going to put on my "not so cheery" hat here... this sort of endless navel gazing just drains everyone's energy. At some point, it just becomes impossible to pay attention to the arguments, and it becomes much more interesting to just do something. More code/themes/documentating/training, less talk!
My point in posting this is not to open the floor for certain people to point out UX problems and their favoured solutions. I'm sick of the constant stop energy too.
TL;DR summary: Code less, talk more, but talk from actual users or those who train actual users. Then lets work out how to get that data to result in code.
My point is this
We don't really do market research. The people who code it mostly ask ourselves and we aren't representative users. We don't know what its like to approach a toolbar or default page concept for the first time
Even if we did market research it would tell us limited information. It wouldn't tell us how improve things. Anyone who has a consultancy knows, building exactly what your customer is asking for is often a way to build something that doesn't solve their problem. But it can tell us actual problems users are having. We need to identify who in the community regularly trains users. This has proved to the most valuable information about what users find hard. We need to ignore everyone else opinions.
We need a way to turn "X is a problem for new users" into motivation to code. This is the hardest part. We shouldn't "just code more". First we need to prove its a real problem and right off the bat to code developers. Experience coders have different problems so don't making api more pythonic becomes higher priority than say "no quick way to preview a change before you save it". And because we are all volunteers, we scratch our own itch. The motivations for companies to pay to make plone easier are also less because they get paid to customise and train, a zero training, zero customisation CMS is more a less money. Instead you need to look at the long game, where simpler, easier CMSs take our market share by being slowly more secure, more flexible while still being simple. Then there are no small or big jobs. We wasted our investment in Plone. So don't trust your instincts. Think long term.
We need to test our designs. Low fidelity prototypes. Also really hard to find the right users to do this with. But some things are being tested via plugins. CastleCMS tests several ideas. Mosaic is being used in production with non technical users. How can we get the feedback from these users back here so everyone is aware of it?
Simplification is in the eye of the beholder and obviously a highl subjective matter. Basic facts that the toolbar was hacked once and did not see any further love since then:
the word break issue remains unsolved. A term broken into two lines based on remaining space instead of proper hyphenation is an issue. It is not a sign of a professional CMS. Open issue since more than 18 month. You can not show such a toolbar with a broken typography to a professional editor..this hurts! Yes, it is easy to blame the browsers here, in particular Chrome...but calling the toolbar usable here is just another symptom of tunnel view.
it is still not possible to place the toolbar individually. The position is configured per site and can not be changed per user.
The state of the toolbar is like this: yes,we want to make it right and do user research first, then you implemented something, you got stuck, somehow brought the toolbar into some state...now it is here and remains as it is until the end of time.
Herein lies the problem. Your opinion is typography makes the toolbar unusable, but the proof is where? I'm not saying you are wrong, but how do we prioritise this issue vs others? The only way we can solve this is data from real training courses or user testing which shows users found it hard to learn/use because of typography.
Of course, nothing stops you fixing these things yourself which bypasses that process and puts it right at the top of the priority list. You are a good developer. The two issues you talk about don't sound nearly has hard as some of the bigger issues with the toolbar like not being able to rearrange it into a more logical order, or that there are too many items on the toolbar. Not all fixes require cooperation.
The problem lies in the misunderstanding that user research leads automatically to better usability and better results. There is the huge black hole in between called "implementation". And I want to emphasize that even such small issues are important. Fortunately a bunch of the toolbar issues have been fixed -they have been been fixed after I reported them short before the 5.0 release..many smaller and bigger issues that were so obvious that one must have spotted these by yourself. So back to the original posting: user research is overrated? Yes because it is the result (the implementation) that counts as benchmark. But calling other persons opinion BS is perhaps a bit over the top - well, not taking it personally...it is giving and taking
<rant>
Instead of complaining, fixing in own code and writing lengthy articles - what about fixing upstream and creating pull requests and if necessary PLIPs together with its implementation? </rant>
The whole thing is a bit as if my 15yo must tidy up his room: He complains and discusses for hours, starts to do different work, but at the end, the core work has to be done.
Whom are you referring to by "we"? Nathan and I and others at Wildcard work with "real users" and content editors and site admins and we train them so we see directly any pain points they encounter. We feed that knowledge back into Castle and Plone, either via PRs or issues. We don't do this in a vacuum.
I know you mean well and you contribute a lot to Plone, so when I call BS it is only on that particular thing I quoted, not on you at all. And as I know you're more than capable of dishing out BS calls, you'd also better be able to take them
If a problem is a big deal to you and you have ideas on how to fix it, make the PR! Or cajole/voluntell/pay someone to make the PR...
< plug >
When I first joined Wildcard I told them I'd be able to get the company some paid work with Europeans... so bring it on
< /plug >
Everyone I've shown the new toolbar loves it, and it's clear it simplifies editors' view of their content by not cluttering it up.
