How can we achieve a fully open future?

Openness Movements
,
1

I have a new article out, which I have also submitted as an editorial. Would love any feedback on this while it goes through review. The key idea revolves around research funders setting research institutes a new type of mandate for open science,

Source: https://osf.io/preprints/socarxiv/9kjwp/

Abstract: It is time for a new type of mandate. Plan S has catalysed all sorts of action, and confusion, in the world of scholarly publishing. But it lacks teeth. Instead of encouraging libraries and research institutes to continue to prop up a dysfunctional and out-dated system with taxpayer money, research funders should mandate institutes to create a fully open, modern, technical scholarly infrastructure. This would help to overcome so much of the inertia behind the adoption of open research practices, while simultaneously resolving outstanding issues with reliability, affordability, and functionality in scholarly communication.

1 Like
2

i skimmed through the article
 quick feedback
 as discussed with @Gavin at another thread, the term ‘taxpayer money’ should be replaced with ‘public money’


i also note this sentence in the article: “With so much public money exchanging hands, why is no public tender being put in place for the publishers to compete fairly as service providers?”

would try to reply more later


ps: this quote happens to be on my screen and i would try to explain more later its relevance to this paper
 :slight_smile:

"“Those who create and issue money and credit direct the policies of government and hold in the hollow of their hands the destiny of the people.” The words, attributed to a 20th century British banker, capture an emerging consensus. Money, governance, and public welfare are intimately connected in the modern world. More particularly, the way political communities make money and allocate credit is an essential element of governance. It critically shapes economic processes – channeling liquidity, fueling productivity, and influencing distribution. At the same time, those decisions about money and credit define key political structures, locating in particular hands the authority to mobilize resources, determining access to funds, and delegating power and privileges to private actors and organizations.

Recognizing money and credit as public projects exposes issues of democratic purpose and possibility. In a novel focus, this conference makes those issues central. Scholars, policy makers, and students have often assumed that money and credit emerge from private exchange and entrepreneurial activity. Recent work, by contrast, emphasizes that modern currencies depend on collective orchestration. That approach resets the frame."

source: https://lpeblogorg.files.wordpress.com/2018/10/money-as-a-democratic-medium.pdf

2 Likes
3

Epic quote, terima kasih, Surya! And thank you for your comments too, as always I look forward to hearing more feedback from you :slight_smile:

4

Rather than splitting this into multiple threads, I just had two more preprints uploaded too.

The first is about creating a value-proposition for Open Science: https://osf.io/preprints/socarxiv/k9qhv/ and is related to @rebecca’s recent post too on the different frames within ‘open science’.

Abstract: Open Science has become commonly understood in terms of its practices. Open Access, Open Data, and Open Source software are all becoming commonplace in academia. However, unlike the Free and Open Source Software movement, Open Science seems to have become largely divorced from its pluralistic philosophical and ethical foundations, which seem to have reignited from the humanities at the turn of the Millennium. To close this gap, I propose a new value-based proposition for Open Science, that is akin to the “four fundamental freedoms” of Richard Stallman that catalysed the Free Software movement. In doing so, I hope to provide a more common, unified, and human understanding to notions of openness in science.

The second is about the exploitation of free academic labour during peer review, and was inspired by @surya. https://osf.io/preprints/socarxiv/6quxg

Abstract: Commercial publishing houses continue to make unbounded profits while exploiting the free labour of researchers through peer review. If publishers are to be compensated financially for the value that they add within a capitalist system, then all others who add value should be similarly, including reviewers. I propose that peer review should be included as a professional service by research institutes in their contracts with commercial publishers. This would help to recognise the value of peer review, and begin to shape it into a functional form of quality control.

What, did you all think I was going to just sit back and let OpenCon take my life from me? :wink:

3 Likes
5

Very interesting pieces @jon_tennant, thank you for sharing!

I was reflecting upon the unpaid reviews issue once more in the last days, as I am reviewing a manuscript. It seems that, when you are under a stable contract at university or other institutes, doing reviews is considered as part of your job, even though you happen to work for another employer and without getting paid. It is completely absurd but I guess the reasons that kept this practice alive is that reviewing is considered something you do for your peers and to maintain science integrity, how could you get paid for it!!?? Also, I guess it worked better when researchers had fixed contracts and did not have to struggle with funds applications every year.

Personally, doing review now that I work independetly helps me to highlight the true nature of this work. It is just free work! And quite a long work if you want to do it correctly (which many researchers do not do. Another big issue that should be tackled). I am still doing it to help my peers and protect science rigour, but why should a private company profit from it??

2 Likes
6

@Enrico.Fucci: Right on! You hit the nail on its head.

More dire is the tradition of free reviewing, which i think proliferated in a much less neoliberal time at post-colonializing countries, have now being exported to post-colonialized countries.

So it’s a double whammy. Not only almost all scientists and scholars in post-colonialized countries do not get the compensation that they deserve for their paid job, they also receive no compensation for their unpaid job.

Given this condition, i further suggested to jon that scientists and scholars all over the world now need to demand retroactive compensation for all their reviewing job for for-profit publishers.

I am extremely sick of the argument that ‘publishers add value to the publishing process hence they deserve their profit’.

Well, if publisher deserve value, what more the writer, reviewer, editor, and everyone else involved in the publication process? Would there be any value without their work?

I think this is the sucker punch that needs to be delivered to the for-profit publishing industry. Like other multi-national corporations skilled at externalizing their cost but internalizing their profits, they need to be knocked out of their complacency and embrace a fairer way of doing business.

2 Likes
7

For the sake of balance I asked Björn Brembs to comment here, as he made some good remarks on Twitter and I can’t be doing much with that platform for nuanced discussions like this. https://twitter.com/brembs/status/1236953115695480832?s=20

It seems though that his argument is based on the concept that academics are already paid salaries so should not be paid again in any for their work as reviewers. This is discussed somewhat in the original preprint.

1 Like
8

To be very practical here: I am right now reviewing a manuscript. It will take me about 5 hours to do it properly. I do not perceive any salary and work as freelancer. I am doing this work completely pro bono. The result of my work will be used by a for-profit company. How can that sound right?

Here are two alternatives:

  • I work pro bono and the result of my work 1) do not lead to profit 2) Is openly available 3) Is recognized somehow (this last point is not very important)

  • I get paid for my work (also as a way to ensure that I can dedicate to it the right amount of time and efforts)

I am eager to listen to any argument against these points :slight_smile:

2 Likes
9

I’m not sure what you are freelancing as. In general, everybody in science is welcome and every idea counts. However, one needs to keep in mind systemic effects. If everyone in science were a freelancer, paid by, say, paper output, everyone would just go and publish as much as possible,. regardless of content. So while freelancing may work at a very low level, systemic freelancing would be the death of science, IMHO.

At this low level, freelancers should not work for free in principle. All that being said, if the freelancer is a billionaire retiree, doing science for fun because they don’t know what to do with their time, they should not take any money at all :slight_smile:

1 Like
10

Most of my arguments around peer-review being somehow “free” are laid out in this post: http://bjoern.brembs.net/2016/07/peer-review-is-not-free-its-a-subsidy-for-publishers From my perspective in the life-sciences, nearly 100% of reviewers earn a salary and reviewing is what one does if one wants to have one’s own papers reviewed: you can’t go around publishing in peer-reviewed journals and don’t contribute. That’s anti-social behavior. So as long as you publish in these venues, it goes without saying that you also review. If publishing is in your job description, so is reviewing, at least by extension. For my part, I review, because I consider it to be part of my job and I’d be anti-social if I wouldn’t do it.

So from that perspective, peer-review is not free. It’s paid labor, only that others are making a profit off of it. This is nothing new: tax payers pay public institutions, which in turn pay libraries to pay subscriptions or APCs: public money is converted into private profits along the entire pathway of science. There is no need to emphasize the review part: it’s just more of the same.

If, however, now something changes and not reviewing is not anti-social any more, perhaps because someone thinks it ought to be paid extra, then of course, there is little reason for me to keep reviewing: after all, someone is being paid for it (usually someone more in need of the moeny than me!) and it’s not anti-social any more to not review: no money - no work clearly isn’t anti-social, on the contrary: any review that I do, means someone more in need will have less food on the table. So by not reviewing, I’d even be helping the precariat out!

So, if reviewing gets paid extra and people think logically, everyone with a professorship stops reviewing and uses the time to write papers or do experiments or write grants, while everybody who can use a few extra bucks, e.g., ECRs takes time away from paper writing, paper reading and experimenting to review.

I think I do not have to elaborate any further that this is not where any sane person would want science to go, but give me one more scenario: in the US, scientists already receive only 9 months salary, as the university think they can get the remaining 3 months from grants. If reviews earn you money, why shouldn’t the universities decide to cut another three months for which you can do reviews?

So chances are, more likely than not, try to somehow compensate for peer-review and all kinds of detrimental effects start to happen.

There are some hilarious solutions such as taxing publishers for every word they get in subsidized reviews, but the best solution remains: we need to get a modern infrastructure where we control the amount of profit being made by actual competition. Short of that, everything else will just be a stop-gap or a patchy band-aid, and not a solution.

2 Likes
11

You touch on peer review as a form of quality control for science, but you don’t mention quality control for peer review itself. Copying in a comment I made on EA Forum that seems relevant here as well:

One of the big drawbacks of peer review is the hugely variable quality of reviews that are provided. As an example simply in terms of the level of detail provided, I have had comments of one paragraph and three pages for the same article.

I think a key reason for this is there isn’t really any standardized format or expectations for reviews nor is there much training or feedback for reviewers. One thought I’ve had is that paying peer-reviewers would allow journals to both enforce review consistency and quality - although publishers have such large profit margins that it this could be feasible, they have no incentive to do so as scientists accept the status quo. In the absence of paid peer-review, I think that disclosing reviewer names and comments helps prevent ‘niche guarding’ and encourage reviewers to provide a useful and honest review (eLife does this currently, not sure if any other journals do so).

Note that while I suggested paying peer-reviewers in that comment as a way to enforce high quality reviewing standards, this could also be done via contracts to the research institutes that employee the researchers/reviewers as well, so it’s compatible with the model proposed by @jon_tennant .

2 Likes
12

Welcome to the IGDORE form Björn @brembs!

With regards to freelancing - @Enrico.Fucci and I both do ‘academic freelancing’ where we essentially work for a daily rate on short term post-doc projects. In this case, choosing to do a review probably reduces the time you have available for consulting work and thus reduces your income (in most cases it will be unreasonable to charge the University you are currently contracting to for time you spend reviewing). However, most researchers work in longer-term and salaried positions, so I don’t think our freelancing cases are very representative of the general relationship between scientists, universities, and publishers. Our situation is probably more similar to researchers who have left academia for industry being asked to do reviews for a while after making the move.

While I appreciate that other reviewers have previously taken the time to review my articles, I currently refer review requests to my co-authors who are still in salaried academic positions. However, I am also starting to do independent research, and when I start publishing the results of that in peer-reviewed journals I would then be more open to accepting peer-review requests made in relation to that work, as I agree it’s important to give back to the peer-review community if you are taking reviews from it.

2 Likes
13

Thanks for these comments, Björn. I’ll try and respond to the main ones here.

Isn’t this the main problem though? That academic labour is constantly out-sourced not only for free, but often at a cost, to commercial third parties? The article could have focused on the whole chain of value production, but I wanted to just focus on peer review as that is the part I know the best.

Just again to clarify, at no point does the article say reviewers should be paid. Rather, it makes the argument that their time should be appropriately compensated, and can be tied into existing contracts between institutes and publishers. I see no reason why this can’t be extended to authoring of articles too, as well as other labour which is currently exploited by publishers.

Again, the crux of the argument is that if publishers add value and deserve to be reimbursed for those services, then so do all others who add value.

Universities should not cut grants, and again that isn’t the argument proposed. Indeed, if institutes started charging publishers comensurately for their services they already provide for free, then if anything there would be more money in the system.

2 Likes
14

@Gavin more to come on quality control and peer review soon - article is currently in the proofing stage and goes into much more detail. Will post here soon. Preprint is here for now: https://osf.io/preprints/socarxiv/jq623/

One way to enforce standards for peer review might also be to, you know, make it into a standard. Eg my latest proposal for Palaeontology https://paleorxiv.org/qzycs/ - but this is more at the journal level than the institutional level.

The part of the preprint that also discusses this comment:

Institutes pay researchers to do research, but many researchers are independent or are salaried via other means. Peer review takes away time from research, and the value is capitalised on by a third-party at the expense of an institute. I doubt that any institute would be particularly happy in knowing the total amount of unfinanced labour their staff give out to for-profit entities each year. In job adverts and descriptions for researchers, it very rarely says anything along the lines of “perform professional services for third-party commercial entities without any form of compensation.” Because of this, peer review work is often done outside of professional hours as a form of unpaid overtime. This is an increased burden on an already over-worked community. If reviewers stopped reviewing, their salaries would remain exactly the same, and they would get more personal work done for their employers and funders. Reviewers already get ‘rewarded’ in a way as they are able to list their reviewer activities on their CV, or on platforms such as Publons. However, performing a professional service for the potential perception of prestige or privilege is akin to the ‘do it for the publicity’ arguments that plague the art and media industries.

Peer review is not dependent on the publishing industry. If the private sector did not exist, peer review would still function through existing alternatives. Yet, the entire publishing sector is utterly dependent on peer review to function. This represents an incredible asymmetry in power, and yet the value flow is on the opposite direction. If all researchers decided in synchrony to stop peer reviewing, the entire publishing industry would collapse. Or at least, the substantial revenues and profits that the private publishing sector make could be greatly reduced. This is a collective action problem, but such a simple notion indicates precisely where the core value in the process should lie.

3 Likes
15

@Gavin already explained what kind of “freelancer” I am (maybe freelancer is not the right term, but neither independent researcher, given that many scientists who have a permanent position in university describe themselves as “independent”). I am curious though to know more about the view you expressed here. Why freelancing would be the end of science?

It is interesting that you give “being paid by publications” as an example. Isn’t it the case right now? Many institutions reward publications with actual monetary compensations, a big number of them requires researchers to publish a certain amount of papers and basically all of them evaluate researchers based on the publication outputs (not to talk about funding bodies).

16

Just a quick note on that, China did recently ban those cash rewards https://www.nature.com/articles/d41586-020-00574-8

3 Likes
17

The main socioeconomic driver behind the replicability crisis seems to be pressure to publish, see, e.g., the situation in Australian universities: https://www.fwc.gov.au/documents/decisionssigned/html/2020fwc416.htm No papers, no job.

So far, this situation is very heterogeneous between countries, between institutions and between fields. What some of us are working towards is to decrease these incentives, to get rid of them. Increasing them, by making more researchers dependent on their output, arguably stands to make the crisis worse.

Why does such pressure increase the unreliability of science? Two main factors among likely several:

  1. If the number of publications is important, then the content isn’t and hence you, e.g., cut corners and decrease sample size to get more experiments/studies per unit time: https://royalsocietypublishing.org/doi/full/10.1098/rsos.160384
  2. If the venue counts, then people will try to publish to-good-to-be-true results in journals that look for such results. The consequence is, that many of such journals publish even less reliable research than other journals: https://www.frontiersin.org/articles/10.3389/fnhum.2018.00037/full

Finally, If politicians who already like to see science funding slashed realize that the funding largely goes to unreliable stuff, they will feel validated and reinforced in cutting funding for science:

So the more unreliable our research gets because we reward unreliability, the more ammunition for the anti-science movement.

Of course this doesn’t mean all ‘freelancers’ publish unreliable science! But what it does mean is that being rewarded for publishing and not for the reliability of the content, means that a larger fraction will focus more on publishing than on the reliability of the outcome: after all, that’s what’s rewarded!

18

Thank you @brembs for the clarification. It seems to me that you pair the term “freelancer” with “getting paid for outputs”, which is fair enough, considering that this is how “freelance” jobs are often described. That is why I doubt the term “freelance” best describes researchers like myself who decided to step out of traditional academia (@gavin explained it better above). But let’s even imagine that scientists work as freelancers in the traditional acception of the term. Why should they be paid in exchange of publications? Why not thinking of another type of “output” that is the object of monetary compensation? And if there is another type of output (let’s say, for instance, “replicable empirical evidence”), why bein a freelancer would be detrimental for science?

In my view, going independent (in the true sense of the term) was a way to tackle the replicability crisis. A way to not being submerged by those issues that affect traditional academia and often are the source of bad science.

So, we might simply have different views on what a “freelance” sicentist could look like and what they could achieve in terms of contributing to the progress of scientific research. But this probably deserves a separate dicussion.

Regarding peer-reviews, I am happy to do it as a prosocial act, free of charge. Again though, why should a private corporation profit from it? And how can we make sure that reviews are respecting high standards?

19

Why should they be paid in exchange of publications? Why not thinking of another type of “output” that is the object of monetary compensation?

Probably because it is not trivial to quantify “reliable science” in a way that is conducive to exact remuneration?

As Einstein said: “not everything that can be counted counts and not everything that counts can be counted” :slight_smile:

In my view, going independent (in the true sense of the term) was a way to tackle the replicability crisis.

I can understand why people feel compelled to do this. Largely because I have no answer to the question above, my answer is to instead make scientists independent by giving them job security and baseline funding.

Again though, why should a private corporation profit from it?

There is no principle problem with having companies profit from our work, as long as there is a functioning market: if there is demand, there will be profit, without demand, no profit. What I find the much more difficult to answer question is: collectively, we have all the money, we are the customers and we have the choice of how to let companies engage with us. And yet, we let them pull one fast one after another on as and are throwing money out of the window as if it was burning holes in our pockets. I understand the historical reasons for this irrational state, i.e., how it came to be that way, but why on earth we keep continuing this travesty, I honestly cannot tell you.

And how can we make sure that reviews are respecting high standards?

The same way we make sure our students leave with the best possible education: we teach them. When receive review invitations that falls within the project (one) of my students, I do the review with them.

20

I completely agree that the pressure to publish is one major reason why we have a replicability crisis. However, as I’ve argued in this essay, traditional academia itself is another major reason and that can’t be solved without fundamental changes to how traditional academia works. From my essay:

I fear that the only change we will accomplish within the current system is to raise the stakes; adding pressure to already stressed out researchers by telling them that they now also need to conduct science in new and additional ways.

With “new and additional ways” I referred to open and replicable practices such as preregistration, open data and materials, multi-lab collaborations, etc. I continue:

Raising the stakes at this point, in traditional academia, is not the solution we are looking for. Academics will find shortcuts to meet the new requirements for publication, just like they did with the previous requirements.

The goal of academia has for at least 200 years primarily been to make money to fund expensive and inefficient habits such as superfluous administration; buildings; subscription fees for access to scientific articles; and more recently, to fund proprietary software. I think it is pretty fair to say that academia has become counter-productive to science. The change we are longing for can thus not be found within the existent academia.

So what is the solution?

I believe the solution is pretty straightforward: science would be better off without academia, so why not cut the ties? Many of the great discoveries did not come from the universities, Babbage (1830) pointed out, but from scientists working for themselves. Researchers don’t need academia, we never have. We can organise ourselves in other ways or reinvent academia from scratch: less and more efficient administration; cheaper buildings or no buildings at all; open access (or Sci-Hub) instead of subscriptions; free and open source software; location independence to allow researchers, students, administrative and technical staff to work from wherever they want to or need to be; and completely flexible work hours (researchers should probably not count work hours at all).

Researchers have tried for at least 200 years to change academia and they have all failed. So would we. So let’s not do that. Let’s instead build something new, a new academia. We can create many new ways for scientists to organise themselves.

I think it’s a mistake to think of ‘independent/freelancing’ researchers as being more dependent on their output than researchers within traditional academia. There are many ways to support oneself financially in ‘new academia’: freelance teaching, research consultancy, clinical work, non-research/education related work. Thus it can in many - perhaps even most - cases be less dependence on output in new academia than in traditional academia: a scientific publication doesn’t really do anything for you financially unless you’re in traditional academia where number of publications is important for promotion etc.

1 Like