Proprietary Information & Lack of Openness in Industrial/Organizational Psychology

Openness Movements
,
#1

Hi all,

I work in the field of I/O psychology, where it´s customary for consultancy firms to sell solutions supposedly based on research. The best example are psychometric tests. Firms will publish on their websites the technical manuals or whitepapers for their tests, which include the “research” supporting their products. Unfortunately, often times these do not include basic things such as sampling method. And even in cases where they do include information, it is often incomplete or questionable (i.e. full test statistics, power analysis, effect size reporting, etc.). These tests are then used for actual scientific research, from dissertations to peer-reviewed journals. I think this is very problematic, and am actually thinking of exploring this systematically. Does anyone else find this in their fields? Is anything being done about it?

I´ll appreciate everyone´s thoughts.

Best Fernando

1 Like
#2

There’s a similar problem in engineering research. Organizations, public and private, have technical reports or white papers which are cited in the broader literature despite the reports not being readily available, the reports omitting critical details, and/or the quality of the studies being questionable. Documents called “white papers” are far more likely to be questionable in my experience and probably serve more to get consulting clients or other external funding than actually advance understanding.

Sometimes technical reports are submitted to journals, often in in abbreviated form. But they aren’t always accepted. And in that case, even if the technical report is good, it can be very difficult to locate.

The issue of “grey literature” is related: https://en.wikipedia.org/wiki/Grey_literature

I don’t know what’s being done to increase the quality of grey literature in general. There are some efforts to index grey literature more effectively, but none are particularly successful in my view. Indexing grey literature is very hard, given its atomized nature.

1 Like
#3

the https://curatescience.org/app/ initiative by etienne lebel could contribute a solution to these problems… :slight_smile:

#4

This is interesting. I have cited technical data-sheets in papers before and it’s quite possible they won’t remain accessible after the products are discontinued. Perhaps it would be good to have some kind of greyXriv where people could both post the current copy of grey literature (TDS, white papers, etc.) they cite (so it can get DOI) and also flag missing technical literature for others to upload. Although I guess there could be some legal issues attached to this as I expect that while a lot of grey literature is freely available, it’s probably also proprietary.

1 Like
#5

The access issue is the most critical problem with grey literature in my view.

I doubt most existing companies would mind if you archive their technical data sheets. Technical reports on the other hand, they probably would mind, particularly given that some companies sell copies of their own technical reports.

For documents from entities which no longer exist, I think one could make an orphan work archive to handle these in general. Your chance of getting sued is fairly low if the entity which owned the document no longer exists and has no successor.

If the report was publicly funded and/or produced by a public entity (even if it no longer exists) then you might have some luck getting the successor or responsible organization to put a copy online. I was able to put this report online because I found a niche archive that apparently has the rights to the reports in this series now. I’ve done similar things for government entities.

For reports which are currently online but may not be online in the future, I’ve used Perma.cc to archive them.

With permission I’ve also uploaded an unpublished Master’s thesis to the Internet Archive. Unfortunately this appears to be poorly indexed by Google; it doesn’t even appear in Google Scholar.

Certain libraries also hold large amounts of older grey literature. I’m a particular fan of the Library of Congress’ Technical Reports and Standards unit. The main problem with libraries and archives for grey literature, however, is that you typically don’t know if they have a particular report until you visit or ask them.

Also related are translations, a fairly unusual form of grey literature. I wrote a long Stack Exchange post about how to find translations. It would be really nice to have a website where you can enter a DOI and find out immediately if there’s a translation. Unfortunately translations went out of style at the end of the Cold War, and as far as I’m aware the largest citation database for translations no longer exists.

2 Likes
#6

recreating academic/scientific translation database sounds like an awesome idea… :slight_smile:

2 Likes
#7

Thanks all, it´s unfortunate to see this is an issue elsewhere. Hopefully I can address this issue at some point, in which case I´ll gladly share here. Best Fernando

1 Like