On a very cold afternoon in Berlin I managed to ask a few questions to Andrew Preston (co-founder of Publons) who was in town for the Academic Publishing Europe 2014 conference:
- Could you tell us a bit more about the platform you are building - publons.com. Where the idea came from? What kind of problems are you trying to solve?
- Absolutely! I did a PhD in physics as well as a postdoc in the US, so I have experience the enjoyment and the trial of publishing papers first hand and I also have been a reviewer. By going through that process as well as talking to colleagues I realized that there must be a better way of doing this. That is the genesis of Publons.
- What actually is a Publon?
Good question! It is a joke in physics where there is a concept of fundamental particles: the electron is the fundamental unit of charge, the photon is the quantum of light etc. The Publon is the facetious term we use to refer to the minimum publishable unit of academic research. We thought it was a cool name for a business. It also speaks to some of the approaches in academic publishing that prevail in the system.
- So back to the concept...
The idea of what we are doing is, right now if you are an academic the single most important thing that you can do for your career is to build your resume by publishing peer reviewed journal articles. So reviewers are incredibly important, but what do they get for their work? The answer is nothing right now. There is an imbalance in the system. The authors who need to have their papers peer reviewed, get credit for their work. On the other hand, reviewers, who provide a valuable service to authors and the general public, get nothing for their efforts. So the question behind Publons is: how can we get reviewers credit for their valuable work in a way that becomes a part of their official resume and helps them securing funding and jobs?
The way that we've done that is to build a platform that collects peer review information from all possible sources: whether it is pre publication peer review, post publication peer review, open peer review (where the authors know who the reviewers are), blind peer review (where the authors don’t know who the reviewers are) double blind peer review (where in addition reviewers don’t know who the authors are). You can also choose to publish the content of your peer review or not. Publons as a platform aims to support all these kinds of reviews in various ways and extract information that we can use to build a reviewers resume -- to make them look good.
- That sounds very exciting, but how does it work in practice? I can imagine that when someone wants to comment on or review an existing paper and make it publicly available that is the easiest use case for you.
Yes we support that. If you go to publons.com you can import any paper with a DOI, we will bring the metadata on to publons. Then you can write a post publication peer review. We actually support the ability to do that anonymously or to do that with your name attached. Which I personally thing is a much better way of doing post publication peer review and we would like to encourage it more.
- You also mention that you are interested in giving reviewers credit for reviewing paper in a standard way - pre publication. How does this work?
- By and large those reviews will be blind unpublished reviews. What we figured out on publons is a way to interface with publishers to collect that information and to expose it in a way that preserves your anonymity, but also proves that you are doing a good job as a reviewer for various journal. That is something we are really excited about.
- So the reviewer gets in touch with you and claims he or she reviewed certain paper and then you get in touch with the publisher?
- Actually it works in both ways. So if you are a reviewer and you have done a pre publication peer review you can for sure upload this information to publons and we will get in touch with the publisher to confirm it and validate it. This way it becomes an official validated part of your resume. We are also starting to work with publishers to pull the peer review information the other way, to automatically populate your profile, should you wish.
- How successful are in talking to the publishers?
- It is still early days, but the conversations that we have had with the publishers are all very promising. So for example we already incorporate some pre publication peer review on our site from places like PeerJ and Biology Direct. We can do that easily because they are open access journals and they provide peer review information under an open license. But if we talk to any publisher they are all very much in favour of making their reviewers look awesome. We are starting a series of pilots now.
- Speaking of making reviewers look awesome, and getting credit for reviews - how is it incorporated into one's CV? How do you imagine people using publons to show their commitment to scientific community?
- The first thing is your reviewer profile on publons where we collect all this information and give you essentially a score as a reviewer, a number that you can go up by doing more reviews and doing better reviews. Obviously as we progress along our development roadmap, we will give you an option to download this as an automatically formatted resume, a document that you can for example show to funding agencies. The other thing that is worth mentioning is that for both collection as well as dissemination of reviews we provide an API. You can come to Publons, sign up for an API key, and do interesting things with this review information. For example we provide peer review information to altmetric.com now, so any paper that has a peer review on our site will get an altmetric boost on altmetric.com because of the peer review. We’ll have some other announcements in this area very soon too.
- I also heard that you are providing DOIs for the reviews. Is that true? Could you tell a bit more about this?
One of the important things about peer review is that it is a contribution in it’s own right to scientific literature. Peer review is not just an evaluation of a paper, but it also sets the context of what’s going on in the paper and enables a dialog. We feel it is good for science to expose as much of that as possible, and to encode it to make it a part of the sphere of human knowledge. One way we figured out to do that is that if there are good and valuable reviews on publons.com we will give them a DOI so they can become citable. The question is how do you decide if a review is worthy enough to get a DOI. We have the concept of endorsement - so if enough members of the community endorse a review we will assign a DOI to it.
- How does it compare to how dialog about papers have been done so far. When there is a controversial paper coming out scientists who don’t agree with the claim in such paper will write a response. Then the original authors will write a response and so on. All these responses are published as papers, usually in the same journal. How do you see this in the context of what publons is doing?
- I see it as exactly the sort of thing we want to support and encourage, but I also that there is a huge body of peer review that is not based around controversy, it is based around not arguments but additions to the published papers. I think it is important to have a platform that supports both of those things. You don’t want to be a platform where people fight about things, you want to be a platform where people collaborate.
- All of that sounds really great, but how much does it cost?
- It’s a good question, because right now as an author if you want to get credit for your research you publish a paper and you will actually pay open access journals. Thousands of dollars for something like that. There is no question that if somebody was able to provide a lot of value to reviewers, making them look good, then that probably would be worth money to reviewers, but that’s definitely not our business model. It’s not the way we aim to make money. The idea is in my mind to be as open as possible. So you want the information to be free and available to make reviewers look awesome. So we don’t charge anything.
- Do the authors have to transfer the copyright of their reviews to you? What license do you use to distribute the reviews?
- It depends on the type of the review we are talking about. If we have a PeerJ review on our site, the PeerJ license is CC-BY. So obviously when we put it on our site we will keep the same license. It is going to vary by publisher, but things that are uploaded by reviewers to publons.com are distributed under a CC-BY license.
- The lack of fees makes me wonder how do you plan to maintain the financial continuity of the project?
- It is really important to have a good idea of how you want to make money, because you want something like Publons to be sustainable and stable. The first thing to say is that peer review is an integral part of academic publishing and something that publishers put a lot of effort into. One problem that publishers find in their peer review process is sometimes it can be tricky to find a reviewer that can be qualified to review a paper and is motivated to do it, but does not have a conflict of interest. With Publons I think we can solve some of those problems with reviews and there are definitely ways to make money.
- Yes it is very hard to find good reviewers. There is an imbalance in terms of incentives: authors are very motivated to publish, but there are only few good reasons to review papers. Some do it to stay up to date with the bleeding edge or try to influence the publishing landscape with their own agenda. Recently Nature published a letter from one of their readers suggesting that journals should require senior authors on submitted papers to prove their activity as reviewers. This would be a necessary condition for the journal to even consider the paper for review. What do you think about this?
- It’s a good idea. The thesis behind what we are doing is that reviewers should get more credit for what they are doing. However, your review record stopping you from publishing seems to be a little bit extreme.
- Publons is not the only platform that is out there doing post publication peer review. How does it compare to PubPeer.com which gain a lot of publicity recently, as well as PubMed Commons which launched few months ago.
- First of all I think both of those services are great. PubPeer is interesting because it is anonymous so it’s more focused on discussing articles behind the veil of secrecy. Publons on the other side is more focused around getting credit for your peer review and assigning it to your profile. So I see what we do as a little bit different at some levels. PubMed Commons is an interesting thing. I am curious how it will develop. It is quite exciting because PubMed has a critical mass of readers and content.
- We see a lot of discontent about the peer review process within the scientific community. How do you see post publication peer review changing the landscape in the short and long term future? Do you think it will replace or complement the traditional pre publication reviews?
- At Publons we see the peer review process developing into something that reviewers get credit for. In that case it does not necessary matter if it’s post publication peer review or pre publication peer review. In all cases the process will be improved. People have a better incentive to do a good job of review and that will speed up science. This is our focus. Rather than forcing people to do a specific type of review, it’s a matter of a empowering them by giving them credit for whatever type of review that is consistent with the culture of their field of study and their peers.
- Do you think post publication peer review could drastically change the landscape of publishing?
- It could at some point in the future, but the more I learn about academic publishing the more I learn how entrenched a lot of behaviours are. I think at some point post publication peer review will take off, but it’s a question where that critical mass will come from.
- You already expressed your opinion about anonymity of peer review. If I understand correctly you are in favour of open non anonymous peer review. Could you explain why do you think this is a better option?
- I am in favour in any kind of peer review. The more peer review the better. On publons.com you can get credit for anonymous and non-anonymous peer review - that is one of the things that is really cool about us. But what I would say is that the blind peer review culture and the idea of anonymity is caused by a fear of somebody being angry at what you say - the mythical somebody. In most cases reviewers do a great job and authors have no problems with the reviews. I think we could open up a lot of the system and it wouldn’t actually drastically affect us and we’d all be better off for it. Anonymity is a great way to hide and attack people for things that could be addressed in a better way. You see that on PubPeer sometimes.
- Do you think part of the success of PubPeer is that it makes making anonymous reviews so easy?
- Yes, potentially yes.
- There are also some problems with anonymous reviews. Some authors find handling a discussion that drags on very hard. Especially when they have to respond to anonymous comments.
- I do think that in any environment, whether it’s academic or on the web, you do want to build a culture that is healthy and good for everybody. I think anonymity isn’t always the best way to build that sort of culture. That said there is always a place for it.
- You also mentioned giving DOIs as a reward for good reviews. We see DOIs being used by different publishing entities, for example figshare, ScienceOpen etc. However, at the moment if something has a DOI it most likely means it was published in a peer review journal - it’s a proxy for quality. Do you think the value of DOIs will diminish?
- That question can be prefaced by asking how a DOI is different from URL system that we have on the Internet today? It seems to me that it’s a slightly more formal version of the same thing. In that case, as DOIs gain traction over the long term, they will tend toward being seen as simply a way of making a contribution to science citable and indexable, which is something we support. It is true though, that at the moment there is an additional perceived value to adding DOIs to your resume and there is no reason why we at Publons should not help reviewers by enabling this.
To learn more about publons check out Publons.com or @Publons. This review was recorded on the 29th of January.
- Absolutely! I did a PhD in physics as well as a postdoc in the US, so I have experience the enjoyment and the trial of publishing papers first hand and I also have been a reviewer. By going through that process as well as talking to colleagues I realized that there must be a better way of doing this. That is the genesis of Publons.
- What actually is a Publon?
Good question! It is a joke in physics where there is a concept of fundamental particles: the electron is the fundamental unit of charge, the photon is the quantum of light etc. The Publon is the facetious term we use to refer to the minimum publishable unit of academic research. We thought it was a cool name for a business. It also speaks to some of the approaches in academic publishing that prevail in the system.
- So back to the concept...
The idea of what we are doing is, right now if you are an academic the single most important thing that you can do for your career is to build your resume by publishing peer reviewed journal articles. So reviewers are incredibly important, but what do they get for their work? The answer is nothing right now. There is an imbalance in the system. The authors who need to have their papers peer reviewed, get credit for their work. On the other hand, reviewers, who provide a valuable service to authors and the general public, get nothing for their efforts. So the question behind Publons is: how can we get reviewers credit for their valuable work in a way that becomes a part of their official resume and helps them securing funding and jobs?
The way that we've done that is to build a platform that collects peer review information from all possible sources: whether it is pre publication peer review, post publication peer review, open peer review (where the authors know who the reviewers are), blind peer review (where the authors don’t know who the reviewers are) double blind peer review (where in addition reviewers don’t know who the authors are). You can also choose to publish the content of your peer review or not. Publons as a platform aims to support all these kinds of reviews in various ways and extract information that we can use to build a reviewers resume -- to make them look good.
"Reviewers are incredibly important, but what do they get for their work?"
- That sounds very exciting, but how does it work in practice? I can imagine that when someone wants to comment on or review an existing paper and make it publicly available that is the easiest use case for you.
Yes we support that. If you go to publons.com you can import any paper with a DOI, we will bring the metadata on to publons. Then you can write a post publication peer review. We actually support the ability to do that anonymously or to do that with your name attached. Which I personally thing is a much better way of doing post publication peer review and we would like to encourage it more.
- You also mention that you are interested in giving reviewers credit for reviewing paper in a standard way - pre publication. How does this work?
- By and large those reviews will be blind unpublished reviews. What we figured out on publons is a way to interface with publishers to collect that information and to expose it in a way that preserves your anonymity, but also proves that you are doing a good job as a reviewer for various journal. That is something we are really excited about.
- So the reviewer gets in touch with you and claims he or she reviewed certain paper and then you get in touch with the publisher?
- Actually it works in both ways. So if you are a reviewer and you have done a pre publication peer review you can for sure upload this information to publons and we will get in touch with the publisher to confirm it and validate it. This way it becomes an official validated part of your resume. We are also starting to work with publishers to pull the peer review information the other way, to automatically populate your profile, should you wish.
- How successful are in talking to the publishers?
- It is still early days, but the conversations that we have had with the publishers are all very promising. So for example we already incorporate some pre publication peer review on our site from places like PeerJ and Biology Direct. We can do that easily because they are open access journals and they provide peer review information under an open license. But if we talk to any publisher they are all very much in favour of making their reviewers look awesome. We are starting a series of pilots now.
- Speaking of making reviewers look awesome, and getting credit for reviews - how is it incorporated into one's CV? How do you imagine people using publons to show their commitment to scientific community?
- The first thing is your reviewer profile on publons where we collect all this information and give you essentially a score as a reviewer, a number that you can go up by doing more reviews and doing better reviews. Obviously as we progress along our development roadmap, we will give you an option to download this as an automatically formatted resume, a document that you can for example show to funding agencies. The other thing that is worth mentioning is that for both collection as well as dissemination of reviews we provide an API. You can come to Publons, sign up for an API key, and do interesting things with this review information. For example we provide peer review information to altmetric.com now, so any paper that has a peer review on our site will get an altmetric boost on altmetric.com because of the peer review. We’ll have some other announcements in this area very soon too.
- I also heard that you are providing DOIs for the reviews. Is that true? Could you tell a bit more about this?
One of the important things about peer review is that it is a contribution in it’s own right to scientific literature. Peer review is not just an evaluation of a paper, but it also sets the context of what’s going on in the paper and enables a dialog. We feel it is good for science to expose as much of that as possible, and to encode it to make it a part of the sphere of human knowledge. One way we figured out to do that is that if there are good and valuable reviews on publons.com we will give them a DOI so they can become citable. The question is how do you decide if a review is worthy enough to get a DOI. We have the concept of endorsement - so if enough members of the community endorse a review we will assign a DOI to it.
- How does it compare to how dialog about papers have been done so far. When there is a controversial paper coming out scientists who don’t agree with the claim in such paper will write a response. Then the original authors will write a response and so on. All these responses are published as papers, usually in the same journal. How do you see this in the context of what publons is doing?
- I see it as exactly the sort of thing we want to support and encourage, but I also that there is a huge body of peer review that is not based around controversy, it is based around not arguments but additions to the published papers. I think it is important to have a platform that supports both of those things. You don’t want to be a platform where people fight about things, you want to be a platform where people collaborate.
"You don’t want to be a platform where people fight about things, you want to be a platform where people collaborate."
- All of that sounds really great, but how much does it cost?
- It’s a good question, because right now as an author if you want to get credit for your research you publish a paper and you will actually pay open access journals. Thousands of dollars for something like that. There is no question that if somebody was able to provide a lot of value to reviewers, making them look good, then that probably would be worth money to reviewers, but that’s definitely not our business model. It’s not the way we aim to make money. The idea is in my mind to be as open as possible. So you want the information to be free and available to make reviewers look awesome. So we don’t charge anything.
- Do the authors have to transfer the copyright of their reviews to you? What license do you use to distribute the reviews?
- It depends on the type of the review we are talking about. If we have a PeerJ review on our site, the PeerJ license is CC-BY. So obviously when we put it on our site we will keep the same license. It is going to vary by publisher, but things that are uploaded by reviewers to publons.com are distributed under a CC-BY license.
- The lack of fees makes me wonder how do you plan to maintain the financial continuity of the project?
- It is really important to have a good idea of how you want to make money, because you want something like Publons to be sustainable and stable. The first thing to say is that peer review is an integral part of academic publishing and something that publishers put a lot of effort into. One problem that publishers find in their peer review process is sometimes it can be tricky to find a reviewer that can be qualified to review a paper and is motivated to do it, but does not have a conflict of interest. With Publons I think we can solve some of those problems with reviews and there are definitely ways to make money.
- Yes it is very hard to find good reviewers. There is an imbalance in terms of incentives: authors are very motivated to publish, but there are only few good reasons to review papers. Some do it to stay up to date with the bleeding edge or try to influence the publishing landscape with their own agenda. Recently Nature published a letter from one of their readers suggesting that journals should require senior authors on submitted papers to prove their activity as reviewers. This would be a necessary condition for the journal to even consider the paper for review. What do you think about this?
- It’s a good idea. The thesis behind what we are doing is that reviewers should get more credit for what they are doing. However, your review record stopping you from publishing seems to be a little bit extreme.
- Publons is not the only platform that is out there doing post publication peer review. How does it compare to PubPeer.com which gain a lot of publicity recently, as well as PubMed Commons which launched few months ago.
- First of all I think both of those services are great. PubPeer is interesting because it is anonymous so it’s more focused on discussing articles behind the veil of secrecy. Publons on the other side is more focused around getting credit for your peer review and assigning it to your profile. So I see what we do as a little bit different at some levels. PubMed Commons is an interesting thing. I am curious how it will develop. It is quite exciting because PubMed has a critical mass of readers and content.
- We see a lot of discontent about the peer review process within the scientific community. How do you see post publication peer review changing the landscape in the short and long term future? Do you think it will replace or complement the traditional pre publication reviews?
- At Publons we see the peer review process developing into something that reviewers get credit for. In that case it does not necessary matter if it’s post publication peer review or pre publication peer review. In all cases the process will be improved. People have a better incentive to do a good job of review and that will speed up science. This is our focus. Rather than forcing people to do a specific type of review, it’s a matter of a empowering them by giving them credit for whatever type of review that is consistent with the culture of their field of study and their peers.
"People have a better incentive to do a good job of review and that will speed up science."
- Do you think post publication peer review could drastically change the landscape of publishing?
- It could at some point in the future, but the more I learn about academic publishing the more I learn how entrenched a lot of behaviours are. I think at some point post publication peer review will take off, but it’s a question where that critical mass will come from.
- You already expressed your opinion about anonymity of peer review. If I understand correctly you are in favour of open non anonymous peer review. Could you explain why do you think this is a better option?
- I am in favour in any kind of peer review. The more peer review the better. On publons.com you can get credit for anonymous and non-anonymous peer review - that is one of the things that is really cool about us. But what I would say is that the blind peer review culture and the idea of anonymity is caused by a fear of somebody being angry at what you say - the mythical somebody. In most cases reviewers do a great job and authors have no problems with the reviews. I think we could open up a lot of the system and it wouldn’t actually drastically affect us and we’d all be better off for it. Anonymity is a great way to hide and attack people for things that could be addressed in a better way. You see that on PubPeer sometimes.
"...you can get credit for anonymous and non-anonymous peer review..."
- Do you think part of the success of PubPeer is that it makes making anonymous reviews so easy?
- Yes, potentially yes.
- There are also some problems with anonymous reviews. Some authors find handling a discussion that drags on very hard. Especially when they have to respond to anonymous comments.
- I do think that in any environment, whether it’s academic or on the web, you do want to build a culture that is healthy and good for everybody. I think anonymity isn’t always the best way to build that sort of culture. That said there is always a place for it.
- You also mentioned giving DOIs as a reward for good reviews. We see DOIs being used by different publishing entities, for example figshare, ScienceOpen etc. However, at the moment if something has a DOI it most likely means it was published in a peer review journal - it’s a proxy for quality. Do you think the value of DOIs will diminish?
- That question can be prefaced by asking how a DOI is different from URL system that we have on the Internet today? It seems to me that it’s a slightly more formal version of the same thing. In that case, as DOIs gain traction over the long term, they will tend toward being seen as simply a way of making a contribution to science citable and indexable, which is something we support. It is true though, that at the moment there is an additional perceived value to adding DOIs to your resume and there is no reason why we at Publons should not help reviewers by enabling this.
To learn more about publons check out Publons.com or @Publons. This review was recorded on the 29th of January.
Great job simplifying a tough topic! Your explanations are always so clear and useful. Excited to see what you write next! Luxury villas with pools in Gwalior
ReplyDelete