Systematic Review Search Strategy Development: (Very Nearly) A Thing of the Past?

A guest post by Rachel Pinotti, MLIS, AHIP

Recently, a faculty member sent me a copy of a June 2017 editorial published in Annals of Internal Medicine entitled Computer-Aided Systematic Review Screening Comes of Age along with the article which it accompanied.  The editorial argues, in short, that machine learning algorithms generate superior results to human-designed search strategies.  It asks (and answers), “Is it time to abandon the dogma that no stone be left unturned when conducting literature searches for systematic reviews? We believe so, because it has a deleterious effect on the number and timeliness of updates and, ultimately, patient health.” (Hemens & Iorio, 2017)

As a librarian who conducts, consults, and teaches systematic review searching, this unleashed a flood of thoughts and questions.

On a philosophical level, these authors’ thesis raised a real tension that I feel with regards to so many topics I teach about: the tension between teaching students about the way things are now vs. the way they very likely will be in the near-to-medium term future. As of now, I don’t think GLMnet and GBM, the machine-learning algorithms utilized in the original article which the editorial accompanies (Shekelle, Shetty, Newberry, Maglione, & Motala, 2017) are widely utilized for systematic review searching, but they quite possibly may be in 3-5-7 years’ time (or less).  Are students better off learning to design and execute comprehensive search strategies, a skill that will serve them in the immediate term and perhaps a few years hence or better off learning how to use GLMnet and GBM, tools that may come into wide use a few years from now?  The answer is probably that they are best off learning both.  Unfortunately I don’t know of anyone within my institution who could teach the current cohort of students these new tools.  (Maybe such people exist and I’m not familiar with them, maybe they don’t exist, or maybe they exist but exercise their skills exclusively for research, not teaching purposes….)

Even once these tools come in to wide use, I wonder if teaching students to design and execute comprehensive search strategies is a bit like teaching them long division – not something they are likely to use frequently or maybe ever in their day to day work, but you need to learn long division in order to understand the concept of division so you understand what is happening when you type 48756/38 into a calculator (or enter your initial search terms into a machine learning search tool).

On a practical level, a big concern with machine learning algorithms is whether they are able to effectively handle multiple information sources and grey literature?  Shekelle indicates, “Although initial results were encouraging, these methods required fully indexed PubMed citations.”  The algorithms could likely be adapted for Embase and other databases, though this might require permission from database providers. Grey literature (conference abstracts, theses, etc.) often does not have complete abstracts and almost by definition is not fully indexed.  Excluding grey literature from a systematic review or meta-analysis introduces a real risk that publication bias will produce a biased result, as documented by McAuley, Pham, Tugwell, & Moher, 2000 and others.

I’ve always felt that some of the best practices recommended in systematic review searching, such as the recommendation to use both index terms and keywords, are redundant and unhelpful.  So I don’t doubt – and on the contrary am quite receptive to –the argument that current best practices in systematic review searching favor sensitivity far too greatly over precision. Some filters (e.g. language) are much more effective than others (e.g. age, gender). I absolutely think that filters, especially Clinical Queries, are useful when you need to find the few most relevant articles on a topic, but do not believe that they are yet able to unearth all the studies that would have a bearing on a topic, as suggested by Hemens.

The editorial asserts, “Simple Boolean filters applied when searching increased precision 50%…while identifying 95% of the studies…” (Hemens & Iorio, 2017) with an underlying, unstated assumption that these results would hold true across fields and topics. Thinking of this in evidence-based medicine terms, these articles present case-series level evidence for the effectiveness of machine learning search algorithms.  Before changing our standard of practice, I would like to see larger scale studies (perhaps cross-over studies in which the same review is conducted both using evidence found through machine learning and traditional search methods) which indicate that machine learning search tools are as effective or more effective than traditional search methods before advocating that they be widely adopted.

The ideal outcome would be getting to a point where machine learning is used to effectively search traditional published material allowing reviewers to focus their energy on searching grey literature. However until we reach the point where these methods have been validated and can be widely used and taught, current best practice remain just that: the best methods for unearthing all evidence that may have a bearing on a given research topic.

Rachel Pinotti, MLIS, AHIP
Assistant Library Director, Education & Research Services
Levy Library, Icahn School of Medicine at Mount Sinai
Box 1102 – One Gustave L. Levy Pl.
New York, NY 10029-6574

Email: rachel[dot]pinotti[atsign]mssm[dot]edu
Phone: available via MLA members list

Follow us on Twitter @Levy_Library to learn about Levy Library events and initiatives.

References:

Hemens, B. J., & Iorio, A. (2017). Computer-aided systematic review screening comes of age. Annals of Internal Medicine, 167(3), 210-211. doi:10.7326/M17-1295

Shekelle, P. G., Shetty, K., Newberry, S., Maglione, M., & Motala, A. (2017). Machine learning versus standard techniques for updating searches for systematic reviews: A diagnostic accuracy study. Annals of Internal Medicine, 167(3), 213-215. doi:10.7326/L17-0124

McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000). Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? The Lancet, 356(9237), 1228-1231. doi:http://dx.doi.org/10.1016/S0140-6736(00)02786-0

PubMed’s Backdoor Makes Me Question Quality

Ok, I am dancing in my seat saying “I told you there is a problem with PMC as the backdoor to PubMed.” I know it is gloating but librarians get so little to gloat about so forgive me.

Back in 2011 I wrote the post, Backdoor Method to Getting Articles into PubMed: Is indexing so important?  At the time I was more concerned about the findability and lack of indexing of PMC articles that found their way into PubMed.  It wasn’t until a few years later with the popularity of Beall’s List did I begin to think about the quality of PMC articles now in PubMed.  In a discussion on the Medlib-l listserv regarding Beall’s list I mentioned the PMC backdoor again.  As I said, ther are “some researchers who see no distinction between PMC submitted journal articles from non-indexed journals and ones that are indexed in MEDLINE. To them it is in PubMed and that is good enough.”

I am so happy that Kent Anderson in the Scholarly Kitchen wrote the post, “A Confusion of Journals – What is PubMed Now?”  His post along with articles and communications published in peer reviewed journals Neuroscience, Lancet, and Archives of Physical Medicine and Rehabilitation mention the ever growing problems of predatory journals within the profession.  Neuroscience and Lancet specifically mention the problem within PubMed.

Unfortunately, many doctors and researchers believe what Jocalyn Clark and Richard Smith write in the editorial piece in The BMJ, “Articles in predatory journals, although publicly available through internet searches, are not indexed in reputable library systems. The articles are not discoverable through standard searches, and experienced readers and systematic reviewers will be wary of citing anything from these journals. The research is thus lost. ”

That is a partially true statement. The articles in PubMed Central are NOT indexed in MEDLINE, HOWEVER they are totally discoverable through standard searches in PubMed.  The “research” from that predatory journal is not lost.  It is is very discoverable in PubMed.

Personally, I think this is a mess.  NLM should not have given its repository the name PubMed Central, it is confusing and just the name muddies the waters.  But that ship has sailed.  NLM needs to put some serious authority control on PMC quickly.  Not only do articles in predatory journals water down what is already indexed in MEDLINE and within PubMed but it could be have significant problems for the treatment of patients and researchers.  Fake news seems to be the phrase of the day recently.  More than ever people need reliable resources where they can find credible information.  Allowing articles from predatory journals into PubMed through PMC creates a credibility problem with PubMed.  It is basically like seeing Gweneth Paltrow’s Goop medical advice published on WebMD.

I really hope Kent’s post and the other researcher’s articles get the attention they deserve.  I hope it leads others to look at and question the quality of articles allowed into PubMed by virtue of PMC.  Then perhaps NLM will look at quality control methods for PMC.  If they don’t they run the risk of ruining PubMed.  We already use another (pay) database when we must assess the quality of article.

Unsustainable Costs of Library Resources

Sometimes I feel like medical librarians have been talking to brick walls.  Either that, or we are talking to bobble heads who don’t really listen to us but nod their heads in agreement.

I get a weekly email summarizing the healthcare industry.  It is broken into local and national information and it is often an interesting quick read.  Today I read the article “US medical expenditures on the rise, except for primary and home health.” The largest expenditures were attributed to prescribed medications, specialty physicians, visits to the emergency department and inpatient hospitalizations.  While that was interesting, what really caught my eye were the links to the Top 40 articles in the past 6 months on the right:

 

Besides the one article about Trump’s budget, the first 9 articles listed were all about hospitals losing money or going bankrupt.

While Medicaid enrollment has increased, its reimbursement is significantly less than private insurers.  Then you have the increased costs of providing care. “In 2016, Cleveland Clinic’s expenses totaled $7.3 billion, up 19 percent from $6.1 billion in the year prior. The increase was largely attributable to growth in pharmaceutical, labor and supplies costs, which climbed 23 percent, 19 percent and 13 percent, respectively, year over year.”  The Cleveland Clinic is not alone,  Nationally, hospitals’ operating margins have shrunk due to smaller reimbursement, regulatory uncertainty and new alternative payment models.

Yet medical library resource vendors operate business as usual by increasing the costs of their products to clients (libraries) that are viewed as expensive cost centers in an industry that is losing money.  When librarians complain to the vendors known for price gouging, the vendors answer is to have the hospital shift the cost of their product out of the library budget and to IT, operations or another department with more money.  It doesn’t take a financial analyst to know THAT ISN’T A SUSTAINABLE SOLUTION!!! The hospital still pays for that product at a rate far above inflation and far above their reimbursement.

Librarians have been telling vendors for years that their large price increases are unsustainable.  You could say we are complicit because we suck it up and pay for the price increases by finding the money through cutting book budgets, dropping other products, etc.  However, we are between a rock, a hard place, and an abyss.  The doctors need the price gouging resources so they can practice and treat patients. The vendors justify their price gouging by saying they actually “save the institution much more” because the doctors use it.  Hospital administrators end up cutting budgets overall to get handle on increasing costs.

It isn’t a pretty picture and it will get worse.  Are they getting the same healthcare industry news I am getting?  Are the just putting their head in the sand?  Is it like the home loan industry where the banks knew something was amiss but they rode that horse until it died a national debt crippling death?

Just wondering…..

 

 

 

Research Impact Part 2: Whole New System

This is a 3 part blog series.
Research Impact Part 1: Moving Away from tracking authors’ articles.

I am not going to mentioned the company we went with.  The primary reason for this is because I am trying to write this as broadly as possible so that it applies to anybody who is considering this type of endeavor, not the nitty gritty of a specific software.  While there is always room for improvement I am happy with what we chose and I am very happy with the support we have received upon implementing it.  If you are interested in learning more about the specific products we chose, email me and I will answer those questions.

As I mentioned in Part 1, there are a lot of products out there, Converis, InCites, Profiles, Pure, Plum, etc.  After looking at several products we ended up choosing two products by the same vendor.  The two products allowed us to upload HR data so that articles would be automatically sorted and indexed by author AND department, and it also included article level metrics that were more informative than just the journal impact factor.

There were a few major points that we had have in our system.  I recommend creating your list or requirements BEFORE you start contacting vendors because it easy to get caught up in all of the cool things their products can do which may or may not be compatible with your needs.  For example, you don’t want to get excited about new dishwasher that has a new wash cycle that gets your dishes so clean it could wash the white off of them when that model only comes with large handle that blocks your silverware drawer making it necessary to always open the dishwasher before opening the silverware drawer (or completely re-design your kitchen).  So have your must haves ahead of time.

Our must haves:

Automation –  That sounds stupid but there are some systems that are more automated than others.  All require some human touching even after implementation. Think about how much time you want/can spend on the system once it is all up and running.

Institutional organization structure – It must be able to organize published articles from all of our employees by department and institute. (Institute has several departments within it.)  This was a requirement because Administration wants to know the authorial output of each department and institute and annual performance review time.  So we need to click on Urology and see the papers written by people in Urology.  Do you need to track secondary appointments? Be careful that can be a long dark rabbit hole to go down.

Impact – While almost all of the products we looked at had some type of article impact number/indicator we needed to communicate with Administration as to the one that THEY wanted and felt the most comfortable with.  This is VERY important. There are about as meany methods of measurement as there are digits in pi.  Our Administration is very traditional, so that required us to look at product that used well established metrics that have been around for many years that our people were familiar with.

Things we didn’t need:

Repository system – Currently our institution has no interest in hosting a repository of the papers written by authors.  Obviously this could change, and if it does then it requires a fresh new look at things.

Author submission – Authors are not reliable providers of the citations they publish.  We had 20+ years of experience with this. Some authors don’t have the time to upload anything. Some authors add citations to their weight loss article in Ladies Home Journal.  Other authors have citations that say “in press” from 5 years ago.  Your data out is only as good as the data you put in, and we needed tight control over the data. So we didn’t want author submission. If it was a feature, it had to be something that could be turned off.

ORCID – That sounds odd.  We actually need ORCID. Everybody needs ORCID.   Until there is a mandate that requires an author to provide their ORCID number upon publication then ORCID will just be something “nice to have.” Even in a heavy research and publishing institution, ORCID is still something of a novelty.  We did not want something that was overly built on ORCID.

Panacea systems – Many of these products track everything under the sun.  They track grants, funding, etc.  There are systems that track the entire research life cycle, from the sparkle in that researcher’s eye to the mature cited paper and everything in between.  Like many institutions we have a various systems (some home grown) that track a lot of things that the “all in one” systems track.  Unless you have buy in across the institution to change every part of the research IT process then an all in one system may be overkill.

Lessons we learned:

HR or organizational data is messy –  Unfortunately this is not unique to us.  I have heard from people at several large institutions to learn that this type of data is often not clean.  What do I mean by that?  Assuming HR will let you have an HR dump of all employees (they are often very reluctant to do this) you might discover that there is missing or duplicate information.  You might find out that several people’s secondary appointment is an entire hospital (not a department).  You might find that HR data doesn’t include graduate students.  We had to piece together our data with several institutional systems and we created a python script to strip, clean, and organize the data into the format that they vendor used.

***
Updated Paragraph 5/19/17

Regarding HR uploads….. Think very carefully about if you want your entire HR list of employees added into the system.  That could be a lot of unnecessary data.  Do you need/want people in environmental services, security, IT, etc. in your system?  Do you only want doctors and researchers?  What about nurses, PA’s, medical students, residents, and allied health who publish?  You need to sit down and figure who you want to track and how you are going to get that list of people.

*****

Comparisons – You have to be very mindful if you use one of these systems to compare your institution with another.  If your administration is competitive and likes to see how they are ranked in their disciplines or overall, they are going to ask you to use the product to compare themselves against their peers.  Most of the products we looked at could compare different institutions, disciplines, and people.  But you must do this carefully.  For example: You cannot compare a large research hospital system with university hospital system.  Even though they are peers, the university system includes many more researchers and disciplines that can skew the results.  While you can compare disciplines or subjects, you cannot compare departments.  One institution’s cardiovascular department may include pediatric cardiology while another may not.

 

 

 

Future of Biomedical Publishing

A medical librarian friend of mine agreed to answer questions for a week on NEJM Resident 360. It involved some future casting and she emailed the medical librarian listserve to pick our brains. I sent her a few crystal ball predictions and she thought they were good and I should post them on the blog to further the discussion.

So, here is the question: What does the future of delivering medical literature and latest research hold?<https://resident360.nejm.org/posts/6339>

Here are my thoughts:

  • We are going to see more movement in the area of Open Textbooks.  Open access journals have started paving the way and now with more institutions really looking into curbing the costs of textbooks you are going to see medical schools and hospitals go in that direction once the larger universities really start committing to that idea.
  • There are going to be some big changes to peer review and publishing editorial boards to have more transparent data, information, etc. Currently we are living in a world that questions established medical facts as false.  Part of the problem is that there wasn’t enough vetting or the ability to vet information that allowed questionable, conflict of interest,  or fake articles to be published.  These questionable articles hurt the entire profession and cause people to distrust good information.  It took over 10 years to Andrew Wakefield’s article to be officially retracted. We need to ask ourselves, would the autism vs vaccines controversy have become as big as it was if the data was published immediately?
  • Reproducible data is getting more and more important.  With NIH’s data sharing requirements and the increase in data repositories, the ability reproduce research based on the data is extremely important.  However, a recent Nature study http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 found that 70% of researchers tried and failed to reproduce another scientist’s experiments and more than half of the scientists failed to reproduce their own experiments.  Yet we must be able to sort through the false leads from the latest discoveries.
  • Access will be more integrated.  Currently you can do a search in PubMed and links to the full text are available along with similar articles and citing articles.  Electronic medical health records can integrate health information such as UpToDate into the medical record.  I think as we move forward the literature will be more “on demand” and more integrated in other resources.
  • We will see more medical literature delivered via social media in the next few years.  The real growth is customized on demand information retrieval.  I can see where something like Amazon’s Alexa or Google Home could interface with medical journals table of contents and articles and give you the latest updates or sync with your device or car and listen to the article while you are commuting.  Similar to a Browzine for the Echo or Home.

What do you see in your crystal ball?

What Have You Done For Me Lately?

A few years ago I got started down a path thanks to a library director friend in Oklahoma who asked me to teach a class specifically for hospital librarians.  While he worked in an academic medical library he noticed that hospital librarians in his area needed some help thinking of different ways to prove their value or justify their existence, especially in light of the Affordable Care Act (which was very new at the time and nobody knew what would happen…but they knew it would have a pretty big impact).

In preparation for the class, I did a lot of research on the ACA.  I was and still am not an authority on the ACA but I did learn a lot about the changing landscape that hospitals would be (and now are) dealing with.  The biggest change was moving from a fee for service model to a value based model of providing healthcare services.

(Gross over simplification of the ACA coming next….if you want more info click here for a very detailed LibGuide on it.)

Traditionally hospitals were paid on how many procedures they did and billed.  If you had a heart attack were admitted, then released, and then readmitted a few days later…they were paid for the services they provided each time you were admitted.  The ACA now penalizes hospitals that have readmission rates for certain conditions, procedures, etc. In the heart attack example the hospital would still be paid for the services they provided BUT they would also incur a penalty if their readmission rate for heart attacks was too high.  In the beginning these penalties happened for just a few conditions such as heart failure but it has evolved to include more conditions AND acquired hospital conditions like infections.  Every hospital is ranked on these things and those with the poorest score are penalized.  Those with the best score are rewarded.

Not only are hospitals in competition with each other over their reimbursement rates. But they are also in competition for patients.  Now days it is very common for patients to have a very high deductible for care.  If a patient has to spend $4000 before their insurance kicks in, they are going to look for and compare the costs of hospitals.  Both patients and insurance companies are doing this.

So what does this mean for the librarian and why on earth did I focus on this in my class in Oklahoma?  It means that hospitals have to completely change their financial and caregiver goals.  This also means that they are looking at EVERY department in the hospital to see how it fits into these new goals of the hospital.  So cardiology, environmental services, IT, and yes the library all are judged by administration as to how they help the hospital meet their goals.  Clinical departments have a leg up on departments like the library because they can point directly to numbers to tell hospital administration what they are directly doing to impact the hospital goals.  Heck even environmental services has a leg up on the library.  “Was your room clean?” is a question on the HCAHPS hospital survey that patients receive to rate their hospital experience…..which also directly impacts ACA reimbursement.

So while other departments are able to point to specific data to illustrate to the administration how their department helps the hospital achieve its goals, the library has no such data.  Sure we do literature searches and support the caregivers, but what data do we have to show that those searches impact the hospital’s goals?

I recently attended a meeting where hospital administration explained the hospital’s strategy and goals and then explained how the clinical departments fit into those goals.  The administrator then explained how they will be working with non-clinical departments in the near future to develop appropriate measures to support the hospital goals.  DING DING DING At least the administration realizes the value of the non-clinical departments and will be working with them to create MEASURABLE goals that help the hospital meet its goals.  I am not so sure every hospital administration is reaching out to their non-clinical departments like this.

After hearing of the hospital administration mentioning non-clinical departments accountable for providing measurable data that supports the hospitals goals, I had a flashback to my Oklahoma group 2 years ago and to the other ACA and the library classes I taught after them.

Hospital administration wants to know specifically how you help them meet their goals.  They are asking “What have you done for me lately?”  I hope for your sake you just don’t tell them you do literature searches to support the doctors and nurses who treat patients.  Because while that is true….they are going to need to know in a 3 bullet point slide how literature searches lead to their hospital goals being achieved.  It is not their job to fill in the blanks as to how the lit searches do that.  It is your job.

Three Legged Dogs are Cool

I know I owe you all part 2 post on Life and MLA President but I have gotten delayed.  My 1 yr old puppy, Chewie (as in Chewbacca of course) was diagnosed with cancer and had to have his leg removed.  That kind of sucking away all of my available free time.

Chewie doing good after surgery.

While Chewie doesn’t have much to do with medical librarianship, I have to say the little dude has inspired.

Life as the President of MLA: Part 1

I’m baaaaaaaaaack.

It has been a while since I have posted.  I want to thank all of the guest posters who have written and kept things going while I took a year off.  You guys were great.

So, one of the questions I have gotten over the year is what is like to be President of MLA.  I want to take the time to answer that question.  I will try to provide as much information without getting so wordy that it becomes the longest blog post ever.  With that in mind I am going to make this a two part series.  The first part is going to focus on my experiences while the second part will get into the nitty gritty of time and how things are done.

First and foremost it was an awesome experience.  I wish everyone who wants to run as President will get to be President.

Meeting Librarians:

Attending the Chapter Meetings and the CHLA/ABSC meeting provided me with a rich opportunity to meet more librarians than I would have ever met in my lifetime.   Meeting people and sharing stories and experiences, brainstorming ideas, learning from each other is the one thing, hands down, that made the whole Presidential term great.  Unfortunately I wasn’t able to attend every Chapter’s meeting, they all overlap a great deal.  However, I want to still attend every Chapter’s meeting at least once.  I learned so much from different groups that I feel like it will do me good to step out and go to a different Chapter every once and a while.  I think we are sometimes so wrapped up in our own groups that we don’t always know what is going on elsewhere and big MLA annual meeting can sometimes be information overload.  I kind of got this idea from a librarian friend whose child was graduating college during her Chapter meeting.  She decided to go to her child’s graduation and then go to a different Chapter’s meeting (which didn’t conflict with the graduation) and present a poster.  So a few years down the road if I happen to be at your Chapter meeting, I am just branching out and expanding my boundaries.

Capitol Hill Visits:

I was both equally nervous and excited when I joined the AAHSL and MLA Joint Legislative Task Force in Washington DC.  Because NLM and the NIH are government entities they cannot lobby on behalf of themselves.  Who is to lobby for the things we medical librarians use? Well, medical librarians will lobby on their behalf.  The first day we met to learn and discuss all of the most relevant and timely issues that we needed to make our Senators and Representatives aware of.  We were given packets and created talking points.  The next day we were set loose in 3 teams of 3-4 people.  We met with the staffers responsible for health affairs for the Senators or Representatives of our states.  It helped to be in a group because the members in your group chimed in and provided support while you were talking with your state’s staffer.  It was such a cool experience walking the halls of the Senate and House buildings.  I felt a part of the governmental process that I learned about when I was a kid in school.

MLA Historical Marker:

I got to be a witness to library history.  Last November I went to Philadelphia to attend the unveiling of the MLA Historical Marker  at 1420 Chestnut Street (later designated 1420-1422 Chestnut Street).  MLA is the world’s oldest association of medical librarians and information professionals!  To be a part of such a respected and long standing association and to see it being recognized by the Pennsylvania Historical and Museum Commission was humbling and wonderful at the same time.

MLA Business:

I am still honored that I was elected by the membership to serve as President and to guide MLA forward as an association.  Associations and organizations are changing rapidly.  It was a great privilege to be a part of the process as MLA members, Board members and staff worked to help the association evolve to meet the needs of future medical librarians.  All of us worked together to look at things from a different perspective and not rely on the old “we always did it that way” principle.  Rome was not built in a day and we have had some hiccups along the way but to be a part of the evolutionary process is something that I hold near and dear to my heart.  As I went to the Chapter meetings, Board meetings, and spoke or emailed with so many people in the association, I always did my best to try and understand people and perspectives and to help the association. I love the people in this organization and I believe that we all are working to make things great.

Family:

If you were at the Toronto meeting you might have noticed that I teared up a little and my voice cracked when I thanked my family for their support.  I am not a crier.  I’m a suck it up kind of person. Being MLA President provided a wonderful personal experience for me as well.  My kids have always known that I help doctors and nurses find information to help treat patients.  During my term as President they were able to see medical librarianship beyond my day to day job.  They would see me chatting on Twitter with other librarians from all over the world (*see funny story below).  When I left for Chapter meeting I would tell them where I was going and what I would be doing.  My trip to Philadelphia for the historical marker gave me the opportunity to talk to them about the history and evolution of medical librarianship.  Finally, they were able to hear about their mom going to Washington DC to talk to our elected officials (well their staffers) to try and influence positive change. Through all of this they saw first hand of what “Oh, the Places You’ll Go!” can be.

My experience as MLA President will stick with me forever.  I enjoyed and learned so much during my time.  However, I am glad to slow down just a bit and be the past President now.  My kids will be sad that they won’t get as much Garrett’s popcorn, but I am ready for the next phase of my career (whatever that may be) to begin.

*Funny story*
One Thursday evening while participating in the #medlibs chat, my son asked me who I was texting.  I told him I was Tweeting and talking with other medical librarians around the US and the world.  He paused a second then asked if I knew every medical librarian in the US in the world.  I said no of course not.  He then asked if I had met the people I was chatting with.  I told him that I had met some but not all.  He looked at me and said, “I thought you told us we weren’t supposed to chat online with people we haven’t met in real life. You should be careful Mom.” -I got a dose of my own internet safety speech from my kid.

Ithaka S+R Local Faculty Survey and Health Sciences Libraries:

In 2014, the Virginia Commonwealth University (VCU) Libraries administered the Ithaka S+R Local Faculty Survey to our faculty to measure their changing needs and perceptions of library resources. The survey, consisting of seven modules including the health sciences module, was distributed across our campus. The health sciences module targets faculty with patient or clinical care responsibilities. At this time, few health sciences libraries have used this instrument. Survey questions focused on attitudes and skills related to evidence-based scholarly resources as well as access and use of other library services and resources.

Of course we all know students’ research skills especially in finding evidence based scholarly research are often lacking. This came out clearly from the results of the Ithaka survey. Again we were not surprised to see that faculty also see these research skills as a very important aspect of the students’ learning. However, it is still amazing that a large number of faculty did not see teaching the skill of finding evidence based scholarly materials and research skills as primarily the librarians’ function. A timely reminder for us to continue informing our faculty that we indeed have are more than capable to teach students research skills especially when it comes to evidence based practice. What a great opportunity for us to collaborate with our faculty and remind them about everything else we bring to the table!

Hopefully more health sciences libraries will use this survey instrument to measure their faculty perceptions because I think it would be interesting to compare the VCU’s Tompkins-McCaw Library’s findings with other libraries that have surveyed their health sciences faculty.

Post Publication Review: Librarians’ Role

On Monday I spoke to a group of physicians, hospital administrators and other medical professionals on the impact of the publishing industry on hospitals and medicine.  While I spoke about the elephant in the room, sky high subscription rates for institutions, I also spoke about the role of post publication review in medical literature.

The example I gave was Amanda Capes-Davis who comments within PubMed Commons on mistaken identities of cell lines within the medical literature and her efforts to inform readers of potential cell line problems.

I wish I had seen Melissa Rethlefsen’s PubMed Commons post when I was creating my presentation.  It is a great example of how medical librarians can examine the published literature for inconsistencies regarding the methodologies of their search of the literature when conducting research.

Melissa reviewed the article “Comparative efficacy and safety of blood pressure-lowering agents in adults with diabetes and kidney disease: a network meta-analysis.” Lancet. 2015 May 23;385(9982):2047-56. doi: 10.1016/S0140-6736(14)62459-4.  She and her colleagues at the University of Utah Spencer S. Eccles Health Sciences Library reviewed and tried to duplicate the authors’ Embase search strategy which the authors reported in the Appendix (pages 13-14). According to the PRISMA flow chart the authors retrieved 1,371 results (Appendix page 37).

According to Melissa,

This study highlights the need for more accurate and comprehensive reporting needed for search strategies in systematic reviews and other literature search-based research syntheses, and the need for better peer review of search strategies by information specialists/medical librarians. Though the searches in the Appendix are on face value replicable and high quality, on closer inspection, they do not in fact meet the reporting standards as outlined by PRISMA Statement items #7 and #8: “Describe all information sources in the search (e.g., databases with dates of coverage, contact with study authors to identify additional studies) and date last searched” and “Present the full electronic search strategy for at least one major database, including any limits used, such that it could be repeated.”

For me, this comment within PubMed Commons highlights the need for librarians to analyze search strategies in the literature and to speak up and set the record straight when things are not correct or there are issues of reproducibility.  Just like Amanda Capes-Davis who sheds light on cell line problems or the statisticians who questioned the math in an NEJM article (later retracted), we are subject experts and it is important that we help contribute to post publication peer review.

Medical librarians all around the world can point to examples of when a poor literature search could have saved lives or prevented injuries, death or illness.  I am not suggesting the literature review in the article by Palmer et al. could cause patient harm.  But PubMed Commons provides librarians with an avenue by which to question literature reviews presented in research.  Hopefully by highlighting questionable search strategies or corroborating effective search strategies it will lead to better use of librarians and better research all around.