Research Impact Part 2: Whole New System

This is a 3 part blog series.
Research Impact Part 1: Moving Away from tracking authors’ articles.

I am not going to mentioned the company we went with.  The primary reason for this is because I am trying to write this as broadly as possible so that it applies to anybody who is considering this type of endeavor, not the nitty gritty of a specific software.  While there is always room for improvement I am happy with what we chose and I am very happy with the support we have received upon implementing it.  If you are interested in learning more about the specific products we chose, email me and I will answer those questions.

As I mentioned in Part 1, there are a lot of products out there, Converis, InCites, Profiles, Pure, Plum, etc.  After looking at several products we ended up choosing two products by the same vendor.  The two products allowed us to upload HR data so that articles would be automatically sorted and indexed by author AND department, and it also included article level metrics that were more informative than just the journal impact factor.

There were a few major points that we had have in our system.  I recommend creating your list or requirements BEFORE you start contacting vendors because it easy to get caught up in all of the cool things their products can do which may or may not be compatible with your needs.  For example, you don’t want to get excited about new dishwasher that has a new wash cycle that gets your dishes so clean it could wash the white off of them when that model only comes with large handle that blocks your silverware drawer making it necessary to always open the dishwasher before opening the silverware drawer (or completely re-design your kitchen).  So have your must haves ahead of time.

Our must haves:

Automation –  That sounds stupid but there are some systems that are more automated than others.  All require some human touching even after implementation. Think about how much time you want/can spend on the system once it is all up and running.

Institutional organization structure – It must be able to organize published articles from all of our employees by department and institute. (Institute has several departments within it.)  This was a requirement because Administration wants to know the authorial output of each department and institute and annual performance review time.  So we need to click on Urology and see the papers written by people in Urology.  Do you need to track secondary appointments? Be careful that can be a long dark rabbit hole to go down.

Impact – While almost all of the products we looked at had some type of article impact number/indicator we needed to communicate with Administration as to the one that THEY wanted and felt the most comfortable with.  This is VERY important. There are about as meany methods of measurement as there are digits in pi.  Our Administration is very traditional, so that required us to look at product that used well established metrics that have been around for many years that our people were familiar with.

Things we didn’t need:

Repository system – Currently our institution has no interest in hosting a repository of the papers written by authors.  Obviously this could change, and if it does then it requires a fresh new look at things.

Author submission – Authors are not reliable providers of the citations they publish.  We had 20+ years of experience with this. Some authors don’t have the time to upload anything. Some authors add citations to their weight loss article in Ladies Home Journal.  Other authors have citations that say “in press” from 5 years ago.  Your data out is only as good as the data you put in, and we needed tight control over the data. So we didn’t want author submission. If it was a feature, it had to be something that could be turned off.

ORCID – That sounds odd.  We actually need ORCID. Everybody needs ORCID.   Until there is a mandate that requires an author to provide their ORCID number upon publication then ORCID will just be something “nice to have.” Even in a heavy research and publishing institution, ORCID is still something of a novelty.  We did not want something that was overly built on ORCID.

Panacea systems – Many of these products track everything under the sun.  They track grants, funding, etc.  There are systems that track the entire research life cycle, from the sparkle in that researcher’s eye to the mature cited paper and everything in between.  Like many institutions we have a various systems (some home grown) that track a lot of things that the “all in one” systems track.  Unless you have buy in across the institution to change every part of the research IT process then an all in one system may be overkill.

Lessons we learned:

HR or organizational data is messy –  Unfortunately this is not unique to us.  I have heard from people at several large institutions to learn that this type of data is often not clean.  What do I mean by that?  Assuming HR will let you have an HR dump of all employees (they are often very reluctant to do this) you might discover that there is missing or duplicate information.  You might find out that several people’s secondary appointment is an entire hospital (not a department).  You might find that HR data doesn’t include graduate students.  We had to piece together our data with several institutional systems and we created a python script to strip, clean, and organize the data into the format that they vendor used.

***
Updated Paragraph 5/19/17

Regarding HR uploads….. Think very carefully about if you want your entire HR list of employees added into the system.  That could be a lot of unnecessary data.  Do you need/want people in environmental services, security, IT, etc. in your system?  Do you only want doctors and researchers?  What about nurses, PA’s, medical students, residents, and allied health who publish?  You need to sit down and figure who you want to track and how you are going to get that list of people.

*****

Comparisons – You have to be very mindful if you use one of these systems to compare your institution with another.  If your administration is competitive and likes to see how they are ranked in their disciplines or overall, they are going to ask you to use the product to compare themselves against their peers.  Most of the products we looked at could compare different institutions, disciplines, and people.  But you must do this carefully.  For example: You cannot compare a large research hospital system with university hospital system.  Even though they are peers, the university system includes many more researchers and disciplines that can skew the results.  While you can compare disciplines or subjects, you cannot compare departments.  One institution’s cardiovascular department may include pediatric cardiology while another may not.

 

 

 

Future of Biomedical Publishing

A medical librarian friend of mine agreed to answer questions for a week on NEJM Resident 360. It involved some future casting and she emailed the medical librarian listserve to pick our brains. I sent her a few crystal ball predictions and she thought they were good and I should post them on the blog to further the discussion.

So, here is the question: What does the future of delivering medical literature and latest research hold?<https://resident360.nejm.org/posts/6339>

Here are my thoughts:

  • We are going to see more movement in the area of Open Textbooks.  Open access journals have started paving the way and now with more institutions really looking into curbing the costs of textbooks you are going to see medical schools and hospitals go in that direction once the larger universities really start committing to that idea.
  • There are going to be some big changes to peer review and publishing editorial boards to have more transparent data, information, etc. Currently we are living in a world that questions established medical facts as false.  Part of the problem is that there wasn’t enough vetting or the ability to vet information that allowed questionable, conflict of interest,  or fake articles to be published.  These questionable articles hurt the entire profession and cause people to distrust good information.  It took over 10 years to Andrew Wakefield’s article to be officially retracted. We need to ask ourselves, would the autism vs vaccines controversy have become as big as it was if the data was published immediately?
  • Reproducible data is getting more and more important.  With NIH’s data sharing requirements and the increase in data repositories, the ability reproduce research based on the data is extremely important.  However, a recent Nature study http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 found that 70% of researchers tried and failed to reproduce another scientist’s experiments and more than half of the scientists failed to reproduce their own experiments.  Yet we must be able to sort through the false leads from the latest discoveries.
  • Access will be more integrated.  Currently you can do a search in PubMed and links to the full text are available along with similar articles and citing articles.  Electronic medical health records can integrate health information such as UpToDate into the medical record.  I think as we move forward the literature will be more “on demand” and more integrated in other resources.
  • We will see more medical literature delivered via social media in the next few years.  The real growth is customized on demand information retrieval.  I can see where something like Amazon’s Alexa or Google Home could interface with medical journals table of contents and articles and give you the latest updates or sync with your device or car and listen to the article while you are commuting.  Similar to a Browzine for the Echo or Home.

What do you see in your crystal ball?

What Have You Done For Me Lately?

A few years ago I got started down a path thanks to a library director friend in Oklahoma who asked me to teach a class specifically for hospital librarians.  While he worked in an academic medical library he noticed that hospital librarians in his area needed some help thinking of different ways to prove their value or justify their existence, especially in light of the Affordable Care Act (which was very new at the time and nobody knew what would happen…but they knew it would have a pretty big impact).

In preparation for the class, I did a lot of research on the ACA.  I was and still am not an authority on the ACA but I did learn a lot about the changing landscape that hospitals would be (and now are) dealing with.  The biggest change was moving from a fee for service model to a value based model of providing healthcare services.

(Gross over simplification of the ACA coming next….if you want more info click here for a very detailed LibGuide on it.)

Traditionally hospitals were paid on how many procedures they did and billed.  If you had a heart attack were admitted, then released, and then readmitted a few days later…they were paid for the services they provided each time you were admitted.  The ACA now penalizes hospitals that have readmission rates for certain conditions, procedures, etc. In the heart attack example the hospital would still be paid for the services they provided BUT they would also incur a penalty if their readmission rate for heart attacks was too high.  In the beginning these penalties happened for just a few conditions such as heart failure but it has evolved to include more conditions AND acquired hospital conditions like infections.  Every hospital is ranked on these things and those with the poorest score are penalized.  Those with the best score are rewarded.

Not only are hospitals in competition with each other over their reimbursement rates. But they are also in competition for patients.  Now days it is very common for patients to have a very high deductible for care.  If a patient has to spend $4000 before their insurance kicks in, they are going to look for and compare the costs of hospitals.  Both patients and insurance companies are doing this.

So what does this mean for the librarian and why on earth did I focus on this in my class in Oklahoma?  It means that hospitals have to completely change their financial and caregiver goals.  This also means that they are looking at EVERY department in the hospital to see how it fits into these new goals of the hospital.  So cardiology, environmental services, IT, and yes the library all are judged by administration as to how they help the hospital meet their goals.  Clinical departments have a leg up on departments like the library because they can point directly to numbers to tell hospital administration what they are directly doing to impact the hospital goals.  Heck even environmental services has a leg up on the library.  “Was your room clean?” is a question on the HCAHPS hospital survey that patients receive to rate their hospital experience…..which also directly impacts ACA reimbursement.

So while other departments are able to point to specific data to illustrate to the administration how their department helps the hospital achieve its goals, the library has no such data.  Sure we do literature searches and support the caregivers, but what data do we have to show that those searches impact the hospital’s goals?

I recently attended a meeting where hospital administration explained the hospital’s strategy and goals and then explained how the clinical departments fit into those goals.  The administrator then explained how they will be working with non-clinical departments in the near future to develop appropriate measures to support the hospital goals.  DING DING DING At least the administration realizes the value of the non-clinical departments and will be working with them to create MEASURABLE goals that help the hospital meet its goals.  I am not so sure every hospital administration is reaching out to their non-clinical departments like this.

After hearing of the hospital administration mentioning non-clinical departments accountable for providing measurable data that supports the hospitals goals, I had a flashback to my Oklahoma group 2 years ago and to the other ACA and the library classes I taught after them.

Hospital administration wants to know specifically how you help them meet their goals.  They are asking “What have you done for me lately?”  I hope for your sake you just don’t tell them you do literature searches to support the doctors and nurses who treat patients.  Because while that is true….they are going to need to know in a 3 bullet point slide how literature searches lead to their hospital goals being achieved.  It is not their job to fill in the blanks as to how the lit searches do that.  It is your job.

Three Legged Dogs are Cool

I know I owe you all part 2 post on Life and MLA President but I have gotten delayed.  My 1 yr old puppy, Chewie (as in Chewbacca of course) was diagnosed with cancer and had to have his leg removed.  That kind of sucking away all of my available free time.

Chewie doing good after surgery.

While Chewie doesn’t have much to do with medical librarianship, I have to say the little dude has inspired.

Life as the President of MLA: Part 1

I’m baaaaaaaaaack.

It has been a while since I have posted.  I want to thank all of the guest posters who have written and kept things going while I took a year off.  You guys were great.

So, one of the questions I have gotten over the year is what is like to be President of MLA.  I want to take the time to answer that question.  I will try to provide as much information without getting so wordy that it becomes the longest blog post ever.  With that in mind I am going to make this a two part series.  The first part is going to focus on my experiences while the second part will get into the nitty gritty of time and how things are done.

First and foremost it was an awesome experience.  I wish everyone who wants to run as President will get to be President.

Meeting Librarians:

Attending the Chapter Meetings and the CHLA/ABSC meeting provided me with a rich opportunity to meet more librarians than I would have ever met in my lifetime.   Meeting people and sharing stories and experiences, brainstorming ideas, learning from each other is the one thing, hands down, that made the whole Presidential term great.  Unfortunately I wasn’t able to attend every Chapter’s meeting, they all overlap a great deal.  However, I want to still attend every Chapter’s meeting at least once.  I learned so much from different groups that I feel like it will do me good to step out and go to a different Chapter every once and a while.  I think we are sometimes so wrapped up in our own groups that we don’t always know what is going on elsewhere and big MLA annual meeting can sometimes be information overload.  I kind of got this idea from a librarian friend whose child was graduating college during her Chapter meeting.  She decided to go to her child’s graduation and then go to a different Chapter’s meeting (which didn’t conflict with the graduation) and present a poster.  So a few years down the road if I happen to be at your Chapter meeting, I am just branching out and expanding my boundaries.

Capitol Hill Visits:

I was both equally nervous and excited when I joined the AAHSL and MLA Joint Legislative Task Force in Washington DC.  Because NLM and the NIH are government entities they cannot lobby on behalf of themselves.  Who is to lobby for the things we medical librarians use? Well, medical librarians will lobby on their behalf.  The first day we met to learn and discuss all of the most relevant and timely issues that we needed to make our Senators and Representatives aware of.  We were given packets and created talking points.  The next day we were set loose in 3 teams of 3-4 people.  We met with the staffers responsible for health affairs for the Senators or Representatives of our states.  It helped to be in a group because the members in your group chimed in and provided support while you were talking with your state’s staffer.  It was such a cool experience walking the halls of the Senate and House buildings.  I felt a part of the governmental process that I learned about when I was a kid in school.

MLA Historical Marker:

I got to be a witness to library history.  Last November I went to Philadelphia to attend the unveiling of the MLA Historical Marker  at 1420 Chestnut Street (later designated 1420-1422 Chestnut Street).  MLA is the world’s oldest association of medical librarians and information professionals!  To be a part of such a respected and long standing association and to see it being recognized by the Pennsylvania Historical and Museum Commission was humbling and wonderful at the same time.

MLA Business:

I am still honored that I was elected by the membership to serve as President and to guide MLA forward as an association.  Associations and organizations are changing rapidly.  It was a great privilege to be a part of the process as MLA members, Board members and staff worked to help the association evolve to meet the needs of future medical librarians.  All of us worked together to look at things from a different perspective and not rely on the old “we always did it that way” principle.  Rome was not built in a day and we have had some hiccups along the way but to be a part of the evolutionary process is something that I hold near and dear to my heart.  As I went to the Chapter meetings, Board meetings, and spoke or emailed with so many people in the association, I always did my best to try and understand people and perspectives and to help the association. I love the people in this organization and I believe that we all are working to make things great.

Family:

If you were at the Toronto meeting you might have noticed that I teared up a little and my voice cracked when I thanked my family for their support.  I am not a crier.  I’m a suck it up kind of person. Being MLA President provided a wonderful personal experience for me as well.  My kids have always known that I help doctors and nurses find information to help treat patients.  During my term as President they were able to see medical librarianship beyond my day to day job.  They would see me chatting on Twitter with other librarians from all over the world (*see funny story below).  When I left for Chapter meeting I would tell them where I was going and what I would be doing.  My trip to Philadelphia for the historical marker gave me the opportunity to talk to them about the history and evolution of medical librarianship.  Finally, they were able to hear about their mom going to Washington DC to talk to our elected officials (well their staffers) to try and influence positive change. Through all of this they saw first hand of what “Oh, the Places You’ll Go!” can be.

My experience as MLA President will stick with me forever.  I enjoyed and learned so much during my time.  However, I am glad to slow down just a bit and be the past President now.  My kids will be sad that they won’t get as much Garrett’s popcorn, but I am ready for the next phase of my career (whatever that may be) to begin.

*Funny story*
One Thursday evening while participating in the #medlibs chat, my son asked me who I was texting.  I told him I was Tweeting and talking with other medical librarians around the US and the world.  He paused a second then asked if I knew every medical librarian in the US in the world.  I said no of course not.  He then asked if I had met the people I was chatting with.  I told him that I had met some but not all.  He looked at me and said, “I thought you told us we weren’t supposed to chat online with people we haven’t met in real life. You should be careful Mom.” -I got a dose of my own internet safety speech from my kid.

Ithaka S+R Local Faculty Survey and Health Sciences Libraries:

In 2014, the Virginia Commonwealth University (VCU) Libraries administered the Ithaka S+R Local Faculty Survey to our faculty to measure their changing needs and perceptions of library resources. The survey, consisting of seven modules including the health sciences module, was distributed across our campus. The health sciences module targets faculty with patient or clinical care responsibilities. At this time, few health sciences libraries have used this instrument. Survey questions focused on attitudes and skills related to evidence-based scholarly resources as well as access and use of other library services and resources.

Of course we all know students’ research skills especially in finding evidence based scholarly research are often lacking. This came out clearly from the results of the Ithaka survey. Again we were not surprised to see that faculty also see these research skills as a very important aspect of the students’ learning. However, it is still amazing that a large number of faculty did not see teaching the skill of finding evidence based scholarly materials and research skills as primarily the librarians’ function. A timely reminder for us to continue informing our faculty that we indeed have are more than capable to teach students research skills especially when it comes to evidence based practice. What a great opportunity for us to collaborate with our faculty and remind them about everything else we bring to the table!

Hopefully more health sciences libraries will use this survey instrument to measure their faculty perceptions because I think it would be interesting to compare the VCU’s Tompkins-McCaw Library’s findings with other libraries that have surveyed their health sciences faculty.

Post Publication Review: Librarians’ Role

On Monday I spoke to a group of physicians, hospital administrators and other medical professionals on the impact of the publishing industry on hospitals and medicine.  While I spoke about the elephant in the room, sky high subscription rates for institutions, I also spoke about the role of post publication review in medical literature.

The example I gave was Amanda Capes-Davis who comments within PubMed Commons on mistaken identities of cell lines within the medical literature and her efforts to inform readers of potential cell line problems.

I wish I had seen Melissa Rethlefsen’s PubMed Commons post when I was creating my presentation.  It is a great example of how medical librarians can examine the published literature for inconsistencies regarding the methodologies of their search of the literature when conducting research.

Melissa reviewed the article “Comparative efficacy and safety of blood pressure-lowering agents in adults with diabetes and kidney disease: a network meta-analysis.” Lancet. 2015 May 23;385(9982):2047-56. doi: 10.1016/S0140-6736(14)62459-4.  She and her colleagues at the University of Utah Spencer S. Eccles Health Sciences Library reviewed and tried to duplicate the authors’ Embase search strategy which the authors reported in the Appendix (pages 13-14). According to the PRISMA flow chart the authors retrieved 1,371 results (Appendix page 37).

According to Melissa,

This study highlights the need for more accurate and comprehensive reporting needed for search strategies in systematic reviews and other literature search-based research syntheses, and the need for better peer review of search strategies by information specialists/medical librarians. Though the searches in the Appendix are on face value replicable and high quality, on closer inspection, they do not in fact meet the reporting standards as outlined by PRISMA Statement items #7 and #8: “Describe all information sources in the search (e.g., databases with dates of coverage, contact with study authors to identify additional studies) and date last searched” and “Present the full electronic search strategy for at least one major database, including any limits used, such that it could be repeated.”

For me, this comment within PubMed Commons highlights the need for librarians to analyze search strategies in the literature and to speak up and set the record straight when things are not correct or there are issues of reproducibility.  Just like Amanda Capes-Davis who sheds light on cell line problems or the statisticians who questioned the math in an NEJM article (later retracted), we are subject experts and it is important that we help contribute to post publication peer review.

Medical librarians all around the world can point to examples of when a poor literature search could have saved lives or prevented injuries, death or illness.  I am not suggesting the literature review in the article by Palmer et al. could cause patient harm.  But PubMed Commons provides librarians with an avenue by which to question literature reviews presented in research.  Hopefully by highlighting questionable search strategies or corroborating effective search strategies it will lead to better use of librarians and better research all around.

Doing Something Neat with Technology? Submit Your Project to JMLA Virtual Projects

The Journal of the Medical Library Association (JMLA) Virtual Projects Committee is seeking innovative and notable technology projects for the October 2016 JMLA Virtual Projects column. The annual column focuses on library virtual spaces that extend the library “presence” outward to support users in their digital spaces, wherever and whenever needed.

To be considered for this column, please submit a 200-word abstract of your virtual project or a link to your project web page that describes the project and why it is innovative or notable. Technology projects must have been implemented within the past two years. Send your submissions to Susan Lessick  slessick[atsign]uci[dot]edu AHIP, FMLA, by March 17.

Submissions of virtual projects may demonstrate the implementation of a new technology or a new application of an older technology. Focus areas or technologies of special interest include, but are not limited to:

  • electronic medical record (EMR)/electronic health record (EHR) integration and “meaningful use” programs
  • data management
  • data visualization
  • assessment metrics
  • gamification
  • flipped classroom
  • adaptive learning technologies
  • virtual reality (VR)
  • beacons
  • social media technologies

    Please consider sharing your knowledge and experiences with implementing virtual projects in your library to inspire and encourage your peers, partners, and communities.

    JMLA Virtual Projects Advisory Committee: Patricia Anderson; Janis F. Brown, AHIP; Michelle Kraft, AHIP; Susan Lessick, AHIP, FMLA (column editor); and Elizabeth Whipple, AHIP.

NLM Georgia Biomedical Informatics Course

I recently attended the NLM Georgia Biomedical Informatics Course at the lovely Brasstown Valley Resort in Young Harris, GA. This week-long semiannual course is hosted by the Robert B. Greenblatt, M.D. Library, Georgia Regents University and funded by the National Library of Medicine. If you’ve ever heard library colleagues talk about the Woodshole course, this is the current version of that course. The content changes every session, which is necessary in such a fast moving field.

Attendees were a nice mix of librarians, clinicians, researchers and others involved in medical information technology. Instructors who are in the forefront of their field came from around the country to teach in this prestigious course. I found it to be a great overview of current important topics in informatics, and I learned so much about the breadth of this essential field from both the instructors and the other attendees. We also did some networking and shooting pool at the local watering hole, Brassies.

Read more to see what was covered (and some cool pictures from a field trip we took)

Continue reading NLM Georgia Biomedical Informatics Course