Tag: text mining
Team Awarded Grant to Help Digital Humanities Scholars Navigate Legal Issues of Text Data Mining
We are thrilled to share that the National Endowment for the Humanities (NEH) has awarded a $165,000 grant to a UC Berkeley-led team of legal experts, librarians, and scholars who will help humanities researchers and staff navigate complex legal questions in cutting-edge digital research.
What is this grant all about?
If you were to crack open some popular English-language novels written in the 1850’s–say, ones from Brontë, Hawthorne, Dickens, and Melville–you would find they describe men and women in very different terms. While a male character might be said to “get” something, a female character is more likely to have “felt” it. Whereas the word “mind” might be used when describing a man, the word “heart” is more likely to be used about a woman. Yet, as the 19th Century became the 20th, these descriptive differences between genders actually diminish. How do we know all this? We confess we have not actually read every novel ever written between the 19th and 21st Centuries (though we’d love to envision a world in which we could). Instead, we can make this assertion because researchers (including David Bamman, of UC Berkeley’s School of Information) used automated techniques to extract information from the novels, and analyzed these word usage trends at scale. They crafted algorithms to turn the language of those novels into data about the novels.
In fields of inquiry like the digital humanities, the application of such automated techniques and methods for identifying, extracting, and analyzing patterns, trends, and relationships across large volumes of unstructured or thinly-structured digital content is called “text data mining.” (You may also see it referred to as “text and data mining” or “computational text analysis”). Text data mining provides humanists and social scientists with invaluable frameworks for sifting, organizing, and analyzing vast amounts of material. For instance, these methods make it possible to:
- Detect racial disparity by evaluating language from police body camera footage;
- Develop new tools to enable large-scale analysis of television series and photographs; and
- Capture and design new physical representations of naturally occurring laughter
The Problem
Until now, humanities researchers conducting text data mining have had to navigate a thicket of legal issues without much guidance or assistance. For instance, imagine the researchers needed to scrape content about Egyptian artifacts from online sites or databases, or download videos about Egyptian tomb excavations, in order to conduct their automated analysis. And then imagine the researchers also want to share these content-rich data sets with others to encourage research reproducibility or enable other researchers to query the data sets with new questions. This kind of work can raise issues of copyright, contract, and privacy law, not to mention ethics if there are issues of, say, indigenous knowledge or cultural heritage materials plausibly at risk. Indeed, in a recent study of humanities scholars’ text analysis needs, participants noted that access to and use of copyright-protected texts was a “frequent obstacle” in their ability to select appropriate texts for text data mining.
Potential legal hurdles do not just deter text data mining research; they also bias it toward particular topics and sources of data. In response to confusion over copyright, website terms of use, and other perceived legal roadblocks, some digital humanities researchers have gravitated to low-friction research questions and texts to avoid decision-making about rights-protected data. They use texts that have entered into the public domain or use materials that have been flexibly licensed through initiatives such as Creative Commons or Open Data Commons. When researchers limit their research to such sources, it is inevitably skewed, leaving important questions unanswered, and rendering resulting findings less broadly applicable. A growing body of research also demonstrates how race, gender, and other biases found in openly available texts have contributed to and exacerbated bias in developing artificial intelligence tools.
The Solution
The good news is that the NEH has agreed to support an Institute for Advanced Topics in the Digital Humanities to help key stakeholders to learn to better navigate legal issues in text data mining. Thanks to the NEH’s $165,000 grant, Rachael Samberg of UC Berkeley Library’s Office of Scholarly Communication Services will be leading a national team (identified below) from more than a dozen institutions and organizations to teach humanities researchers, librarians, and research staff how to confidently navigate the major legal issues that arise in text data mining research.
Our institute is aptly called Building Legal Literacies for Text Data Mining (Building LLTDM), and will run from June 23-26, 2020 in Berkeley, California. Institute instructors are legal experts, humanities scholars, and librarians immersed in text data mining research services, who will co-lead experiential meeting sessions empowering participants to put the curriculum’s concepts into action.
In October, we will issue a call for participants, who will receive stipends to support their attendance. We will also be publishing all of our training materials in an openly-available online book for researchers and librarians around the globe to help build academic communities that extend these skills.
Building LLTDM team member Matthew Sag, a law professor at Loyola University Chicago School of Law and leading expert on copyright issues in the digital humanities, said he is “excited to have the chance to help the next generation of text data mining researchers open up new horizons in knowledge discovery. We have learned so much in the past ten years working on HathiTrust [a text-minable digital library] and related issues. I’m looking forward to sharing that knowledge and learning from others in the text data mining community.”
Team member Brandon Butler, a copyright lawyer and library policy expert at the University of Virginia, said, “In my experience there’s a lot of interest in these research methods among graduate students and early-career scholars, a population that may not feel empowered to engage in “risky” research. I’ve also seen that digital humanities practitioners have a strong commitment to equity, and they are working to build technical literacies outside the walls of elite institutions. Building legal literacies helps ease the burden of uncertainty and smooth the way toward wider, more equitable engagement with these research methods.”
Kyle K. Courtney of Harvard University serves as Copyright Advisor at Harvard Library’s Office for Scholarly Communication, and is also a Building LLTDM team member. Courtney added, “We are seeing more and more questions from scholars of all disciplines around these text data mining issues. The wealth of full-text online materials and new research tools provide scholars the opportunity to analyze large sets of data, but they also bring new challenges having to do with the use and sharing not only of the data but also of the technological tools researchers develop to study them. I am excited to join the Building LLTDM team and help clarify these issues and empower humanities scholars and librarians working in this field.”
Megan Senseney, Head of the Office of Digital Innovation and Stewardship at the University of Arizona Libraries reflected on the opportunities for ongoing library engagement that extends beyond the initial institute. Senseney said that, “Establishing a shared understanding of the legal landscape for TDM is vital to supporting research in the digital humanities and developing a new suite of library services in digital scholarship. I’m honored to work and learn alongside a team of legal experts, librarians, and researchers to create this institute, and I look forward to integrating these materials into instruction and outreach initiatives at our respective universities.”
Next Steps
The Building LLTDM team is excited to begin supporting humanities researchers, staff, and librarians en route to important knowledge creation. Stay tuned if you are interested in participating in the institute.
In the meantime, please join us in congratulating all the members of the project team:
- Rachael G. Samberg (University of California, Berkeley) (Project Director)
- Scott Althaus (University of Illinois, Urbana-Champaign)
- David Bamman (University of California, Berkeley)
- Sara Benson (University of Illinois, Urbana-Champaign)
- Brandon Butler (University of Virginia)
- Beth Cate (Indiana University, Bloomington)
- Kyle K. Courtney (Harvard University)
- Maria Gould (California Digital Library)
- Cody Hennesy (University of Minnesota, Twin Cities)
- Eleanor Koehl (University of Michigan)
- Thomas Padilla (University of Nevada, Las Vegas; OCLC Research)
- Stacy Reardon (University of California, Berkeley)
- Matthew Sag (Loyola University Chicago)
- Brianna Schofield (Authors Alliance)
- Megan Senseney (University of Arizona)
- Glen Worthey (Stanford University)
What a semester! What’s up next?
Is it just us, or was fall semester a whirlwind? The Office of Scholarly Communication Services was steeped in a steady flurry of activity, and suddenly it’s December! We wanted to take a moment to highlight what we’ve been up to since August, and give you a preview of what’s ahead for spring.
We did the math on our affordable course content pilot program, which ran for academic year 2017-2018 and Fall 2018. This pilot supported just over 40 courses and 2400 students, and is estimated to have yielded approximately $200,000 in student savings. We’ll be working with campus on next steps for helping students save money. If you have questions about how to make your class more affordable, you can check out our site or e-mail us.
We dug deep into scholarly publishing skills with graduate students and early career researchers during our professional development workshop series. We engaged learners in issues like copyright and their dissertations, moving from dissertation to first book, and managing and maximizing scholarly impact. Publishing often isn’t complete without sharing one’s data, so we helped researchers understand how to navigate research data copyright and licensing issues at #FSCI2018.
We helped instructors and scholars publish open educational resources and digital books with PressbooksEDU on our new open books hub.
On behalf of the UC’s Council of University Librarians, we chaired and hosted the Choosing Pathways to OA working forum. The forum brought together approximately 125 representatives of libraries, consortia, and author communities throughout North America to develop personalized action plans for how we can all transition funds away from subscriptions and toward sustainable open access publishing. We will be reporting on forum outcomes in 2019. In the meantime, one immediate result was the formation of a working group to support scholarly society journal publishers in flipping their journals from closed access to open access. Stay tuned for an announcement in January.
We funded dozens of Open Access publications by UC Berkeley authors through our BRII program.
We developed a novel literacies workflow for text data mining researchers. Text mining allows researchers to use automated techniques to glean trends and information from large volumes of unstructured textual sources. Researchers often perceive legal stumbling blocks to conducting this type of research, since some of the content is protected by copyright or other use restrictions. In Fall 2018, we began training the UC Berkeley community on how to navigate these challenges so that they can confidently undertake this important research. We’ll have a lot more to say about our work on this soon!
Next semester, we’re continuing all of these efforts with a variety of scholarly publishing workshops. We invite you to check out: Copyright & Fair Use for Digital Projects, Text Data Mining & Publishing: Legal Literacies, Copyright for Wikipedia Editing, and more.
We would like to thank Arcadia, a charitable fund of Lisbet Rausing and Peter Baldwin, for their generous support in helping to make the work of the Office of Scholarly Communication Services possible.
Lastly, we’d like to thank all of you for your engagement and support this semester! Please let us know how else we can serve you. In the meantime, we wish you a Happy New Year!
E-mail: schol-comm@berkeley.edu
Twitter: @UCB_scholcomm
Website: lib.berkeley.edu/scholcomm
HathiTrust Research Center (HTRC) UnCamp Fellowships
Literatures and Digital Humanities Librarian
438 Doe Library | University of California, Berkeley | Berkeley, CA 94720
sreardon@berkeley.edu
Where to Find the Texts for Text Mining
Text mining, the process of computationally analyzing large swaths of natural language texts, can illuminate patterns and trends in literature, journalism, and other forms of textual culture that are sometimes discernible only at scale, and it’s an important digital humanities method. If text mining interests you, then finding the right tool — whether you turn to an entry-level system like Voyant or master a programming language like Python — is only a part of the solution. Your analyses are only as strong as the texts you’re working with, after all, and finding authoritative text corpora can sometimes be difficult due to paywalls and licensing restrictions. The good news is the UC Berkeley Libraries offer a range of text corpora for you to analyze, and we can help you get your hands on things we don’t already have access to.
The first step in your exploration should be the library’s Text Mining Guide, which lists text corpora that are either publicly accessible (e.g., the Library of Congress’s Chronicling America newspaper collection) or are available to UCB faculty, students, and staff (e.g., JSTOR Data for Research). The content of these sources are available in a variety of formats: you may be able to download the texts in bulk, use an API, or make use of a content provider’s in-platform tools. In other cases (e.g., ProQuest Historical Newspapers), the library may be able to arrange access upon request. While the scope of the corpora we have access to is wide, we are particularly strong in newspaper collections, pre-20th century English literature collections, and scholarly texts.
What happens if the library doesn’t have what you need? We regularly facilitate the acquisition of text corpora upon request, and you can always email your subject librarian with specific requests or questions. The library will deal with licensing questions so you don’t have to, and we’ll work with you to figure out the best way to make the texts available for your work, often with the help of our friends in the D-Lab or Research IT . We also offer the Data Acquisition and Access Program to provide special funding for one-time data set purchases, including text corpora. Your requests and suggestions help the library develop our collection, making text mining easier for the next researcher who comes along.
Important caveats:
- Unless explicitly stated, our contracts for most Library databases and library resources (e.g., Scopus, Project MUSE) don’t allow for bulk download. Please avoid web scraping licensed library resources on your own: content providers realize what is happening pretty quickly, and they react by shutting down access for our entire campus. Ask your subject librarian for help instead.
- Keep in mind that many of the vendors themselves are limited in how, and how much access, they can provide to a particular resource, based on their own contractual agreements. It’s not uncommon for specific contemporary newspapers and journals to be unavailable for analysis at scale, even when library funding for access may be available.
Related resources:
- Library Text Mining Guide
- Library Data Acquisition and Access Program
- D-Lab Computational Text Analysis Working Group
- D-Lab Learn Python Working Group
Stacy Reardon and Cody Hennesy
Contact us at sreardon [at] berkeley.edu; chennesy [at] berkeley.edu