Introduction

Tom English:

It's my pleasure today to be joined by three academics. We've got Professor Jaap Verheul and Bob Nicholson [with Tom Tunstall-Allcock to join shortly], so Jaap and Bob please could you introduce yourselves, respectively, just in terms of your backgrounds, and which areas of teaching and research you are really interested in.

Jaap Verheul:

My field is cultural history originally, I'm interested in the way cultures influence each other, look at each other, and--let's say--mental images of each other; and from there my specialization is in transatlantic studies. That's how I met Bob Nicholson, because the first huge digital humanities project I was involved with looked at the way--in the Netherlands--images of the United States were to fill up the newspapers. So my digital identity is very much in newspaper research; I'm very interested to debate how you use digital tools to look at long-term changes and developments in public rhetoric using digitized newspapers. So: American studies, transatlantic studies, digital methodologies; that space, as it were.

Tom English:

Super, thanks Jaap. If we go on to Bob (and welcome Tom Tunstall as well), could you give us a quick intro in terms of what areas you focus on as well? We will do a round table here to start things off before we dig into the meat and drink of the conversation.

Bob Nicholson:

My name is Bob Nicholson, I'm--I guess--a cultural historian too: I could basically do the same intro as Jaap, it's very similar topics! I focus on the history of the 19^th century, the history and culture of that time period, and I'm particularly interested in Victorian popular culture in all of its forms. The project I'm working on at the moment is exploring the history of Victorian humour and comedy, which crops up in all sorts of unusual places and all sorts of digital archives. We'll get into this in the discussion, but digital archives have been really central to the way that I've been trying to unearth and deconstruct the comic culture of the 19th century. So yeah, that's me--and I also use digital archives constantly in my teaching as well, that may also come up.

Thomas Tunstall-Allcock:

Sorry for the late joining, I've come straight from a lecture on campus! I'm Tom Tunstall-Allcock, I'm an Americanist and a modern Americanist. I probably started as a diplomatic historian, and that's still broadly the area that I work in, focusing particularly on the American presidency. I think-- hopefully--like some other people's work here as well, I’m increasing the interest in cultural history and the cultural history of the Presidency, and basically the cultural history of presidential diplomacy. The project I'm working on at the moment is about presidential entertaining and diplomacy, particularly looking at the hosting of foreign leaders, and the traditional political aspects of that; but I’m also really interested in the things around it that have often been dismissed --the kind of ceremony and pomp, but also the entertaining, the food, the body language, the decoration, style--when hosting is taken away from Washington to various different locations too. Essentially, that cultural side of it, and really tying it into how that relates to bigger political questions. In terms of the sort of archives that I'm using, I'm looking at traditional presidential and political archives, but also any way that I can get any of the colour and detail about what's going on at these events. So I'm drawing on as wide a range as I can; in-person archives and also digital archives.

Tom English:

Super, thank you Bob, and Tom as well for adding to that. Bob, I've got to say I've just remembered that you've got a fantastic Twitter account that I think our viewers will be very interested in. In terms of Victorian culture, you share a lot of that on Twitter, so feel free to shout out your handle for Twitter, I'm sure there will be interest in following that.

Bob Nicholson:

Thank you, I never refuse a chance to plug this! There are two actually. My main research one is @DigiVictorian, and that's largely filled with general things I'm researching. At the moment I'm mostly doing things based on physical copies of books, but there's a lot of stuff from digital archives on there. And there is a spin-off project which is at @VictorianHumour (spelled the British way with a 'u'), and pretty much every day we post a joke from a Victorian newspaper or magazine. I can't promise they'll be funny, but they're kind of weird and interesting and curious, so if you like terrible puns and you want to inflict them on your family, by all means go to @VictorianHumour or @DigiVictorian. I almost live blog my research as I do it there, so as I'm discovering things in archives, or sometimes when I'm teaching them, I try to share it there as I do it. So it gives you a sense of what I'm doing with these sources.

Tom English:

Just on that point, I'm glad you mentioned that, as that reminded me that you did a thread on how to use the tools in Gale Primary Sources, and how to use them well. Things like proximity, just in the Gale Primary Sources platform, and that was extremely useful because it's always good to be able to get that information out there to the student body. We might know how to use these things, but students don't necessarily know how to do that. So that was really helpful, really useful.

How have Gale Primary Sources impacted your research?

Tom English:

So I want to just have a little bit fun: Bob's into jokes, I don't know if everybody else is, but one of the things that really struck me--and this is something I share with colleagues all the time--is when, sitting down and talking with academics and talking about dissertations, for example, something that came up (I think it was Dave Appleby at Nottingham University), he said to me that every single year students go to him and say: 'how can I get first on my dissertation?'; and the answer is simple: 'tell me something that I don't already know'.

Now we all know that the only way to do that with any credibility is through primary sources, because primary sources are the bricks and the mortar that build the proverbial house, if you want to look at it like that. So my first question, and we can go around in the same order again, is: what is something new that you have learned from Gale Primary Sources, from Gale's collections?

Jaap Verheul:

That's a good question, and it will take me several hours to answer this, because the Gale collection has been so central to my digital humanities research and identity. But I must confess that, in my research, I mostly use the back-end of the Gale collections in the university subscription that we have. Gale gave us also the hard-disk metadata, and that's what we have used in many projects--also in comparative history, and in building tools that are seminal.

I too have a claim to fame here. With my postdoc I published an article, based on the Gale collection of The Times, that analyzes 100 years of migration debate in the United Kingdom. And that's the thing that you can only do with digital sources--because combining distance reading, finding patterns in the way migration and immigration and emigration are discussed in The Times--to combine that with close reading gives you a perspective on a very important debate in the United Kingdom that you could not have distilled from that source with other means. It's called 100 Years of Migration Discourse in The Times. So my point here is that digital sources, especially newspapers, can do two things that can help you find needles in haystacks. So a person's name, or event, that otherwise you would not be able to find. What I've been engaged with is the other way around. I look at distant reading methodologies, look at large questions that you compose, and in doing so can formulate and do big history; so there is the largeness of the debate that, without digital methodologies, you could never achieve. So that's my take from digital humanities.

Tom English:

That's very salient as well. As you know the work we do at Gale--you mentioned that the TDM [hard-disk] drives that you've gotten--also we've got the Digital Scholar Lab now, so very salient, thank you for that. On to Bob and then Tom please...

Bob Nicholson:

I guess I took similar approaches as well for the distant reading of articles; but to give you an example more of the ‘needle in the haystack’ side of things, which, is also--particularly for students--I think one of the ways that they can really do accessible original research. This is an example from quite a lot of years ago, but it's when I first started becoming interested in the history of comedy, and where I actually (almost by accident) happened upon--this was actually in a physical archive--happened upon a column of jokes imported from the United States that was published in a British newspaper. I remember seeing that and thinking 'that's kind of odd, I wonder if there are more of them?'. And that's where I went into digital archives and I started searching for a whole range of different keywords to try and find examples of imported jokes, jokes written in the United States, published in magazines there, that were then reprinted in Britain. And what was really, phenomenally surprising was that, by using keyword searches, I was able to find not just a few examples, but thousands and tens of thousands of these imported joke columns. And nobody had ever looked at them before. Partly because they were scattered around; they're hidden away in the corners of the archives. So what I was able to do is, firstly, capture the scale of this sort of cultural phenomenon; the idea that the Victorians were importing American humour, way before Hollywood cinema became this dominant form of imported American culture. And then--this is the ultimate needle in the haystack--I was able to start tracing the viral circulation of individual jokes to find ones that were published in America, passed from newspaper to newspaper, the way that we now pass them around on Twitter or on WhatsApp, and to be able to track that; and not get the full picture, but roughly see that movement. That's something that we absolutely could never have done in a million years with physical archives, and it really does feel like my research is just filled with moments like that, where I'm able to find things I would not have been able to find before. The great thing about it is that your students can absolutely do the same. There is so much out there--little things like that, that we don't know about--in newspaper archives. People say, whatever it is, 90% of newspapers haven't been digitized: but in the 10% that have, 99% of that has never been looked at. So the sheer amount of new stuff that can be found--that needle and haystack stuff--is phenomenal; so there's loads more to do.

Tom English:

Yes, plenty yet to be digitized but, like you say Bob, that's a really good point in terms of who's actually looked at this stuff, and how they got into this; and I love the way that you approach that in terms of building on what Jaap had said, looking at the 'distant' element, distant reading, and looking at the macro; but historians are never going to give up the micro, they're never going to give up close reading, they're never going to move away from wanting to look at the detail and the granularity, because some of those details can be extremely important in new information and new narratives to challenge the historiography as well.

Thomas Tunstall-Allcock:

I definitely agree with your point on the macro, because I've moved to a project that was looking at a much longer time period: I don't think I could have done the project I'm doing now with 'pre-digital' research, it would just be impossible in terms of the scale of sources I'm using.

To take a little slightly different tack on it, what really helped me--with having access to so many digital sources—was trying to slightly shift how I was working, and the types of sources I was looking at, and the type of history I was writing. I've been trained in a way where I go through these sort of sets of archives--the presidential libraries, maybe the national archives, a certain set of political files—and this is how you work. And then you realize you want to do a new project as an academic, and you don't have a PhD supervisor anymore, you don't have anyone to really tell you how to do this. So where do you start? If you work in a country and you can't start popping to the archive itself, the place where you go to start trying to teach yourself how to do any sort of new type of history, is to start playing around with digital archives you have access to, and looking for how you might even start to think about a new project. And so, for me, something I found really helpful in the early stages of figuring out this project was having access to the Associated Press archives.

Whereas I was previously looking at things from the official government perspective, when I started looking at events--and particularly in terms of the hosting of things that are outside of Washington--I realized there are masses of reports. Members of the Associated Press, who were part of the White House press corps, would send these huge reports back detailing menus, entertainment, events--colour about events--and giving a whole different perspective on these events I've been looking at from this much more official perspective. And that opened things up to allow me to have a think about how this project might even look; and a lot of things I found in that first set of searches ended up being the first article that came out of my project--about Lyndon Johnson's posting and cultural diplomacy--because it was there that I could find that he was entertaining foreign leaders with these gigantic barbecues, and it was listing kind of the gallons of beer and beans and sides of beef and things that were being done.

Tom English:

Texas Style!

Thomas Tunstall-Allcock:

Texas Style, exactly that: and I never would have got that centering--of really thinking that this is where our project could go-- without being able to just sit and spend some time searching, keyword searching, seeing what's there, to push on what my training had been before. I think that applies--as you said already--a lot of this applies to students; and we might talk about this more in terms of how that works with students. But certainly, as a researcher, you can't do without that visual access.

Tom English:

Brilliant, thanks Tom. I'm glad the Associated Press has been such a useful source; I remember talking to you about that when it was first released, and we were talking about the usefulness of it and how it might be used. Also, I'm glad to hear that our searching platform seems to be working well if you got such a great result off of the first search. So that's really, really good to hear.

How do you use Gale Primary Sources in your teaching?

Tom English:

So we've heard about your research now, we've heard about how you have personally been able to use the Gale archives. I'm interested now to talk about how you use the Gale platforms and collections in the classroom: how this relates to teaching and learning, how you convey this to your students in practice.

Just to set the scene a little bit on this. I am back on the road now, so I'm speaking with academics across the land about different platforms and archives, and I had an interesting conversation with an academic at Leeds last week. I asked him about doing sessions in classes, and what do you do in relation to talking to people about the platforms and everything else. His response was that the Gale Primary Sources--the collections themselves, and the platforms--are pretty indispensable to his work in teaching and learning; however, his point is that he keeps it out of the classroom. He'll send people through to links in the Virtual Learning Environment so that people can get access to the sources outside, in their own time; for example, instead of killing a rainforest to give people sheets of primary sources, he'll direct them to the V.L.E. so they can get it digitally. His view was that he doesn't want technology in the classroom because it means that people aren't talking, people aren't discussing the source material, people aren't discussing the historiography. So that's just a little bit of background for this question, but if we could maybe go around again, and again we'll keep the same model. So if we start with Jaap: how is it--it might be a bit different for you as I know that you're more interested in the macro--but how is it that you use Gale Primary Sources platforms and collections in the classroom?

Jaap Verheul:

Well, I have two answers to the question. First of all, the collections are ideal to give students small assignments, and to let them discover both the possibilities--but also the limitations--of digital research. For instance, an assignment could be to trace a person's name, and see how they developed. Of course, not Lyndon Johnson [for example], because he's too ubiquitous; but smaller figures, to see how--in newspapers--somebody is traced. They'll find that there are ambiguities in names, and that they will find terms that they hadn't expected. That's the starting point to the second issue that I always address with students, because I think it's essential that they always keep a critical perspective on the sources that they find.

So my aim is to teach them digital literacy: to be as critical of digital sources as they would be of normal primary sources. So as a historian I teach my students, from day one, a critical source attitude. If you have a source, a historian should ask the fundamental questions: Who wrote it? What was the audience? What's the platform? What is the intention of the author and writing? To have a conceptual idea of the context in which a source is produced and reused. What I find is one of the challenges, is that students tend to lose that critical attitude if they find a source in a digital collection; and I think that points at a huge challenge for the learning environments that we now have, because if they put a search string into the environment, they will find an individual--or a set of--results that are out of context. Especially if they use the Digital Scholar Lab: it will be able to find sources from different newspapers from archival collections, and the trick is always to make students aware of the context from which a source derives; and that's a little bit challenging but also brings in elemental detective work to their work. A few years ago I was coordinator over a large course on methods and methodologies for first-year history students, and we talked a lot about the way digital collections are formed. We had the great fortune to have Paul Gazzolo and Lena Gerle from [Gale] come to our class to discuss how [Gale] makes strategic decisions about forming the collection, selecting topics--for instance, why newspapers? Why that newspaper?--and the way [Gale] approaches the whole digitization process. I find that a wonderful opportunity, because that gives students the opportunity to look 'behind the scenes', so to speak, and to talk to the people that are responsible for the decisions that eventually lead to the collections that they use. And again, normally, if you send students into an archive, that's the question that they've all immediately asked: Who formed this archive? What's in there? What's not in there? What's the power structure that produced this archive? Whereas with digital collections, I find that it takes an extra effort to bring that critical attitude to students.

Tom English:

I love that answer Jaap, about the critical thinking skills in relation to what's digital, because sometimes people think that just because it's on the Internet, it's true: and this is a big problem that we have in relation to things like fake news as well, and misinformation that is spread. This seems to be a blind spot: maybe things look more 'official' because they're on a screen or because somebody typed it up and they've marketed it well. But there is, like you say, also that problem of context, and that issue of context. This has a big pedagogical implication in terms of how you introduce people to the archive, so it's not necessarily dusting off the cobwebs in the physical archives (although it will be, I'm sure you'll still use the physical archives); but it's when you're using the digital things: the digital things are convenient, but they're not the be-all and end-all, and there are caveats to be applied to them as well, and there are specific things that need to be looked at in terms of making them useful and in full context. Good stuff: Bob please.

Bob Nicholson:

I use digital archives a lot in all of my teaching. Sometimes I take the approach that you mentioned from that colleague at Leeds: just getting people to use them in advance as a way to prep for sessions. For the most part, I guess I've taken almost the exact opposite approach, which is to do all of my classes in computer labs. I initially used to just do one-off sessions where I would say, 'right, here's the week where we teach you how to use these archives'; and generally what happened is that students either forgot that they existed, or that they never really got to grips with the real detail involved in using the archives. For me, if I really want students to make these things integral to their practice as historians, then they have to be integral to the environment in which we teach. I prefer to teach in computer labs and what I can do, then, is really get students familiar with these resources. We use them every week, and sometimes it might just be a case of just doing a little bit of research in response to something that's happening that I've raised in the lecture; sometimes it might be just chasing up a question that we've looked at. A lot of time, it also allows me--over the course of, let's say, 12, or actually in the case of one module, 24 weeks--I can really build up their digital search skills.

What I realized was, if I was just leaving students to search these archives on their own, they were treating them the way they would search Google. Ultimately these are very different things: the search literacy that they've developed--with all the kind of bells and whistles that Google has that interpret what it thinks you mean--do not apply to the digital archives. What I have to do is really teach them, bit by bit by bit, how to select and approach keywords, how to combine them in effective ways, what will work, what won't, and all of that stuff. Some students will pick that up naturally over just experience using it; but in my experience, the vast majority of my students need that taught to them, and they need to have some guidance on how to fix problems when your search isn't working, how to deal with getting a million results. Those are skills that I feel it's my responsibility as a teacher to teach, not just assume they will acquire.

So they're really central to all of my teaching, to the point where I build assessments around them. There's one 24 week module I teach on the history of crime in the 18^th and 19^th centuries, which uses a lot of digital archives because it's a very well provided for area. But the second design for that module is for them to critically review an existing digital archive, which requires them to think about: How was it built? What search tools does it have? How might that affect the research we do using it? I've really built that sort of criticality that Jaap was talking about--about really understanding how these things are built and why that matters--into my assessments now, so that it's not just a thing that crops up in discussion, I've formalized the thinking about it. So that's for one module, which is very much a kind of a 'research methods' module that I've snuck into a history module. In other more conventional history modules, when students are familiar with this stuff, maybe later down the line, you can use them a little bit more in-and-out, you can use them for prep. But I do think students need more than just that one-off session, just that one-off bit of training; they need to really live in the archive, and to do that you've got be using it week after week.

Tom English:

Yes, it's a very interesting point. It takes me back to my undergrad days, particularly in the first year when I was doing modules on primary sources. For example, we did a module on primary sources just to drill us on the value and the limitations, as well as using primary sources period, never mind digital archives. It strikes me now that, maybe, it would be useful to have something like that in a digital context, whether that's appended to the teaching that is done in relation to the more broad primary sources things.

One of the things I talk to students about when I do front of class sessions with them is particularly--and it seems like such a banal point, but it's so important in terms of the actual home screen when you go into Gale Primary Sources--they're confronted with a screen that looks like Google, but it does not work like Google. That's what I say to them, I just use that phrase and say 'this looks like Google, but it does not work like Google'. You ask Google or Siri or Alexa or whatever, and it will give you an answer to a question: it doesn't work like that. I taught them a little bit about how the archives are digitized, just so that they know at least how that goes on - it still doesn't answer the question of how the archives are put together and who's making decisions about what goes in and what doesn't, but at least--from a methodological point of view--they can then get more information out of it. So great points Bob, thank you - on to Tom then please.

Thomas Tunstall-Allcock:

Bob's given me a lot to think about in terms of how I use resources with my students. I think a lot of what we're trying to do with our students, starting from the first year and where we are doing these basic skills courses, I think we often do more at that level around the concept of the primary source and those kind of key critical thinking skills. In some ways you're building on what they might have done at A-Level, but often we are trying to get them to think about sources, and I don't think we necessarily do a huge amount of that, about how to find the source: I think it's often quite generic across different approaches - a medievalist might be teaching the same approach as we are, so I think it's probably true that we probably need to think a little bit more about those actual platforms they seem to be using and how they look at them. In terms of how I use them in teaching, I think I try and progress from an early stage in the first year, and the second is often setting more specific documents; so I use the Gale collections to then be setting more specific tasks in advance of seminars. I think I should probably do a bit more about really working through the platforms with them. But what I try and do for guidance on those instances, and what having these collections allows me to do, is to set the particular preparation tasks; ask them to do particular things which get them thinking about how to look at particular sources, and also then use those different search criteria and filters. One example would be that we do a crisis simulation on the Cuban missile crisis, where different people in the class are assigned to different perspectives and roles to take on. I'm asking them, ‘your group is representing the CIA’, so you need to know: what is the context here? What are the kind of arguments that are put forward? Who is making them? I try and drive them in a way that isn't necessarily just thinking 'go and get this information and stick it in an essay'; I use them in a way where I try and ask them to think of specific questions, specific uses, throughout the course. I can't do that and just say 'and go and find whatever you want on the Internet'. So that is where it's really helpful to be able to point them towards it, to say 'you've seen this platform, using it in your dissertation research and elsewhere, and I want you to ask the specific questions of the sources'; and I find that really helpful in preparation. I'm not going as far as Bob in that I don't try and do this in a class as much, and we're always going to express time constraints or what we want our class time to be, how much time you want to dedicate to these things in class - and I think maybe I might rethink a little bit of that balance.

So I try and set very specific tasks--using what I know is in those archives myself, from having done searches myself--set them for these specific locations. One of the ways I would use them--building on what other people have talked about--is setting a specific path, asking them, with me knowing what's in there, to go and develop these things. Then at the end of the module all my assignments--certainly in my third-year courses--require the use of primary source research in there. I don't set any annual assignments that don't require that. There is that emphasis on them there, and throughout the course of the modules, work on constantly telling you: this is how you build this, this is what needs to be in there, because this will be part of final assignment. Nothing focuses student's minds in terms of doing a particular task like them knowing they're going to be assessed on it at the end of the module.

But I do think one thing I will do now is probably --going back to my first class--talk them through some of the different platforms. Actually going and doing a bit more work in terms of showing them and working through them, that is something that would be really useful. And maybe looking more to do what you have done, Tom [English], in terms of giving those samples and those introductions.

Tom English:

Good stuff, definitely happy to help with that, and that's something that we do--and something that I'm quite keen on doing as well--because I've done sessions all over the place, and I know other colleagues have as well. What strikes me is that--probably none of you three, maybe not so relevant to the three of you, but certainly in other academic's classrooms--I've been doing stuff and answering what I think are fairly simple questions, fairly basic ways of going through the archives, and the academic said to me afterwards, ‘actually, that was really useful for me, I didn't know about those things myself’. So always happy to help. The platform is always changing as well, they're iterative, they're not static platforms, they're being added to and refined all the time, so that's great.

How has the pandemic changed teaching and assessment for you?

Tom English:

Okay, so moving on a little bit now, I mentioned a moment ago about things getting back to normal for us in terms of getting out and seeing people and getting back on the road: things have obviously changed a lot for you in terms of teaching and research over the pandemic, and now that things are returning back to "normal", what does that look like?

Again, I'll couch it in a particular context. I was speaking with an academic--another different academic at Leeds--last week, and he was talking about assessments and exams. We were talking about the time of year, and I was reminiscing about when I had to do my exams in May sort-of time, and cramming my head full of information, and sitting in an exam room for three hours, hoping that I'd crammed enough information into my head, and got all the names and dates and quotes and goodness knows what else I'd memorized right, and in the right order, tuned to the questions sufficiently. What I understand is that, at Leeds, they are having an open book exam, where students--and this is obviously something that's come out of Covid--it's an open book exam where the students have a certain timeframe in which they can complete the assessment, but they're allowed to draw on their notes and allowed to draw on other sources, and if they can get it done in that context. Now, the first thing that struck me when he said that was, wow, that changes the game entirely in terms of what an exam looks like; it changes the examiner - it certainly has to change the examiner’s expectations of what's going to be produced in that exam period. But also in terms of the preparation as well. Is that one of those changes--I know I'm steering you quite strongly here--but how have things changed in relation to the sorts of assessments, and how has the usefulness or the potential use of primary sources changed in that context? What I can imagine, putting myself in the student persona is, back in the day, I would have been ramming my head full of information and either had that information to hand, i.e. the primary sources, or I wouldn't; now, however, what I'm imagining-- and you can correct me if I'm wrong--is that I would have a period of time which I could do more thorough preparation, and I will be going into the archives much more specifically. Is that right, and how have things changed? And how do you think that they will stay changed after the pandemic period? We'll start with Jaap again, please.

Jaap Verheul:

Covid has brought a huge revolution, and I'm sure that many of the results will be there to stay. In terms of my practical teaching, I had to go from in-class teaching to online teaching over the time of a weekend. I had to teach myself to use Zoom and Teams and all those tools and then at one point I got the hang of it. So I work with small assignments and breakout groups, and I think we all did a great job in adapting. However, the students suffered enormously from the online teaching and online research: so there's the downside. Yes, we did it, and you can teach online, and you can advise a thesis online; I've now had the pleasure of the first generation of my students to graduate without me having met them ever, and I don't enjoy that at all. It's a way of coping, but the point here with the archives is, of course, yes you would assume that digital archives will be the solution for students, and they would go to the [Gale] collection and do the research there. However, I guess that's bad news for [Gale], because students have been living online--in the teaching and in all the other aspects of their lives--they also began to resent doing online research. I also see a sort of resistance on the part of students to spending their whole life behind a screen. Having said that, of course, the digital archives are the solution; digital newspapers are the only resource that students still have to do if they wanted to do primary source research. But it definitely has a downside from the perspective of students, to the point that they become, you know, disaffected, dissatisfied, and even depressed, so I wouldn't recommend going online totally.

For perspective, I like the solution that Bob has found, that he teaches in a computer classroom, where you have the students around you and you can talk about all sorts; of the glitches and the things that the filter finds. You still have a real sense of community while doing digital research, so that's perhaps a way to go - although I must say that my university doesn't have that facility, so of course you can ask students to bring their laptops which I sometimes do: it's not quite the same.

The question about open book exams: yes, I've done that, and that works. However, I should also say that the examination committee sometimes gets nervous; you have those courses that are based on a knowledge transfer and you really want students to know at least a core set of effects in the body of knowledge: and that's impossible to assess with an open book or something. However, there are of course computer programs now that help you to do essay questions online, and that's definitely a tool that has a future. [Amindo?] is a good example of a teaching tool that also allows you to upload the norms, which picks up grading anonymously: but I'm not a hundred percent positive about it.

Tom English:

Okay, interesting - the point you made earlier about digital fatigue sounds to me like that's very pandemic-specific; I'm sure that the interest in digital will come roaring back once people spend a little bit more time outside and with friends and going to concerts and sports events and other such things. So hopefully that is just a temporary issue, so thanks a lot Jaap. Bob, please.

Bob Nicholson:

I guess I'm maybe a good example of this because I got rid of exams from my modules years ago; so, in a sense, assessments for me haven't changed much. I know that elsewhere in our course, there are time-limited assessments (we call them), where students have 24 hours to effectively answer an exam question rather than two hours in an exam hall; they have become the norm. I think for me, for history, that feels like a more appropriate way to organize an exam because ultimately I'm not really interested if somebody can remember a date, I'm interested in--with a bit of time--can they puzzle their way through something through research?

In that sense I think that it’s worked well. As I understand it, even though I haven't been involved in those, from what I've seen our expectations from them are nowhere near what we would expect from an essay; though we're still saying to them that we expect the sort of level of detail you would have put into an exam. At this point I don't think we're expecting them to head out into an archive and do original primary research in that 24 hours, but if I was a student taking one of those I absolutely would.

I think there's an interesting point about how we calibrate, how we approach, those assessments, and the expectations we have, particularly when things can happen like losing access to an archive, the Internet goes down; so many potential little problems there that we would have to be really mindful of in terms of creating equality of opportunity between students in those circumstances. But aside from that, my experience--obviously things changed an enormous amount during the pandemic with online teaching--it was striking how difficult it is to teach students to use digital archives remotely. I thought this would be great, they're on their own computers where my teaching doesn't have to change; I was teaching computer labs now they're all sat in front of a computer, it will be fine.

But it was so hard to spot the moments when they're struggling, the kind of slightly odd things they typed into search boxes; so even though you might think that the digital-first approach that I use would be perfect for online teaching, it's actually really challenging to provide that support. You need to be there in person, sat next to people often, to talk through how to fill in the search formula, how to interpret things, to imagine for yourself the view that they have. One-to-one you can do that through sharing the screen, but in a class of 30 people it's very difficult. It's interesting. I guess the only change, the only long-term change, for me--this will depend on whether the UK Government lets me--is that I would much rather pre-record lectures and use my in-person time to do workshops and discussion, rather than having an hour where I just talk at people. I'd rather just pre-record that and have them be able to look at that. So for me, I'm hoping that might be the long-term effect: we are under a lot of pressure in the UK to return to in-person lectures, even if that's not necessarily the best thing, I think. For me—weirdly--the long-term effects haven't been huge: I've largely clicked back into teaching in roughly the same way. Occasionally there are tools that I found I used in online teaching that I now use in the classroom, but not a great deal of change for me.

Tom English:

It sounds like you've already got your students pretty well drilled in actually getting into the archives and the primary sources; I wouldn't imagine, for example--just going back to my original point in terms of the open book element--I wouldn't for a moment imagine that most students will be rifling through Gale Primary Sources within that 24-hour period. But what I could imagine perhaps is, when they're formulating their notes for example, that the notes will be much better formed, and that they might then try to use the primary source material beforehand when they're creating the notes within which they are going to go into that 24-hour period. That’s something that I could imagine being more prominent, but it's interesting - it sounds like you're a bit of a trailblazer Bob in terms of the assessment. I do agree as well in terms of the manner of assessment: what good does it do to really assess somebody's memory and how much information they can cram into the heads when, as a historian, if you forget an exact date, as long as the chronology is broadly accurate, then it is not going to matter because you can check the date very quickly. It's a very good point, so thank you - and finally, on to Tom please.

Thomas Tunstall-Allcock:

I'd agree with you there. I also actually remove exams, certainly from my higher-level teaching: so my third year course, I take the exams out of that, where they have even less relevance, so the better our student’s experience gets. Where I have really seen the change is with mostly first-year modules that still had those big exams: 200 students, two essays, two hours. There was already some conversation about how useful those are. Certainly at my institution, the University of Manchester, those have almost entirely all gone now, to be replaced by a take-home exam. Now you need to give have a justification for why you want an in-person exam if you want one, so it's definitely completely swung away from those. From what I know from external examining at a couple of places too, that is pretty widespread across the board in history departments. I just don't think we're likely to see a mass return of that sort of assessment either. On the one hand, I think it is true that if we're trying to teach them to be historians, I don't know when I would ever be put on the spot and asked to write something in two hours without any access to any material.

But also, increasingly, we are having to think that the majority of our students are not going to be historians, they're going to go and do other things with their lives. Again, how often are they going to be put in the situation in a job where they're in a closed environment, with no access to the Internet, two hours, to produce something? It's much more likely they're going to be told 'I need this', and they have to produce something within a couple of days: we need you to write this report, go and research it, figure it out. I think it's just a much more effective form of assessment anyway. Where I think the interesting question about how much detail comes in is--we're only really seeing this form of assessment for maybe our first year students, maybe our second year students--so I think they're much less likely to be that effective in terms of bringing in other types of sources and primary source research. I would be interested, to pay attention as to how it goes; and what sort of responses we're seeing, as to whether actually, this might be a more relevant form of exam to bring back in for a third-year class: and that's the point that I think we might be able to encourage him to say you have 24-48 hours--whatever it is--and actually we expect to see a little bit of original research and we do expect you to be able to use these platforms to go and bring something here beyond just what you prepared. I think there’s maybe an opportunity there to do something a little bit different, and I do think probably that the era of those big exams--I mean certainly for us--is probably gone really, and I won't especially miss it.

Tom English:

No, I won't have missed them! I remember my final exams as a third-year at Leeds University, a three hour exam, and I forgot my water bottle as well. I forgot to take any water in with me and it was quite an ordeal; it was okay in the end, but it was pretty stressful, to say the least. Lucky, lucky students who don't have to go through those things anymore!

Super! Well, thank you all very much for your time, it's been great talking.