A peculiarity of the Communist Party of Great Britain (CPGB) was that it was one of the very few communist parties that was never proscribed by the state. The closest this ever came to happening was in the winter of 1940-41, as Britain for the time being bore the brunt of the Nazi war offensive. Communists in this period had not merely, like the pacifists, campaigned against the war. They also identified unreservedly with Soviet foreign policy, which in this period was one of friendly collusion (or ‘non-aggression’) with Germany.

Like the fascists, they were seen by the state as jeopardising national security through association with a foreign power. Unlike the fascists, they also belonged to what, until 1943, was a section of a centralised international – the Communist International, or Comintern – directed from a foreign capital. Even so, the treatment of these movements by the state was strikingly different. Already in May 1940, the British Union of Fascists (BUF) had been suppressed and over seven hundred of its members interned. In January 1941 the government did also ban the communist newspaper the Daily Worker, and it sought in other ways to impede the party’s functioning. Following careful deliberation at war cabinet level, however, it drew back from the general proscription of communist activities. The CPGB remained a legal party and, with the USSR’s entry into the war, became a considerably larger one, with over 50,000 members and significant campaigning presence. One could have joined at its formation in 1920 and remained a member until its dissolution without ever having had to operate illegally.

Another of the CPGB’s peculiarities was that, within Britain, the party was more continuously, intrusively and systematically monitored by the state than any other political movement – certainly far more so than the fascists. If ministers in 1940-41 were less draconian than they might have been towards the communists, it was not out of some scrupulous regard for their civil liberties. As the cabinet deliberations make clear, the deciding consideration was a pragmatic and somewhat grudging calculation that firmer action risked alienating a much wider constituency of opinion, particularly in the factories. The difficulty, according to minister of labour Ernest Bevin, lay in distinguishing between subversive propaganda and the ‘genuine grievances’ that communists also pursued, and with which they were publicly identified. Bevin’s Labour colleague, Herbert Morrison, then home secretary, warned that action against communists might provoke industrial disturbances on a scale akin to those of the First World War. The Conservative Sir John Anderson observed that the communists in any case were quite unlike the fascists in operating in a ‘perfectly open manner’. It did not follow that they would also be dealt with openly. Quite the contrary, it was Anderson himself who at just this moment initiated a restructuring of Britain’s secret Security Service (MI5) that included a separate branch for ‘counter- subversion’ and was followed very quickly by the installation of bugging devices in the CPGB’s King Street headquarters.

If the labelling of extremes was a device for setting the boundaries of the socially and politically acceptable, there were, in the Britain of 1940, manifestly differing, conflicting and sometimes rapidly changing views of what these boundaries were. While Mass-Observation had discovered unprecedented levels of public support for the BUF’s suppression, its respondents were marginally opposed to the ban on the Daily Worker and less supportive still of banning the CPGB itself. On the other hand, while the fascists by now were seen almost universally as lying beyond the bounds of acceptability, this had not been the view of large sections of the British establishment until at least the mid-1930s. The communists, conversely, had from the outset been anathema to just those sections of the ruling elite with which the secret state was most closely identified.

The three-way relationship between communism, society and the state was thus one of considerable complexity. Communism in Britain was, from the start, something of an intersection between a wider culture of labour and radical activism and the networks of the Comintern and the ostensible workers’ state at its centre. For Britain’s secret intelligence agencies, this translated readily into the language of subversion and sometimes espionage, hence offering rationale or simply pretext for treating communists as potential enemies of the state.

Nevertheless, there were also many who looked favourably on the USSR without ever joining the CPGB. Many shared the communists’ campaigning objectives, whether within or beyond the workplace; and many too passed in and out of the party without ceasing to share many of its values and commitments.

These connections were the communists’ security in 1940-41. Bevin had wanted to proceed against the ‘small number of intellectuals’ whom he believed to be the central pivot of communist activity. Nevertheless, ministers judged that the different forms of communist activity were too closely interwoven to allow selective interdiction without extending to the party as a whole. What this breadth of contact also meant was that there was a potentially huge and ill-defined pool of ‘sympathisers’ that could also be brought within the reach of state surveillance, and which included some of the major cultural and political figures of twentieth-century British history.

Through the prism of the war years, the precariousness of the CPGB’s status is glimpsed in sharp relief. Not only during the war but throughout its existence, the CPGB was a legal party subjected to surveillance practices with no proper basis in law. No public approval was required, or obtained, for these practices, nor any public knowledge or scrutiny provided for. An incidental by-product for the historian is that no British political party has ever been so well documented by the state. There is however a far from incidental disadvantage: that access to these documents continues to be governed by unaccountable and secret practices that are exempted from relevant legislation and place the prerogatives of the state above those of any notion of public interest, individual rights or democratic accountability. Historians, however, need not be completely deterred, for it is in fact a long-established truth of communist history that the shaping of an archive can be just as illuminating in its way as the individual documents it contains.

The French historian Nicolas Werth has referred to communism as a ‘civilisation of report’. As a leading specialist on the USSR, Werth catches very well the sheer abundance of documentation that the Soviet archives revealed after 1991. For a party of the Leninist type, the written word was a crucial instrument of co-ordination, instruction and control. Operating on the principle of democratic centralism, directions were meant to flow continuously from the centre, and reports on their carrying out flow back. For the system’s optimal functioning, even the oral exchange of views and information was, wherever possible, rendered into archivable text to be collated by some higher body. Where this, moreover, was a ruling party, these practices extended to the ancillary movements and activities which, in the Soviet case, meant those of society as a whole. Through the imagined device of the telescreen, Orwell captured vividly the compulsion for monitoring, recording and thereby controlling, which was the essence of the surveillance state. The result for historians of the USSR is a plethora of so-called ego-documents: diaries, confessions, evaluations, self-criticisms, summary biographies, and the records of letters and conversations which the state kept on its own citizens. Subject to the obvious limitation of resources, the civilisation of report also extended to the international communist movement and to parties like the CPGB.

Among the characteristic practices adopted in Britain by the 1930s were thus the provision of biographical data by communists of any official standing and the production of full stenographic reports of the meetings of leading party bodies. The difference, of course, was that British communists exercised no powers of state surveillance but, on the contrary, were subjected to them with an assiduity that would not have stopped at the telescreen had it been available. With its associated bodies, the CPGB was also the target of numerous police raids and burglaries, which on one occasion even resulted in the publication of selected communist documents at public expense. In their history of inter-war civil liberties, the legal historians Ewing and Gearty comment on the remarkable ‘absence of legality’ in the conduct of the state or of any sense that its agents should feel constrained by law.

A further consideration not applying in the USSR was the free movement of individuals in and out of the party, taking with them whatever documents they obtained there. For the CPGB, unlike its ruling counterparts, there was therefore no final safeguard against the abstraction of key documents by either state or defecting party member. At the same time, the political mystique of British communism, like that of any communist party, lay in claims of unity, discipline and unerring rightness that produced a positive obsession with the non-disclosure of internal differences and inconsistencies. The culture of report in these conditions was thus both dependent on documents and liable to be compromised by them; at one and the same time it needed everything down on paper and nothing down at all. The resolution of this conundrum, from the early 1920s, lay in the extra-territorial character of the CPGB’s ultimate guiding centre. This in any case was where the lines of accountability finally led, and where the machinery existed to digest a truly prodigious supply of text. Until its dissolution during the war, it was thus that the one safe repository for British communist documents was the Comintern in Moscow.

For as long as the Cold War lasted, one therefore had the curious paradox: that no British party was better documented than the CPGB, and yet no party’s historians had fewer documents to work with. This is vividly illustrated by the CPGB’s acceptance of the Comintern’s anti-war stance in October 1939. Arguably the most controversial episode in the party’s history, this not only compromised its anti-fascist credentials but led to a crisis in its relations with other sections of the left and to the removal of its general secretary Harry Pollitt. The deciding debate lasted an entire weekend and every word was captured by stenographers. Thanks to another police raid, the transcript found its way into the custody of the British state as well as the Comintern archives in Moscow. Nevertheless, when forty years later communists revisited the controversy, not even the surviving protagonists had anything to go on but their recollections. Although Pollitt’s anti-fascist sentiments had been vindicated by subsequent events, even he remained, all his life, the disciplined party cadre who had no wish to be proven right against the party. It was not until 1990 that the transcript finally saw the light of day as a by-product of Mikhail Gorbachev’s glasnost.

The passing of the Cold War thus marked something of an archival revolution in communist studies. With the USSR’s collapse in 1991, researchers were admitted to what the Comintern historians Brigitte Studer and Berthold Unfried described as an ‘Ali- Baba cave-system full of dossiers’. Subject at first to remarkably few restrictions, the files included literally thousands relating either to the CPGB, to its individual members or to associated movements around the party. Though the Comintern archive tails off by the war years, it is from just this period that the CPGB itself began to maintain a continuous, albeit far from comprehensive, record of its activities. Though closed to researchers until the very verge of the CPGB’s dissolution, by 1994 these materials were also accessible to researchers in Manchester’s People’s History Museum.

The MI5 files in the National Archives are the third major source for communist history to be made available following the break-up of the Soviet bloc. Prior to the 1990s, this too would have seemed unthinkable. So distant an episode as the forged Zinoviev letter, the intelligence community’s contribution to Labour’s 1924 electoral defeat, was still in the 1980s being argued out on partisan lines and without recourse to documentary proofs. As the USSR followed the Comintern into history, it was nevertheless clear that neither security interests nor ideological warfare offered any credible ground for the continuation of such secrecy. Already in the 1980s, there was a growing public demand for greater transparency sustained by a series of scandals and court actions exposing the abuses and dubious legality of established surveillance practices. Seeking to validate its role in a post-Cold War world, MI5 therefore set about a very well advertised ‘demystification’ of its public profile. A policy of ‘controlled contact’ with the media was followed in 2002 by the appointment of an official historian with some limited scope for strictly controlled disclosure. It was also as part of this rebranding exercise that the decision was taken in 1997 to begin releasing regular tranches of historical files.

The potential scale of the undertaking is staggering. In 2009 there were said to be almost 400,000 surviving paper files, and at very least a similar number had, at various times, been disposed of. In the absence of further information, we cannot tell how many relate to individuals classed as ‘communists or suspected communist and Russian sympathisers’. Nevertheless, over the main period of the CPGB’s existence this was the one durable core component of MI5’s activities.

Taken over from Scotland Yard’s Special Branch in 1931, it was thereafter regarded as a basic form of induction into the service. As late as the 1980s, still betraying the organisation’s military origins and ethos, a senior officer compared investigating communists to ‘learning to shoot for a potential infantry officer’. A typical file derived from these practices may include copies or transcripts of intercepted correspondence; the observations and reports of security personnel; notes of conversations obtained through bugging, tapping and undercover agents; and diverse public documents, such as press cuttings. It may also, of course, be redacted to remove any of these items without explanation. For historians of twentieth-century Britain, the files are a unique and disturbing resource. Conventionally in Britain, we do not refer to ours as a civilisation of report. Every individual, whatever their office or political persuasion, may dispose of their personal papers as they see fit, and such prerogatives are even intrinsic to our received notions of selfhood. Here, however, a whole class of individuals, many of them making no such provision, have had clandestine state functionaries provide the service for them. In redacting the files, they may protect the identities of their fellow operatives, but show no such compunction regarding the most intimate affairs of their subjects. Always this is without their consent, often it is without their knowledge and occasionally it is without even checking whether they are still alive before issuing to the world these sui generis snoopers’ scrapbooks.

They are certainly a most revealing source, both for their own times and for our own. Discussing the infiltration of an earlier generation of radicals, E.P. Thompson observed that spying was nothing new in Britain but rather an ancient part of British statecraft. Even so, with the broadening into principle of a notion of citizens’ rights, such methods had, for a time, been seen as both morally and politically indefensible.

Famously in 1844, there was a national outcry on learning that the correspondence of the Italian exile Mazzini had been pored over without his consent. In a forthright statement of the prevailing public view, Thomas Carlyle described it as a matter of basic principle ‘that sealed letters in an English post-office be, as we all fancied they were, respected as things sacred; that opening of men’s letters, a practice near of kin to picking men’s pockets ... be not resorted to in England, except in cases of the very last extremity’.

Carlyle did allow the possibility of such measures in the event of ‘some new Gunpowder Plot ... some double-dyed high treason or imminent national wreck’. When MI5 was established in 1909, it was at the height of the pre-war German spy scare, and it was on just such grounds that it was equipped with expansive opportunities for letter-opening under the new system of Home Office Warrants (HOWs). As, subsequently, communism displaced the German threat as the principal focus of such activities, the political case for such powers would have been that it too represented just such a plot and potential treason organised on a systematic, well-resourced and continuous basis. As this, however, was a clearly partisan, unavowable and vigorously controvertible rationale – the CPGB was, after all, a perfectly legal organisation – in practice the routinisation of peacetime political surveillance took place as far as possible in conditions of secrecy.

Though retrospectively it is often overshadowed by the issue of espionage, the one consistent pretext for such activities was subversion. Though a number of espionage cases were brought to trial from the late 1930s, no plausible explanation has ever been offered as to why supposed proofs of earlier networks had not also resulted in legal proceedings and public exposure. In any case, the routine surveillance of communists not only predated but long postdated any real concerns about espionage, and was later extended to groupings to the CPGB’s left that had no significant overseas connections. Subversion, meanwhile, was not defined even in general terms until 1972. In practice, this meant leaving it to the covert deliberations of the intelligence community itself to decide who and what was included. For much of the inter-war period this meant sweeping in leftists, including premature anti-fascists and anti-fascsist refugees, without unduly impinging on the patriotic right. It is a remarkable fact that, while hundreds of now-forgotten communist activists experienced invasive surveillance, Oswald Mosley could, for eight years, head the BUF and have Hitler among his wedding guests without yet having his correspondence tampered with.

It was successive home secretaries who declined to authorise a Home Office Warrant on Mosley. It would therefore be a mistake to focus only on MI5, rather than the wider political environment within which it was located. Maxwell Knight, who in this period took the lead in its penetration of the CPGB, had, for a number of years, carried out similar activities while holding high office in the British Fascists.

Recommended for state employment by the vicious anti-Semite and conspiracy theorist Nesta Webster, Knight was an out-and-out anti-democrat whose credentials included the infiltration of the CPGB by fascist agents and the burgling of perfectly legal premises for an anti-trade union agency funded by private business interests. These were not impediments to his employment by the official wing of the movement; they were positive recommendations, and there was evidently a time-lag, still to be fully investigated, before Knight came to see the case for using such methods against the fascists themselves. An MI5 prosopography might offer a fascinating insight into how typical or otherwise such forms of attachment and association actually were, and in tracing the career trajectories of its officers through various forms of military and colonial service. As Bernard Porter has observed, this is a missing dimension of intelligence history that matters all the more given how far MI5’s activities were guided by its own esoteric worldview rather than any wider notion of accountability. However, in a remarkable case of asymmetry between observers and observed, these identities and career histories, unlike the communists’, remain for the most part as closely guarded state secrets.

As historians we cannot therefore simply choose our subject of research and call up the necessary papers. Referring to the Comintern archives, Studer and Unfried warn against researchers reproducing the ‘policeman’s view of history’ they will find there. Carlyle, one imagines, might also have inveighed against the handling of stolen goods. Nevertheless, Carlyle also understood that historians of radical movements have to seize on whatever documents they can come by. ‘Governing Persons, were they never so insignificant intrinsically, have for the most part plenty of Memoir-writers’, he wrote of the toppling of France’s ancien régime. ‘Not so with these Governing Persons, now in the Town hall!’ Trotsky in his autobiography dreamt of a ‘cinematic record of modern parliamentarism’, but with the cameras installed, not at the parliamentary podium, but in offices and private places, and running over ‘the secret correspondence of the party leaders’.

It was a fantasy, of course; it is the ruling powers that have the cameras and the parties of revolt that have their correspondence photographed. So much is secrecy an attribute of power that it is these permanently oppositional movements, ironically, that are in many ways better documented than any of their more mainstream counterparts. As with any cinematic record, we merely need to keep in mind who it was that wielded the camera, and who decides which footage we get to see.

In December 1934, the communist writer Ralph Fox wrote in satirical vein to the New Statesman concerning the unfair advantage his biographers would have over those of his contemporaries. ‘For I have reason to believe’, Fox explained, ‘that for many years now earnest investigation officers in the Post Office have read every scrap of my correspondence before delivering it.’ Other earnest officers had doggedly tracked Fox’s movements and his personal contacts, and found a constant source of instruction in his telephone conversations. The task of Fox’s biographers was therefore simplified into that of obtaining the relevant files from the archives of the state. It being a well-established fact that ‘police evidence is the only kind which is infallible’, biography – ‘police biography’, no less – would at last be elevated to the status of a science.

Just as Fox suggested, the present author has enjoyed precisely these advantages in compiling his entry for the Oxford Dictionary of National Biography. Killed while fighting in Spain in 1936, Fox did not, as far as weknow, leave any personal papers. He was, more than most communists, a public figure whose writings include much journalism and several books. Through two spells working in Moscow, he has also left significant traces in the Comintern archives. Nevertheless, the MI5 files released in 2003 add significant further insight into the complexity of his commitments as one of the foremost communist intellectuals of his time. Fox himself would not have hesitated to use such a resource. Having recently published a life of Lenin, that ‘rather low foreign personage’, in his letter to the Statesman he described how he had drawn on just such caches of the tsarist secret police, the ‘revolution having thrown them open to the vulgar’. Historians of low personages have always had to be resourceful in the use of evidence; often it is state agencies that have had the resources and motivation to document them best. Like Fox in Russia, we are free to read these papers against the grain without having to reproduce the ‘policeman’s view’. But this does as always require critical reflection on how these archives came to be constructed in the form now made available to us.

There is a lesser order of question at the level of the individual file or subject. Here there is the obvious issue of what sorts of information the compilers sought to record, through which of the available forms of data-gathering, and how this has since been redacted. There is also the issue of how far the subject of the files sought to avoid having recorded just the sorts of information that their uninvited confidants were interested in. Users of the files have sometimes remarked on how much purely personal or even trivial information they contain. A communist peace activist of the 1950s has left elsewhere a remarkable collection of letters ranging freely over her beliefs, her daily tribulations and her most intimate personal relationships. ‘I bet the entire world and MI5 know about us’, she writes to her communist lover, ‘and what is more I am proud of it.’ But in another letter she refrains from so much as mentioning another comrade’s name, because ‘one never knows who reads our letters’. On Fox’s files the several HOWs taken out are justified in 1934 as having produced ‘interesting and useful information’ regarding both the open and covert aspects of communist activities. Typed extracts from both public and private sources reveal much that is also ‘interesting and useful’ to historians, including insights into political differences within the CPGB that may not be documented elsewhere. Nevertheless, the only covert activities revealed are of a purely personal character and not traditionally thought the business of the state.
Insights from other sources can sometimes help to clarify the sorts of information the files may not contain, as well as what they do. In the early 1990s, the journalist Sam Russell told of how, in 1952, he returned from Prague having attended the infamous Slansky show trial. Among the trial’s predominantly Jewish victims were communists who, during their wartime London exile, had had the closest relations with their British counterparts, and in one case even married a British party member. Russell described how Pollitt took him onto the King Street staircase, believing it to be free of bugging devices. Quizzing him about the victims’ demeanour, he then insisted on his absolute disbelief in Slansky’s confession of treachery. Pollitt never wavered publicly on the issue and the admission of private doubt cannot be said to reflect well on him. We can, however, only speculate as to why, and from whom, he should have concealed what was, in the end, a mere expression of opinion with no legal or security implications. Earlier indiscretions are certainly recorded on his Comintern file, while MI5 itself had urged publication of the compromising debate over the war in which Pollitt had expressed such forthright misgivings. All that we know for sure is that the exchange is not recorded on either Pollitt’s or Russell’s MI5 file.

A more serious issue for the researcher is the absence of transparency, accountability or system in MI5’s programme of six-monthly releases. The difference with Russia after 1917 is that there has of course been no revolution, nor even the pretence of consultation with the vulgar. An Operational Selection Policy (OSP) was produced in 2001, providing generic guidance for the preservation of selected historical files and none at all for their release into the public domain. The main priority groups identified are senior officials of ‘subversive’ parties, targets of ‘intrusive’ surveillance, agents of hostile intelligence, any individual or their relative not given security clearance, and those achieving positions of ‘public eminence’. No information is provided as to the number of individuals falling within each class, and MI5’s voluminous official history does not even contain such basic information as a listing over time of the bodies it deemed subversive. Its official historian has alone been granted access to such information, but under conditions of confidentiality and with past and present intelligence officers guiding him through the huge number of files that no single historian could digest. No other historian is able to request a file, or even to establish that it exists, or has existed, as Eric Hobsbawm was able to confirm on being denied access to his own. Hobsbawm’s files have, as it happens, been made accessible since his death, subject to what appears to be an unstated fifty-year rule. By policy or good fortune it appears that we are not (except by oversight) to have the benefit of any subject of a file being able to comment upon its contents.

Of the more than 400,000 files that have been incinerated, at least half went with unseemly haste under the ‘review and destruction’ policy introduced following the ending of the Cold War. This principally applies to targets of ‘counter-subversion’, and the OSP merely sets out the object of retaining a relatively tiny number of files not meeting any of its other criteria. MI5 remains exempt from the provisions of the 2000 Freedom of Information Act even in respect of historical files. It retains complete control over its own records and has never engaged in any public consultation with specialists having either an interest or an expertise in radical political movements. Its role in recent British history remains controversial and it makes no secret (for a change) of its manifest continuing interest in the validation of its historical role as a form of public advocacy in a post-Cold War world.

The wider aspect of these questions may not immediately concern the users of these files. What does obviously concern the researcher is the fact that some files may be read comfortably at home while many more have been destroyed without any public trace. Through the combination of selective release, wholesale destruction and the energetic management of MI5’s public reputation, the reader is steered towards a heightened sense of security threats in which the legitimacy of the organisation is seen to lie.

At the same time, the files will now never be made available that would reveal the sheer extent of MI5’s covert surveillance of legitimate political activities, whether carried out by communists or others. We should remember the instincts of that section of the population which in 1940-41 believed that any attack on such activities was an attack on democracy itself. In line with the wider securitisation of public discourse, we need to be wary both of the securitisation of the history of the radical left and of the normalisation of extra-legal techniques for its continuous monitoring. 

In just the spirit that Thompson recommended, we not only need to read between the lines of these reports, as we would of any such document, but to keep in mind the reports that were never made or which have since been despatched to the memory-hole. We will never, sadly, have telephone intercepts on the Conservative Party’s financial affairs, or on its contacts with unelected power-holders in the City, or on the sense of fellow-feeling with leaders like Mussolini with whom Britain did, eventually (unlike the USSR), end up at war. We therefore need to watch out for a ‘secret world’ conception of history that can only be applied to political radicals. As an example of negligent research procedures one might cite the files of the sometime communist and well-known journalist W.N. Ewer, released in October 2002. Within fifteen months an article drawing upon the files had been published by a then PhD student of MI5’s official historian, in a reputable historical journal of which both the latter and the official historian of British intelligence were editorial committee members. No wider research was drawn upon; in a subsequent contribution to the same journal it was argued that the article’s publication did not meet the normal canons of historical research, nor was it subject to the usual critical scrutiny of such obviously partial sources as Maxwell Knight. Readers can make their own minds up regarding the issues discussed; but of all the archives they may choose to use, those of an organisation whose basic operating principle is dissimulation (‘neither confirm nor deny’) may least of all be exempted from routine critical research procedures.

There was, therefore, a conspiratorial aspect of communist politics, but it is one that is contextualised and rendered in its true proportions – in other words, properly historicised. This itself is possible only within a wider framework of understanding, to which these files can contribute in two principal ways. The first is in relation to the biographies of those being monitored. The files can also fill in some of the many gaps in our knowledge of internal party communications and decision-making. For reasons of discretion as well as resources, the CPGB did not, in the post-Comintern years, maintain the same detailed account of its internal proceedings as had earlier been standard practice and, generally speaking, matters of most sensitivity were the least likely to be committed to paper. Nevertheless, the files are organised around the individual, not the subject. Like the Comintern itself, and doubtless many other forms of modern bureaucracy, MI5 was interested in biography as a form of control, and collected data on a large number of individuals who have independently attracted the interest of historians. As well as the obvious category of spies and suspected spies, this is true of the figures of ‘public eminence’ who include many key writers and thinkers of the left – Orwell, Thompson and Hobsbawm have already been mentioned here – and for the CPGB itself there is a clear and explicit bias towards communists holding leadership positions.

A collective biography or prosopography would, in theory, be another possible use of these materials. However, the unavailability of the great majority of files, and the arbitrary way in which those made available are released, means that this would certainly require a much wider range of sources.

Those drawing on methods of social network analysis for the study of radical movements will find in the MI5 files a conspiratorial variant of essentially the same approach. This would not, it is true, pass muster with any respectable committee on research ethics. Nevertheless, there may be opportunities to use the data for purposes very different from those for which it was intended. This is particularly true of the communists’ transnational networks. The defence of the realm for MI5 meant the defence of empire and colonialism: the malignity of communism lay not just in its Soviet connections, but in those with anti-colonial movements as well as European anti-fascists. When the Nazis came to power in Germany, MI5’s first significant response was to accept tze invitation to view police files like those on the communist-organised League Against Imperialism.

Political refugees from Germany itself, and from other countries under fascist regimes, could not draw upon the broader associations that served as the CPGB’s security. While in 1940-41 British communists were spared proscription, thousands of ‘enemy aliens’ were consequently interned in an episode that is strangely underplayed in MI5’s official history. The alien was an object of suspicion; the radical alien ten times more so, and the MI5 files again extend far beyond the ranks of the CPGB itself. Among those already accessible are those relating to future leaders of their countries, like Jomo Kenyatta and Kwame Nkrumah. We are still, however, waiting to see those on India’s first prime minister, Jawaharlal Nehru, who spoke at the CPGB’s central committee in 1938, and his daughter Indira Gandhi, who featured prominently in the ‘People’s Convention’ of 1940-41 that so nearly brought about the CPGB’s banning.

The second important set of research questions are those concerning the relations between communism and the British state. These in turn are matters of very real interest for anyone concerned with the longer Cold War that began in 1917. At the same time, given the restricted optics of the security service view, it is important not to forget that this – as ministers clearly saw in the winter of 1940-41 – was, in reality, a three-way relationship between party, state and society.

In his study of the earlier English radicals, Thompson argued that it was repressive state surveillance that drove them to obscure and conspiratorial methods. In locating these radicals within their wider society, he also refers to the opaqueness of the broader milieu as a form of resistance to state repression. As with many of Thompson’s observations, one wonders how far this drew on his own experiences as a former communist peace activist. Nevertheless, the differences are also apparent. As Thompson’s own case perfectly illustrates, intrusive surveillance in the twentieth century was accompanied by opportunities for open political activity that had no real parallel in the time of the Luddites. The result was a complex interplay between open and covert practices posing crucial political choices on the communists’ side, as well as that of the state. In tsarist Russia, fascist Italy or nazi Germany, the opaqueness of communist activity was a condition of it being maintained in any continuous way at all. But as we saw at the beginning of this article, the continued functioning of the CPGB depended far more on its open associations than on its secret ones.

It would be a mistake to reduce this to a war of subterfuge and counter-subterfuge. The benign official view of MI5 is that its failure to penetrate Soviet intelligence was offset in the 1940s by success in penetrating the CPGB. It is a view that certainly offers insight into MI5’s own values and priorities. Nevertheless, the communists, for their part, had not joined an intelligence agency and their primary objects were political. These certainly included defence of what they saw as the world’s first socialist state and its later satellites. Nevertheless, even these interests were the object of public campaigning in which it was the perceived congruence of Soviet ideals and wider aspirations for change that promised the circumfusion of communist influence.

It was during the latter war years, as the walls in King Street were almost literally installed with ears, that the communists enjoyed their best-ever opportunity to establish themselves at the heart of the British left and the wartime movement for social change. Recruitment to the party was never easier, and contacts and collaborators never so willing nor so numerous. In opening itself up to this broader movement, there was an obvious trade-off for the CPGB between vigilance and political effectiveness. Pollitt, however, had long had a predisposition towards the latter. Following the conviction for spying in 1942 of the CPGB national organiser, D.F. Springhall, older communists with a grounding in conspiracy, most notably through the Comintern’s Lenin School, were, in many cases, supplanted by a popular front recruitment still to be tested in such ways. Far from this jeopardising the party’s future security, had Pollitt known of the cabinet deliberations regarding the CPGB’s possible suppression he might have calculated that the best security was the open connection with a larger public through the ‘genuine’ grievances that united them. It is certainly suggestive that the party, at this point, had not sent leaders, like Pollitt himself, underground, but rather into the factories that were the labour activist’s equivalent of sanctuary.

This at least is one possible reading of the CPGB’s alleged laxity in matters of internal security. In the diary of the Comintern chief Dimitrov, Soviet intelligence sources may be found deploring Pollitt’s ‘strange behaviour’ in allowing British intelligence to plant its agents in both the CPGB itself and Soviet organs. ‘So far it has been impossible to determine whether [he] is carrying out this work deliberately or whether English intelligence is taking advantage of his lack of vigilance.’ The comments date from May 1942: just as membership was surging and the listening devices were going in. Like the rest of the international communist movement, the CPGB did, afterwards, move sharply towards vigilance with stalinism’s Cold War fixation on the figures of the traitor, spy and infiltrator. It was another party organiser, Betty Reid, who had the main responsibility for these activities, including the CPGB biographical files that are now accessible in Manchester. This is initially described in Reid’s MI5 file as illegal party work. It is also said to have provided its ‘main points of interest’ in respect of intelligence and Reid herself is described as quite ‘markedly security conscious’.

By 1951, she was nevertheless discussing the work quite openly over the phone, and, far from being illegal, the work was now characterised as ‘straightforward administrative personnel work on which one would expect a member of the Organisation Department to be engaged’. This entirely misses the political dimension of such activity and its contribution to a culture of party conformism and self-discipline. Nevertheless, it does again pose the question of whether it was really necessary to plant an informer in Reid’s home or (for example) record her movements around a model railway fair with her two small, fractious children.

Comparison is often drawn between the British experience and the harsher treatment of American communists under McCarthyism. Such comparison should not overlook the differences in the political culture and experience of the two communist parties.

The CPGB was always less practised in undercover work than its American counterpart. It was laxer in respect of vigilance, and in such matters as the rooting out of race and gender chauvinism; and it retained in Pollitt the sometime erring leader whose ultimate irreplaceability lay in the credibility he had with a wider labour movement public. It is notable that party defectors after 1956 did not, for the most part, move into new parties of the leninist type but rather, like Thompson, embraced the still greater porosity of the New Left. This did not spare Thompson continuing monitoring and investigation; with his prominence in the peace movement, one would be very surprised if it did not continue until at least well into the 1980s, some three decades after his breaking with the CPGB. Nevertheless, Thompson himself, as the peace movement also experienced sharp fluctuations of fortune, would surely also have calculated that his political effectiveness depended on not being driven into ‘obscure, secretive forms of activity’. There are many questions raised by the files to which our answers must remain provisional and which, in fact, may never be fully resolved.

Researching the history of the radical left has always meant working with materials that are less fully accessible than one might ideally want. Equally, as more materials have become more accessible, it is striking that older debates have not, for the most part, been laid to rest. Instead, they are simply renewed in changing forms depending, for example, on whether one starts from a position closer to the policeman’s view, or to the ranks of the so-called subversives themselves. There are therefore many types of history to be written with these materials; the crucial point is to read them as we would any historical documents, with a critical awareness of how exactly they have come to be displayed on the computer screen in front of us.


The official view of MI5’s history is Christopher Andrew, The Defence of the Realm: the authorized history of MI5 (London, 2009). This contains much interesting information that is drawn upon here. Unfortunately, the constraints under which the account was produced mean that there is rather more information that is excluded and which as yet remains beyond the reach of independent scholarship. The history provides no references for files still held by MI5, nor any details of those destroyed, nor any hard data as to the individuals and organisations targeted by the state. As yet there is therefore still no independent way of corroborating Andrew’s account or of exploring those aspects of MI5’s history that it addresses either perfunctorily or not at all.

An excellent overview of the field from the outside is R.C. Thurlow, ‘The historiography and source materials in the study of internal security in modern Britain (1885-1956)’, History Compass, vi (2007): 147-71. In part this updated the same author’s The Secret State: British internal security in the twentieth century (Oxford, 1995), which remains of considerable use. Important contributions on the relations between communism and the British state include K.D. Ewing and C.A. Gearty, The Struggle for Civil Liberties. Political freedom and the rule of law in Britain 1914-1945 (Oxford, 2000); Charmian Brinson and Richard Dove, A Matter of Intelligence: MI5 and the surveillance of anti-Nazi refugees 1933-50 (Manchester, 2014); Jennifer Luff, ‘Covert and overt operations: interwar political policing in the US and the UK’, American Historical Review, 122, 3 (2017): 727-57. The political affiliations of Maxwell Knight are discussed in John. G. Hope, ‘Surveillance or collusion? Maxwell Knight, MI5 and the British Fascisti’, Intelligence and National Security, 9, 4 (1994): 651-75.

For contrasting views of the proper uses of MI5 files, see Victor Madeira, ‘Moscow interwar infiltration of British intelligence’, Historical Journal, 46, 4 (2003): 915-33; and John Callaghan and Kevin Morgan, ‘The open conspiracy of the Communist Party and the case of W. N. Ewer, communist and anti- communist’, Historical Journal, 49, 2 (2006): 549-64.

There are already a number of examples of how the MI5 files can be used biographically. Geoff Andrews, Shadow Man: at the heart of the Cambridge spy circle (London, 2016) shows how the effective use of the files requires an awareness of the wider range of available sources. Madeleine Davis, ‘Edward Thompson, MI5 and the Reasoner controversy: negotiating “Communist principle” in the crisis of 1956’, Key Words, 15 (2018) is a detailed evaluation focusing on a critical turning point in the CPGB’s history. For contrasting approaches to some of the same individuals, see Jill Pearlman, ‘The Spies Who Came into the Modernist Fold. The Covert Life in Hampstead’s Lawn Road Flats’, Journal of the Society of Architectural Historians, 72, 3 (2013): 358-81; David Burke, The Lawn Road Flats: Spies, Writers and Artists (London, 2014); and John Green, A Political Family: The Kuczynskis, Fascism, Espionage and the Cold War (London, 2017). David Aaronovitch, Party Animals: my family and other communists (London, 2016) describes a more personal encounter with the files. For broader issues regarding the wider accessibility of communist archives, see Brigitte Studer and Berthold Unfried, ‘At the beginning of history. Visions of the Comintern after the opening of the archives’, International Review of Social History, 42 (1997): 419-46. For E.P. Thompson’s discussion of some similar issues in an earlier context, see his Making of the English Working Class (London, 1963), ch. 14, part 2. Within the extensive literature on the CPGB itself, the fullest account of communists and their activities, based on research predating the release of relevant MI5 files, is Kevin Morgan et al, Communists in British Society 1920-1991 (London, 2007). For a broader discussion of biographical approaches to the CPGB’s history, see Kevin Morgan, ‘Comparative communist history and the “biographical turn”’, History Compass, 10, 6 (2012): 455-66.

CITATION: Morgan, Kevin: "Within and beyond the law? British communist history and the archives of state surveillance." Political Extremism and Radicalism in the Twentieth Century, Cengage Learning (EMEA) Ltd, 2018



Any views and opinions expressed in these essays are those of the author in question, and any views or opinions from the original source material are those of the publication in question. Gale, a Cengage Company, provides facsimile reproductions of original sources and do not endorse or dispute the content contained in them. Author affiliation and information within them are correct as of the original publication date.

These essays, unless otherwise stated, are © Gale, a Cengage Company. Further reproduction of this content is prohibited.