Portfolio / Creative Coding /

Conspiracy Machine

Conspiracy Machine

MFA in Computational Art
Goldsmiths, University of London
GPT-2 | BIG SLEEP | Twitter Bots

 

This project is about human and machine pattern recognition and apophenia – the perception of causes and patterns that are not really there. I’m interested in how pattern recognition has been a human tendency and that similarly neural network machine learning and algorithms have been built and designed to recognise patterns, even beyond what a human could recognise. Within these programmed recognition faculties also, arguably, comes apophenia and paranoia. Can the machines then make us more paranoid or do we make the machines paranoid? Are humans and technology paranoid about each other or is the technology a reflection of our own apophenia? After-all people build and design the technology imbuing it with their inherent biases – essentially they made this technology in their image. Furthermore, users are paranoid about the technology they interact with and the implications of surveillance capitalism from the likes of Google.

 

In our current moment, in the face of mistrust and paranoia towards mainstream media outlets, we have Youtube, Twitter and Facebook – builders and connectors of webs and networks – that act as conduits for spreading positive and negative ideologies and in some cases radicalising people. Social networks find patterns and connect people and in some cases connect them to strengthen and spread apophenic ideas leading to conspiracy theory networks such as QAnon. Are recommendation algorithms aiding in fuelling paranoia? I’m interested in the disparities of knowledge between what’s *real, objective* and the way images, news and ideologies are spread across the internet and the effects of the virtual in transforming the physical.

 

To engage with these questions and concerns I wanted to play with the tension between mistrusting technology and the internet and utilising it as a source for information by utilising neural networks and deep learning technology which underpin recommendation systems that deliver us much of our content across virtual networks. Rather than use the technology to recommend content I used it to create fake news and conspiracy content with GANs:

 

Conspiracy Website:

Website: www.marisadimonda.com/conspiracy-machine

 

I created a website that displays images and texts that have been generated with machine learning. The text is generated from a GPT-2 model I trained with a corpus of conspiracy texts scraped from the web which is then used as the prompt to generate images using neural networks – BigSleep.

 

Conspiracy Bot:

Website: www.kaput.marisadimonda.com/gen

Twitter: @MachineParanoid

 

The tweet delivered to @MachineParanoid is first generated on a website with my Flask application hosted on DigitalOcean. The app uses the Generative Pre-trained Transformer 2 (GPT-2) model created by OpenAI in February 2019 to generated the text. I fine-tuned(trained) the GPT-2 model with conspiracy texts scraped from conspiracy websites, archives and articles about conspiracy theories. To generate the text on the website, the app pulls the trending from Twitter UK to use as a prompt. The text is then generated from that prompt. Sometimes this works well and other times it’s non-sensical. The app does not automatically send the generated text to Twitter. The user has the option to tweet it or not with the inclusion of a button.

 

 

I’ve got 99 truths and a fact ain’t one.

 

10 May 2021

 

This essay explores the questions of how can it be possible to maintain the struggle against universalism without falling prey to the bubbles of paranoid conspiracy theories; how can we retain a sense of objective reality without sacrificing pluralism? These questions are posed drawing from the works of Donna Haraway and Bruno Latour and looking at how they laid ground for relativism and post-truth to flourish and then be hi-jacked by conservative, right-wing projects which have harnessed online networks to transform the physical and political world.

 

 

Unmaking Universalism:

 

It matters what matters we use to think other matters with; it matters what stories we tell to tell other stories with; it matters what knots knot knots, what thoughts think thoughts, what descriptions describe descriptions, what ties tie ties. It matters what stories make worlds, what worlds make stories.

— Donna Haraway, Staying with the Trouble: Making Kin in the Chthulucene

 

 

What matters is a matter of worlding. How we build knowledge, establish truths and imbue meaning is how we build worlds. It is an epistemological, ontological, ethical and political project. In 1988, Donna Haraway in her essay, Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective challenges the way Western culture presents knowledge. She described this as a presentation of disembodied, transcendent, universal truths and objective facts. This “view from above, from nowhere” she dubs the “god-trick”.1 The trick is that behind the conquering, neutral gaze resides the “unmarked”, position of the male, white, heterosexual, human. The consequence of this is the rendering of all other positions invalid and held as subjective.2 A bias lens through which reality is described. A lens so vast that it can be hard to recognise and difficult to determine its edge.

 

Haraway argues for a materialist framework of “situated knowledges” that turns away from the objectivity-relativity binary. She argued, scientists set themselves outside of the embodied world as part of an ontological move to retain objectivity.3 She saw that science, treated as a bastion of this objectivity within Western culture, was not created in a vacuum. Scientists, knowingly and unknowingly take their biases and cultural contexts into the lab and these are manifest throughout their practice. Situated knowledges, is a feminist objectivity about locating and positioning knowledge that privileges partiality as ‘the condition of being heard to make rational knowledge claims’.4 All positionings are partial and open to critical re-examination – objectivity is accepted but all claims are not considered as matter of opinion so as not to fall into relativism.5

 

In this framework, knowledge is contingent and based on embodied vision, that is, knowledge is limited to the location of the subject which is actively seeing and organising the world in a relational ontology – the subject and object are not split – allowing for accountability for what we learn how to see.6 Haraway moves away from the possessive individualism into an entangled relationality between knowing subject and an active, rather than passive, object, “The codes of the world are not still, waiting only to be read. The world is not raw material for humanization; […] the world encountered in knowledge projects is an active entity”.7 If the world is bound to the subject it cannot simply be reduced to raw material or a resource. For Haraway, greater vision comes from communities and pluralism, not isolated individuals, “The only way to find a larger vision is to be somewhere in particular”.8 Haraway’s situated knowledges removes the guise from the “god-trick” and embodies it as just one, partial, way of seeing, making way for non-linearity, pluralism, and multivocality. It aims to bring multiplicitous, diverse experiences and orientations up to an equal level of that of the dominant one.

 

The heteronormativity embedded in Western culture is an example of universal truths established by the ‘view from above’. In Orientations Toward a Queer Phenomenology, Sara Ahmed describes the heterosexual couple as a “social gift” that one receives and is pressured to accept. She describes heteronormativity as a “straightening device” to orient the sexuality of society along straight lines.9 The heteronormative worlding produced by the “god-trick” creates the space for the straight body and heterosexual couples. Within this space the queer body becomes a “failed orientation”, “slanted”, “oblique” as it orientates against the straight lines, inhabiting the space as social deviant.10 Situated knowledges can support the slanted and non-linear knowledge of queerness by asserting the contingency of knowledge making – dismantling the norm in heteronormativity.

 

Haraway’s ideas of knowledge as always situated in social, political and cultural context along with the work of Bruno Latour, received a backlash during the Science Wars of the 1990s— a series of public debates among “scientific realists”, who held that facts were objective and neutral and “postmodernists”, who believed that such facts were made by scientific research and the culture there-of.11

 

 

raven-the-science-maven.png 

Image: @ravenscimaven (Twitter)

 

 

Latour claimed, we only have controversies about facts. The controversies are attributed to the human processes involved in the making of scientific knowledge such as the interpretation of data, dependencies on technology to collect and decipher data, decisions surrounding which ideas to pursue, and which are most supported by evidence.12 For example, historically women have been excluded from drug trials. This results in a direct lack of understanding of the efficacy and side effects that a given drug might have on fifty percent of the population.13 These processes then become invisible, or ‘black boxed’ once the ideas are accepted by the scientific community, “When a machine runs efficiently, when a matter of fact is settled, one need focus only on its inputs and outputs and not on its internal complexity.14 Thus, paradoxically, the more science and technology succeed, the more opaque and obscure they become.”15 For Latour, we have no unmediated, unbiased access to truth.16

 

By undermining the objectivity of science and exposing the way facts are constructed and contingent, Latour (and Haraway), laid ground for relativism and post-truth to flourish and then be hi-jacked by conservative, right-wing projects such as climate change denial.

 

 

trump-tweet-snowing.jpegtrump-tweet-china-climate.jpegtrump-tweet-trillions.jpeg 

 

 

In 2004, the same year The Apprentice debuted17, essay, “Why Has Critique Run out of Steam? From Matters of Fact to Matters of Concern”, Latour expressed his own fear and doubts about his critical project.

 

 

I myself have spent some time in the past trying to show “‘the lack of scientific certainty’” inherent in the construction of facts. I too made it a “‘primary issue.’” But I did not exactly aim at fooling the public by obscuring the certainty of a closed argument—or did I? After all, I have been accused of just that sin. Still, I’d like to believe that, on the contrary, I intended to emancipate the public from prematurely naturalized objectified facts. Was I foolishly mistaken? Have things changed so fast?18

 

 

Do we have post-modernists and theorists like Latour to blame for climate change deniers? If all knowledge is situational and science is made in its own bias, faulty world then we arrive at a situation where all views are judged equally regardless of their rigour or all of the processes behind them and science loses its significance. “There is no sure ground anywhere”.19

 

 

kelly-anne-conway-alternative%20facts.png 

Image: Kellyanne Conway (CNN)

 

 

Latour proposes replacing matters of facts with matters of concern where facts become objects. For this epistemological move he draws a concept of objectivity from Heidegger’s Ding(thing) – meaning gatherings.20 Objects become complex realities, historically situated and diverse networks or webs, constituted by their assembly of ideas, participants (human and non-human), arenas and, institutions.21 The matters of concern are the things or issues we care and are concerned for. Latour’s matters of concern take into account the diversity of experience and the relationality of objectivity, respecting the ‘richness’ of the world.22 Critiquing of matters of concern involves examining the scientific, historical, ethical, and social processes of assembly or gathering as opposed to the data or nature of facts.23

 

However, how is this diversity of experience preserved in the face of conflict between communities with conflicting webs of relationality? For example, those who believe in climate change and climate change deniers. Latour, proposes using a multi-disciplinary approach of inquiry “to detect how many participants are gathered in a thing to make it exist and to maintain its existence” being the key to solving conflicts and making meaningful criticism. However, it is not clear how to achieve this as multidisciplinary approaches do already exist. Furthermore, how does this go up against the power of people with large socio- political networks?

 

He asserts that “facts remain robust only when they are supported by a common culture, by institutions that can be trusted, by a more or less decent public life, by more or less reliable media”.24 Who makes up this common culture; who decides which public life is decent; and how is the media deemed reliable?

 

The benefits of the pluralist projects of Latour and Haraway’s are also their downsides. Multiple realities/webs/networks are able to co-exist with violently conflicting realities without the strong tools to deal with the consequences, risks or ways to resolve these situations. It is not enough to simply accept that these ideas are “about being of and for some worlds and not others” and that post-truth is representation rather than an embodied materialist position.25 In our current moment we are living in an era of worlds among worlds – worlds with no climate change, xenophobic and eco-fascist worlds holding claims to the white genocide replacement theory and even worlds in which they call into question their own rotundity.

 

 

patriotic alternative tweetflatproof-twitter.jpeg

Images from letft – @PatAltWestMids (Twitter), @FlatProof (Twitter)

 

 

 

“Yes,” he said, “intelligence does enable you to deny facts you dislike. But your denial doesn’t matter. A cancer growing in someone’s body will go on growing in spite of denial. And a complex combination of genes that work together to make you intelligent as well as hierarchical will still handicap you whether you acknowledge it or not.”

— Octavia Butler, Lilith’s Brood.

 

 

Public Smear

 

Jürgen Habermas described how the ‘public sphere’ once served society as the realm where access is guaranteed to all citizens and society could form a public opinion and engage in debate to establish the governing rules of said society.26 This idea had its routes in physical spaces such as the Ancient Agora of Athens, where citizens would meet face to face, in the presents of other citizens and communicate verbally. However, any such concept of this ‘public sphere’ is now muddied and enmeshed with the virtual sphere. Mainstream big media outlets have lost their objective significance and in the face of P.C culture, mistrust and paranoia much of the public have turned away from them in favour of virtual networks such as Youtube, Twitter, and Facebook. These networks, particularly YouTube, have provided a platform which bypasses the traditional broadcasting gatekeepers to allow for content and commentators who would be deemed too extreme for public television to garner large audiences and in the process have aided them to establish sizeable yet much less regulated media businesses.27

 

 

pewtypie.jpeg 

Image: PewDiePie (YouTube)

 

 

Californian Ideology

 

In the same way that we have explored how science is not created in a vacuum, networks such as Facebook, Google and YouTube did not simply materialise in the absence of a specific culture. They were born from the Californian Ideology, which came from a particular moment in time that shaped their nature and design. The Californian Ideology was a fusion of the commune projects of the counterculture movement of the 1960s and 70s, of which Steve Jobs participated, and a libertarian individualism informed by the objectivist philosophies of Ayn Rand.28 Basically through the intrinsic, self-organising properties of the network, drawn from contemporaneous theories of ecology, it was thought that we could control our own destiny and society could achieve harmony without the need of the state to distribute resources.29 This has resulted however in a network of users in competition with each other, selling their ideas and aesthetic tastes, promoting the production of content aiming to garner the most views, likes, followers and watch-time. An attention economy.

 

These platforms – builders and connectors of webs and networks – act as conduits for spreading ideologies. The creators of content in combination with the inherent functionality of these platforms can, in some cases, create the perfect environment in which to radicalise people. The journey of radicalisation on social network platforms, in particular Youtube, has been investigated by Kevin Roose, journalist for the New York Times and artist Joshua Citarella. Roose describes the journey as a rabbit hole whereas for Citarella it’s a funnel.

 

 

The Funnel describes an informal network of diffuse messaging and recruitment, over time and across channels. The further “down the (funnel)”, the more niche and/or politically extreme the content becomes. Similar diagrams are used to map click-through rates(CTR): “eyes to buys”. At the top of the funnel we find popular channels with varied content and low CTR. Venturing into the deeper tiers clarifies ideological inconsistencies; views go down but CTR goes up.30

 

 

Roose found the YouTube recommendation algorithm was a common factor in leading viewers down the what he calls the “rabbit hole” by populating the user’s homepage and the “Up Next” sidebar with content similar to previously watched videos creating a vacuum.31 This algorithm has determined 70 percent of YouTube viewers’ watch time.32 The YouTube algorithm is aiding in the process of worlding for its users by constructing their network.

 

 

funnel.jpeg 

Image: New Models

 

Citarella observes that the recommendation algorithm has received more attention for the consequence of radicalisation than the socio-economic forces moving people to seek this political content.33 In both investigations people turn to the web as part of a trend “in search of coping strategies, social connections and political solutions”34, where some come across “far-right videos by accident, while others seek them out”35. Citarella claims that this trend will continue and although better recommendations may slow the the speed of radicalisation it will not address why people seek out the content.36

 

 

Subscribe to PewDiePie

 

At times these virtual networks have violently penetrate the physical sphere with disaterous consequences. For example, Brenton Tarrant’s mass shooting of attendants of two mosques in Christchurch, New Zealand, 15 March 2019. Tarrant’s attack was clearly motivated by his enormous consumption of specific kinds of internet content, ranging from edgy memes to dangerous extreme right-wing conspiracy theories. Three days before the attack, he posted a meme to his Facebook page depicting a white “crusader” in the war against white genocide strangling a muslim woman coupled with the text, “The weak should fear the strong”.37 Tarrant had written a 16,000 word manifesto titled, The Great Replacement, to accompany his attack against the muslim worshipers. Central to this was his belief is the conspiracy Replacement Theory. The theory claims the existence of a race war. It purports that white genocide is occurring in the West, that is, white people are being “replaced” by non-white people. He wrote, “we must secure the existence of our people and a future for white children”, a phrase which has been incorporated into the messaging of other far-right terrorists such as UK’s National Action, a neo-Nazi group.38 Tarrant encouraged his followers to spread their ideology through memes, “Create memes, post memes, and spread memes. Memes have done more for the ethno-nationalist movement than any manifesto”.39 Citarella considers memes to be “mental viruses”, explaining that they don’t get tiresome, they exploit the attention economy and their repetition make them difficult to forget, “Memes nudge our way of thinking. They become a type of augmented reality, overlaying the world and social relationships”.40 Memes are an unserious format with serious repercussions. They are an ideal format for normalising extreme views in a sarcastic and ironic way as part of internet trolling culture and an engaging with edgy humour. An insidious message can be disguised by the densely layered irony and pop culture references. Further more, given that they are mostly created usually anonymously the intentionality and agenda behind the memes become blurry.

 

Tarrant posted his manifesto to the unmoderated message board, 8chan, asking users to spread his message before live broadcasting the shooting on Facebook with the camera at an angle emulating first-person-shooter video games.41 In the live broadcast, immediately before carrying out the massacre, Tarrant also refers to Swedish YouTuber, PewDiePie, at the time the most subscribed to YouTube creator on the platform. PewDiePie began by uploading videos of himself playing video games and then moving to creating content that engaged with subscribers directly, cultural commentating, playing with the tensions of PC culture, pranks and even comparing the taste of his own piss after drinking cheap and expensive brands of water.42 He rose to a kind of emperor status of Youtube by gathering millions of subscribers (currently at 110 million subscribers) allowing him to make millions of dollars while mainstream media was struggling to keep people tuned in. He has been criticised by mainstream media outlets, such as the Wall Street Journal, for creating content promoting fascist and Nazi views but PewDiePie responds by breaking down their articles and claiming that they’re trying to discredit him, use him as clickbait for their own failing businesses and that they just do not understand his generation of edgy, ironic internet culture, “I think what this article shows more than anything, old-school media does not like internet personalities because they’re scared of us…Oh, OK, so Wall Street is losing influence and income. Hmm, I wonder who is gaining that right now? Maybe I should buy The Wall Street Journal. Actually, I don’t know. It seems like a failing business to me.”43 PewDiePie represented an era of creatively driven individuality, “He’s really become this symbolic culture warrior for the free and open internet, taking on the establishment on behalf of the little guys.” .44 In 2018, T-Series, an Indian media conglomerate, was close to taking over PewDiePie’s position of being the most popular channel on Youtube. This launched a global campaign, reaching Russia, Lithuania, Estonia, to gather subscribers to allow PewDiePie to maintain his reign. The Wall Street Journal, a Time Square billboard and a network of, 50,000 printers around the world were hacked to spread the message “Subscribe to PewDiePie”.45

 

subscribe-to-pewdiepie-superbowl.jpeg 

Image: Youtube creator, Mr Beast, who has over 14 million subscribers,
bought seats at the 2019 Super Bowl right behind the goalpost so his
“Subscribe to PewDiePie” t-shirts would be seen on TV (BBC)

 

 

 

Tarrant began his massacre by saying, “Subscribe to PewDiePie”, creating an opportunity for the mainstream media to target PewDiePie as an influencer and instigator perpetuating racially driven hate and violence. If you make reference to the most popular YouTuber you get the attention of his millions of fans – was he trying to assimilate these fans of edgy ironic humour with his extreme views or was he or just perpetuating the “subcribe to PewDiePie” meme. A trolling of immense proportions.

 

 

Brenton-Tarrant-aljazeera.jpeg 

Image: Brenton Tarrant making a White Power sign in court (Aljazeera).

 

 

Tarrant’s wrote, “From where did you receive/research/develop your beliefs? The internet, of course. You will not find the truth anywhere else.”46

 

 

In the wake of growing extremism on their platforms, YouTube, Twitter and Facebook have responded by updating policies to ban videos spreading bigotry,47 de-platforming users and intercepting the spread of fake news. A very public example of this is Twitter’s move to add fact checking tags to Trump’s tweets to his complete ban from the platform in an attempt to combat his spread of misinformation:

 

 

trump-fact-check-twitter.png 

 

 

De-platforming can work to the extent of quashing a voice and limiting it’s reach to that audience. For platforms like Twitter and YouTube, that is a large audience not exposed to those voices.48 Furthermore, could the de-platforming cause users to align with Trumps touting of social networks being biased against the right? Could this lead to seeking more extreme, unmoderated platforms like 4chan and 8chan where extreme views can spread and fester?

 

As asserted by Citarella earlier, de-platforming might not satisfy the issue for those seeking out political content or treat the socio-economic pressures driving them to it in the first place. In *Origins of Totalitarianism*, Hannah Arendt, provides insight into the formation of ideology. She ascribes loneliness as the grounding for totalitarian domination, “isolation and lack of normal social relationships . . . it is through surrendering their individual selves to ideology that [they] rediscover their purpose and self-respect”( Noreena Hertz, Financial Times).

 

From observing teens on TikTok with right leanings, Citarella claims that rather than de-platforming users we should first try to understand the content influencing them and then target them with leftist counter narratives.49 Caleb Cain, one of the focus cases in Roos’s podcast, found his way out of the alt-right rabbit hole in just this way and has created content about his experience, joining the YouTube leftist pushback (YouTubers such as Contrapointsand and Zero Books) against alt-right radicalisation.50

 

The work of theorist such as Haraway and Latour sought to expose the fallibility of many of Western Society’s accepted norms. By deconstructing the invisible hierarchies that order society and questioning the objectivity of science they exposed the fragility, inequality and sometimes ridiculous notions upon which society is organised. This way of thinking brings to light the cracks in structures that were previously accepted by many as stable pillars of the modern world. While this has doubtless contributed to some positive gains for many of those previously disenfranchised by these fallible structures it has also perhaps had another effect. Removing the veneer of objectivity in the sciences can mean that a growing distrust in science all-together can develop. And when an individual becomes siloed in a social media echo chamber dangerous, ungrounded narratives can form, multiply and be validated by communities that all share the same beliefs and information diets. When an individual looses their trust in the media they will turn to the array of alternative sources provided on online networks, not just in spite of their historic lack of regulation but often times because of it.

 

Gigantic unanswered questions surround these topics. Is technology the cause or the solution? How can we create a common world or common space which could hold all other worlds within a common, protective and safe structure? Perhaps something like an egg carton.

 

 

Footnotes

 

1 Haraway, Donna. “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective.” Feminist Studies, vol. 14, no. 3, 1988, p. 589. JSTOR, www.jstor.org/stable/3178066. Accessed 10 May 2021.

2 Rogowska-Stangret, Monika. “Situated Knowledges.” New Materialism, newmaterialism.eu/almanac/s/situated-knowledges.html.

3 Marcus, Miranda. “The God Trick in Data and Design: How We’Re Replicating Colonial Ideas in Tech.” Medium, Towards Data Science, 30 Nov. 2020, towardsdatascience.com/the-god-trick-in-data-and-design.

4 Haraway [1] p.589.

5 Rogowska-Stangret [2]

6 Haraway [1] p.583.

7 Ibid., 592-3.

8 Haraway [1] p.590.

9 Ahmed, Sara. “Orientations: Toward a Queer Phenomenology.” GLQ: A Journal of Lesbian and Gay Studies, vol. 12 no. 4, 2006, p. 560. Project MUSE muse.jhu.edu/article/202832.

10 Ahmed [9] p.554.

11 Kofman, Ava. “Bruno Latour, the Post-Truth Philosopher, Mounts a Defense of Science.” The New York Times, The New York Times, 25 Oct. 2018, www.nytimes.com/2018/10/25/magazine/bruno-latour-post-truth-philosopher-science.html.

12 “Scientific Controversy.” Wikipedia, Wikimedia Foundation, 29 Jan. 2021, en.wikipedia.org/wiki/Scientific_controversy.

13 Westervelt, Amy. “The Medical Research Gender Gap: How Excluding Women from Clinical Trials Is Hurting Our Health.” The Guardian, Guardian News and Media, 30 Apr. 2015, www.theguardian.com/lifeandstyle/2015/apr/30/fda-clinical-trials-gender-gap-epa-nih-institute-of-medicine-cardiovascular-disease.

14 Kofman [11]

15 Latour, Bruno. “Pandora’s Hope: Essays on the Reality of Science Studies”. Cambridge, Mass: Harvard University Press, 1999. Print. p.304.

16 Latour, Bruno. “Why Has Critique Run out of Steam? From Matters of Fact to Matters of Concern.” Critical Inquiry, vol. 30, no. 2, 2004, p. 237.

17 Neil, Jonathan T.D. “When Facts Become Concerns.” ArtReview, artreview.com/ar-march-2017-pov-jonathan-td-neil-when-facts-become-concerns/.

18 Latour [16] p.227.

19 Ibid.

20 Ibid., p.233.

21 Ibid., p.246,248.

22 Ibid., p237.

23 Flatscher, M. and Seitz, S., 2020. Latour, Foucault, and Post-Truth: The Role and Function of Critique in the Era of the Truth Crisis. Le foucaldien, 6(1), p.13.

24 Kofman [11]s

25 Weigel, Moira. “A Giant Bumptious Litter: Donna Haraway on Truth, Technology, and Resisting Extinction.” Logic Magazine, 5 Nov. 2019, logicmag.io/nature/a-giant-bumptious-litter/.

26 “Public Sphere.” Wikipedia, Wikimedia Foundation, 17 Mar. 2021, en.wikipedia.org/wiki/Public_sphere.

27 Kofman [11].

28 Citarella, Joshua. “Episode 14 – Shift Alt Right Clique ( Joshua Citarella)” New Models, episode 14. https://Soundcloud.com/Newmodels/Episode-14-Shift-Alt-Right-Clique-Citarella-Busta-Keller-Lilinternet

29 Ibid.

30 Citarella, Joshua. Radical Content. 2021. p.3.

31 Roose, Kevin. “The Making of a YouTube Radical.” The New York Times, The New York Times, 8 June 2019, www.nytimes.com/interactive/2019/06/08/technology/youtube-radical.html?smid=nytcore-ios-share.

32 Ibid.

33 Citarella [30] p.2.

34 Pemberton, Nathan Taylor, and Andrew Marantz. “The Young Political Spaces of the Internet.” The New Yorker, www.newyorker.com/culture/cultural-comment/the-young-political-spaces-of-the-internet.

35 Roose [31]

36 Citarella [30] p.4.

37 Citarella, Joshua. “Irony Politics & Gen Z.” New Models 2020, newmodels.io/proprietary/irony-politics-gen-z-2019-citarella.

38 Ibid.

39 Brzuszkiewicz, Sara. “Jihadism and Far-Right Extremism: Shared Attributes With Regard to Violence Spectacularisation.” European View, vol. 19, no. 1, 2020, pp. 71–79., doi:10.1177/1781685820915972.

40 Citarella [37]

41 Dearden [38]

42 Roose, Kevin. “The Accidental Emperor.” Rabbit Hole, episode 5, The Daily.

43 Ibid.

44 Ibid.

45 Ibid.

46 Brzuszkiewicz [40].

47 Roose [43].

48 Pemberton et. al [34]

49 Citarella [37]

50 Ibid