Test environments for social data and digital platforms
Generative AI can enable new types of criminal and malicious attacks on a massive scale and it is increasingly difficult to detect synthetic content used in these attacks. We therefore need to develop defences against generative AI attacks (preventive, reactive and regulatory).
At the same time we can use generative AI to test different types of content and their impact within a controlled environment. This requires a realistic test environment to evaluate the impact of attacks and the effectiveness of various defences.
The University of Melbourne’s “Mirror World” project was an experiment in developing a test environment for a social media platform led by ADM+S Chief Investigator Prof Chris Leckie in the School of Computing and Information Systems. The project used Open AI’s GPT-3 service to generate messages on different topics, with different sentiments and stances. These messages were then run through a local emulation environment of Twitter to test the impact of mis/disinformation.
From this initial experimentation, the Australian Internet Observatory (AIO) will expand on this work to build a test environment or social “cyber range” for testing and analysing a range of issues already occurring on digital platforms. Test environments, also known as a cyber range are often used in cyber security for testing issues in a contained environment. In a similar way, researchers involved with AIO believe we need a social test environment or “social cyber range” – national research infrastructure for testing and analysing digital platforms and social media content to reduce harms, anticipate problems and support positive outcomes from digital platforms.
What is a cyber range?
A Cyber Range is a facility set up to carry out security testing of equipment or network configurations in a secure environment. It can be a combination of internet-facing and isolated networks. It allows cyber security devices, software, and techniques to be introduced into the environment for certification, or standards-based testing to be performed, to help products to market.
For more on cyber ranges see the Australian Cyber Collaboration Centre
Technical requirements
A social test environment or cyber range would involve both online and offline testing and may include a range of content and tools including:
diverse content: images, voice, text etc both real and generative AI
data donations for online platforms and live content
semantic analysis
low level authentication information
platform understanding
large language models and generative chat
Simulated recommender systems
The facility can be developed in a modular way to provide tools for synthetic content detection (open source or commercial). This will ensure the range can remain agile in response to a rapidly changing information environment.
Use cases for a social test environment
A social test environment will be valuable for many experiments and use cases and will be essential for understanding current and future threats, and preparing practical defences. Negative scenarios include testing malicious attacks from Generative AI, financial fraud, bullying and coercion, child grooming, fake product reviews and system flooding. More positive scenarios may involve testing personalised tutoring, product development, training for help lines and user behaviour in a digital social setting.
Benefits of test environments
The social test environment has the potential to become a key national research infrastructure platform, and environment for training and education on cyber security and mis/dis information. It will provide evidence to guide policy and regulatory efforts and safety guidelines and train graduates with expertise in responding to this new generation of “cognitive/social” cyber attacks. Ultimately it will provide an opportunity to anticipate and test problems and solutions rather than only responding.
