4: Responsible AI Use
- Page ID
- 67039
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\dsum}{\displaystyle\sum\limits} \)
\( \newcommand{\dint}{\displaystyle\int\limits} \)
\( \newcommand{\dlim}{\displaystyle\lim\limits} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\(\newcommand{\longvect}{\overrightarrow}\)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)The previous three chapters were concerned with getting good output. This one is concerned with knowing whether the output you received is any good, and with the obligations that come with using these tools at all. The two subjects belong together. Responsible use is not a disclaimer attached at the end of a project; it is a set of checking habits built into the work itself. This chapter begins with a map of the kinds of risk these systems carry, and then builds a framework for evaluating both tools and outputs to mitigate these issues and use AI responsibly.
Risks, Limitations, and Constraints
Different AI programs and generators have different risks, limitations and constraints. For this reason, you may need to use several tools to achieve the results you are looking for and each comes with drawbacks. Koch (2023) noted that be cause AI requires a lot of data, there can be blind spots, where it won’t know what to do, or times when it does not consider rights or legislation such as privacy laws. Some key risks, limitations and constraints are listed below.
-
Inaccurate information / hallucinations
-
Date cutoff in model training
-
Data bias/misrepresentations. Models cannot think critically about research/outputs.
-
Blindspots due to new situations/ inputs/information so unsure what to do
-
May ignore legislation and privacy laws
-
Limits of use due to pricing/ subscription costs
In your evaluation and analysis of tools, it is good practice to note any risks, limitations or constraints that may have impacted your results.
Bias
Bias is a distortion of facts based on inclination or prejudice. This could lead to unfair results or treatment of people or research. Results that AI generates may have bias, based on how the AI was programmed, or what data/prompts you input into the program. Koch (2023) noted that “If your data isn’t representative, the AI will replicate that bias in its decision making, which is exactly what Amazon saw when its AI recruitment bot penalized women candidates after being trained on resumes in a male-dominate d dataset.” Bias in data or results that AI generates from your input, can impact on your studies. It is therefore important to try to be inclusive with inputs and note this issue or limitation. When analyzing and evaluating data and results, you should be aware of the social biases inherent in both the inputs and the traditionally more educated groups that produce research outputs. It’s important to recognize that the internet allows anyone, regardless of qualifications, to share opinions, which may not always be based on factual research. You should critically assess the bias, authority, and purpose of their sources. Acknowledging these potential biases and limitations is crucial during the evaluation and analysis stages of research.
Legal Issues
Copyright
There are several important considerations related to copyright and AI, including:
- the intellectual ownership of the data used to train AI models
- how and when protected material can be uploaded to AI tools
- the copyright status of the AI tools' outputs
Training
Content creators and owners are increasingly concerned that LLMs have been trained on copyrighted works without permission. There is ongoing litigation about whether AI companies breached copyright. The New York Times sued OpenAI and Microsoft, claiming “unlawful copying and use of The Times’s uniquely valuable works.” As part of the largest copyright settlement in US history, Anthropic agreed to pay authors for Anthropic's use of over 7 million pirated books to train their models.
Be careful not to upload licensed or copyright-protected materials into AI tools.
Privacy
Because data serves as the critical foundation for training and refining these systems, AI developers are incentivized to engage in widespread data acquisition, including web scraping and indiscriminate personal data collection. This intense hunger for data inherently conflicts with core data privacy principles, amplifying existing digital surveillance risks and exposing both individuals and broader society to new privacy vulnerabilities.
While global regulatory frameworks like the European Union’s General Data Protection Regulation (GDPR) and some state laws like the California Consumer Privacy Act (CCPA) attempt to manage data governance through Fair Information Practice Principles (FIPs) and automated decision-making provisions, they face significant limitations in the AI era. Current privacy regimes rely heavily on "privacy self-management," forcing individual consumers to bear the overwhelming burden of monitoring, consenting to, or opting out of data processing across thousands of digital interactions. To ensure that personal data privacy can coexist with safe AI development, policy frameworks must evolve beyond post-hoc individual rights toward proactive data governance and structural systemic changes. Key strategies for mitigating AI-driven privacy harms may include the following proposals adapted from King & Meinhardt (2024):
-
Denormalizing Data Collection by Default: Shift from opt-out paradigms to privacy-by-default standards that enforce strict data minimization, while implementing automated software mechanisms (such as Global Privacy Control) to handle consumer privacy preferences without friction.
-
Focusing on the AI Data Supply Chain: Mandate strict transparency, provenance tracking, and dataset documentation across the entire AI development life cycle, incentivizing the use of ethically sourced and high-quality datasets over indiscriminate web scraping.
-
Flipping the Script on Personal Data Management: Support the creation of technical infrastructure and legal frameworks for regulated data intermediaries (e.g., data trusts or cooperatives), enabling collective bargaining, automated rights management, and user-centered control over personal information.
Uploading Content to AI
Many AI tools automatically incorporate any content you upload into their underlying data. In addition to the obvious personal privacy concerns, you should think carefully before uploading content that is owned or licensed by someone else.
Confidentiality
Confidentiality involves not sharing information or data that should be kept secret (often proprietary corporate information). Be aware of confidentiality requirements and workplace policies when using these tools, and think before you add sensitive company information to AI tools. One way to protect yourself and confidential information is to use tools that apply some level of data protection and security. For example, enterprise-level deployments of most major AI platforms do not use information supplied as part of prompts to train foundation models and also encrypt any information you supply to the tool. Nevertheless, there are many potential risks that generative AI may pose to an individual’s privacy, such as:
- data breaches, such as attacks designed to make a model regurgitate its training dataset
- losing control of personal information
- scams and identity theft
- generating harmful content (e.g. image abuse)
- generating code that can be used in cyber attacks.
- data breaches, such as attacks designed to make a model regurgitate its training dataset
- losing control of personal information
- scams and identity theft
- generating harmful content (e.g. image abuse)
- generating code that can be used in cyber attacks.
Cybersecurity
Increasingly, Advanced models can identify and exploit vulnerabilities in online systems and platforms, posing risks to the cybersecurity of individuals, companies and governments. Anthropic suppressed the release of its recent model, Claude Mythos, in part because of its ability to uncover previously hidden vulnerabilities.
Read Claude Mythos and Project Glasswing: Why an AI superhacker has the tech world on alert.
Ethical issues
AI technology is raising many new ethical dilemmas. These tools can replicate and exacerbate biases that exist in the underlying training data. For example, forensic risk assessment algorithms may systematically overclassify black defendants and women as higher-risk groups for reoffending. As a result, governments, companies, and individuals are beginning to consider their ethical obligations when using and implementing AI systems. For instance, UNESCO has developed a human rights approach to AI, and Maranatha Baptist University has published a set of 10 guiding AI Principles as well as specific policies for faculty, staff, and students.
Trust and integrity
Most students are familiar with the concept of plagiarism and identify it is unfair and unethical behavior. Similarly, using AI tools without appropriate acknowledgment or referencing is unethical. Trust, integrity and ethics are key professional and societal principles. Your instructors expect the work you submit to be your own, unless otherwise attributed. While AI tools can support your learning, you are responsible for learning, so that you ultimately enter the workforce with skills that demonstrate this learning. Society will expect the future university-trained professionals and clinicians to be ethical AI users.
What would you do?
A friend tells you they used ChatGPT to generate their entire lab report and didn’t tell the lecturer. They encourage you to do the same, saying, “It’s just faster.” Take a moment to reflect:
- Would you follow their advice?
- Would you ignore it, report it, or try to have a conversation?
- Your course rules around AI use
- The impact on your learning and future skill development
- What consequences might this have for you, your friend, and your professional credibility?
Social impacts
Employment and the workplace
Artificial intelligence is currently transforming the workforce by changing specific tasks rather than wiping out entire economies. While overall employment numbers remain stable, entry-level workers and routine tasks face early disruption, even as productivity and new tech-driven roles grow (Bonney et al., 2024). Still, AI has the potential to disrupt industries and employment, and radically change the way that we work. The International Monetary Fund estimates that AI will affect almost 40 per cent of the global labour market (Georgieva, 2024). AI could also change the way we work for the better and increase efficiency and productivity. Organizations will need to ensure that workers are AI-literate and have the necessary skills and motivation needed to adjust to these new ways of working. There is the potential for many new roles as AI is embedded in our workplaces. Possible negative impacts could include:
- automation leading to the elimination of some jobs
- unemployment causing a rise in inequality
- increased surveillance in the workplace
- hollowing out of some creative industries.
Human creativity
There are fundamental differences in the way that generative AI and human creativity work. Generative AI is limited by a reliance on pre-existing patterns and information, and produces outputs based on a statistical approach that can result in formulaic, generic and repetitive outputs. Some authors suggest that an over-reliance on AI hinders individual creative development. This technology is also incapable of symbolic or moral reasoning, which are fundamental aspects of human creativity.
Evaluating Models and Outputs
AIxDESIGN & Archival Images of AI / Better Images of AI / AI Am Over It / CC-BY 4.0
You should evaluate the quality and reliability of AI outputs, just as you would information from any source. Information provided by generative AI tools can be:
- incorrect
- out of date
- biased or offensive
- lacking common sense
- lacking originality.
AI tools tend to produce 'middle-of-the-road' answers based on a consensus of the most common information in the AI's training data. You should continue to think critically as you use the tools for your learning. Ask yourself:
- is the response you've been given too conservative?
- is there an alternative viewpoint that has been missed?
- what are your views — do you disagree with the information?
Methods for evaluating information
There are many methods for evaluating information. The TRAAP test is useful when evaluating information and also emphasizes some of the challenges with assessing AI-generated content.
Applying the TRAAP test
- Timeliness
- Relevance
- Authority
- Accuracy
- Purpose
Challenges
LLMs may not always present you with the sources for answers, or may generate answers based on unsuitable sources. Some tools will be trained on out-of-date information. This can make it difficult to judge the relevance, authority, accuracy and purpose of the information.
I can help with a wide range of topics, but there are some limitations. For example, I don’t have access to:
- Personal data unless shared with me during our conversation.
- Real-time data like live sports scores or stock prices.
- Confidential or proprietary information.
- Certain copyrighted content in full, such as books, articles, or songs.
Tips for confirming the information provided by AI tools
- Ask the tool to provide you with sources. You can ask for a specific type of source (peer-reviewed journal articles, news articles or academic sources). You can provide other constraints such as a time limit, e.g. 'Can you provide academic sources from the last 5 years?'. Writing your prompt in academic or formal language will increase the chance of getting those types of sources. Note that there's no guarantee that the AI tool will give you what you ask for but these techniques can increase the chance of better outcomes.
- Locate the sources provided and confirm the information is real. Generative AI tools will present false information as fact and make up references.
- Once you confirm the sources, consider their quality and whether they are appropriate for your task.
- Look for other reputable sources that also confirm the information.
"Treat AI like a slightly unreliable intern. Have a chat, ask some questions, assign basic tasks. Don’t trust the results too much though."
Can AI do your reading for you and should it?
Human in the loop
Evaluating the outputs of AI tools is sometimes referred to as "human-in-the-loop" work. Many of the AI models are based on predictive modelling and contextual understanding of the prompts they’re given. These models make mistakes!
Constant feedback by the human-in-the-loop can improve your specific output and also the AI tools and models "and enhance the accuracy, reliability, and adaptability of ML systems, harnessing the unique capabilities of both humans and machines" (Source: What is Human-in-the-Loop in AI & ML?).