Guidance on AI Text Generators
New open-access AI technologies are emerging (such as ChatGPT) that are able to generate written text based on user prompts. Several at Villanova have put this new software through its paces, and we believe that most of the time, it does not generate college-level text suitable for paper submission.
At this time, AI-generated responses to broad essay-style questions lack detail and analytical depth. However, the technology will generate a starting point that a student could potentially edit and expand upon to generate an essay that will meet college standards. Furthermore, it can generate short answers that could be suitable for blogs, discussion boards, and more.
We have put together some recommendations that faculty may consider implementing in the coming semester and in the future as AI tools become readily available and more sophisticated.
- Consider placing a statement in your syllabus or on assignments that explicitly stipulates that the use of AI technologies to generate text is not allowed and will be treated as an academic integrity violation.
- When assigning a grade penalty for an academic integrity violation, note that the violation must be reported through the formal procedures. This guarantees students recourse to an appeal and makes it possible for the University to track repeat violations. For more information, consult the Academic Integrity Policy.
- Make a renewed commitment, at the individual and disciplinary level, to Villanova’s grading standards. Avoid assigning an A or B to work that is not “distinctive,” “outstanding,” “considerably above acceptable standards,” or demonstrating “a very high degree of understanding” or “unusual effectiveness.” Consider including these standards on your course syllabi and discussing them with students early in the semester. Emphasize that the value of a Villanova degree hinges on the University’s maintaining these standards.
- Talk to your students about AI text-generating technologies and the importance of doing their own work. Discuss how the writing process nurtures skills and abilities that are necessary to their employability and success after graduation.
- Use the free detection tools that are becoming available and inform students that detection tools will be evolving right along with AI technologies. Note that these are not 100% reliable but can be useful. Currently, Safe Assign and TurnItIn cannot detect AI-generated work.
- Meet as a department early in the semester to discuss this topic, and run some tests together to develop strategies that are specific to your discipline and student population. Include adjuncts in these conversations, as many adjuncts teach introductory-level courses and courses for non-majors.
- Consider forming a standing departmental committee on academic integrity whose purpose is to keep current on developments in digital technologies and detection tools pertinent to your discipline, to report those developments to the department, and to suggest ways to address them.
- Formulate your writing prompts to require responses that are currently difficult for AI to answer well (adapted from Dan Cannity, UMass, Amherst):
- Use prompts that are very specific or require application of a concept to a unique problem or phenomenon. Avoid very general questions or questions that invite generalization.
- Require that students offer detailed support from course material and/or class discussion.
- Use recent topics. ChatGPT's data is about a year old, so asking about something more recent means it won't have good information about it. We do expect this data to be updated and the system to continue to learn and improve, so stay vigilant.
- Ask questions that ChatGPT cannot discuss. It is programmed to be limited (for now, at least) to avoid controversial topics like the current war in Ukraine. If the course material allows for this and doing so is appropriate, adding in questions to which the bot cannot respond will mitigate its use.
- Focus on process questions rather than informational ones. Examples: "Which applies better and why"; "explain a statement and why you think it means that"; "which solution to a problem is more appropriate and why." The chatbot cannot coherently explain its rationale to specific problems the way a human can.
- When assigning and evaluating discussion board posts, emphasize the quality of students’ engagement with each other’s work.
- Try using ChatGPT with your essay prompts to see what it generates and learn how to identify its output. Test a given prompt multiple times, as each text generation by the software is unique. Narrow and adjust your prompts to find the limits of what the technology can competently produce.
- Submit student work through Safe Assign in Blackboard. This software checks for plagiarism and will help deter students from using AI-generated text as that text is often repeated and could be matched to other students’ work.
- Consider using the tool to generate text and have the students analyze the results and discover its shortcomings. Show them how easy it is for you (and them) to tell something was generated by AI.
- Consider using class time for writing so you can work with students and observe their processes. Move other class content online to compensate for this.
- Use multi-step writing assignments (such as abstract, draft, final draft, revision) that require students to show their work at various stages.
July 2023 Updates
Dear Faculty Colleagues,
We would like to update you with some information around A.I. generators such as ChatGPT in preparation for the fall semester. A.I.’s capacity to generate text, computer code, graphics, music, and more is continuing to improve, and its use is becoming more commonplace. Most of our students are aware of these tools and have used them. While some time savings and useful information can be generated by A.I., substituting A.I. use for human cerebration can be detrimental to student learning in many situations, while in others it can be extremely helpful. It will continue to be up to the individual faculty member to allow or prohibit its use in their courses. We ask that you:
- Become familiar with A.I. generators through readings and personal use. Review our previous advice (see above) and VITAL’s newsletter on how to generate assignments in the age of A.I. Continue to have discussions in faculty meetings about this issue. As this technology is here to stay and is steadily evolving, we need to continue to familiarize ourselves with its benefits and drawbacks and consider it within our disciplinary instructional contexts.
- Design instructional activities and assignments to directly support your courses’ learning goals. Should you find that A.I. use supports and/or enhances the students’ achievement of these goals, then consider its educational use. If not, you may consider prohibiting its use for that activity, assignment, or the entire course. Some forms of higher-order thinking, such as analytical reading, critical thinking, problem solving, evaluation, metacognition, and synthesis of knowledge, might not be mastered with the use of A.I.
- Discuss with your classes the proper use (or prohibition) of A.I. and explain the reasoning behind your instructional decisions given the courses’ learning goals. Focus on the skills and cognitive capabilities your course intends to foster in your students and how A.I. can either support or inhibit building students’ competencies. Students must be able to utilize modern tools, develop abilities that exceed those of machines, evaluate the reliability of all information including that is generated by A.I., and meet the higher-order demands of engaged citizenship in an evever-changingcomplex world.
- Set clear expectations and guidelines around appropriate A.I. use in your courses, and consider including one of the following statements for your syllabus/assignment that we have developed; you may adjust to reflect the nature of your course:
- The use of A.I.-generated content is not permitted in this course/on this assignment. Its use will result in an academic integrity violation and a zero on the assignment.
- The use of A.I.-generated content is allowed in this course/on this assignment. Even if you have significantly edited A.I.-generated material, you must identify the A.I. tool used to assist in generating your work. You are required to provide the name of the tool, date used, and prompts used to generate the output. As you may be required to submit the original A.I. output you must keep a copy of the original output and provide it when requested. If questions arise about the authorship of submitted work, you are responsible for authenticating your authorship. You should save evidence of your authorial process, such as drafts, notes, version histories, and complete transcripts of A.I. assistance.
- The use of A.I.-generated content is permitted as follows (a) for generating a first draft or (b) for generating an outline or (c) for generating XXX. Even if you have significantly edited A.I.-generated material, you must identify the A.I. tool used to assist in generating your work. You are required to provide the name of the tool, date used, and prompts used to generate the output. As you may be required to submit the original A.I. output, you must keep a copy of the original output and provide it when requested. If questions arise about the authorship of submitted work, you are responsible for authenticating your authorship. You should save evidence of your authorial process, such as drafts, notes, version histories, and complete transcripts of A.I. assistance.
- Understand it is challenging to detect A.I.-generated material. A.I. detectors are not fully reliable, and there are apps available whose specific intent is to render A.I.-generated content undetectable.
- When filing an academic integrity violation for a student’s work, point to specific parts of the Code of Academic Integrity and, when possible, focus on aspects of the student’s work that are indisputably problematic. The Academic Policy Committee is in the process of updating the Code to address A.I. more explicitly with an expected announcement early in the fall semester. Some relevant sections of the Code (which you may decide to include in your syllabus) are:
- "When completing an individual class assessment (i.e. assignment, quiz, lab report, exam, etc.) students shall rely on their own mastery of the subject and not attempt to receive help in any way not explicitly approved by the instructor.”
- “Plagiarism is defined as the appropriation of another's work and the unacknowledged submission or incorporation of that work as one's own offered for credit.” This statement applies to A.I.-generated material even if edited.
- “Students shall not falsify, invent, or use in a deliberately misleading way any information, data, or citations in any assignment.” Often A.I. will generate false citations or actual citations that do not correspond with the material.
- Consider providing the attached “Guidelines for Students Regarding Responsible Academic Use of A.I.-Generated Material” to the students in your course and/or discuss topics contained within it.
Sincerely,
Gabriele Bauer, Director, Villanova Institute for Teaching and Learning
Alice Dailey, Professor of English and Director of Faculty Affairs and Chair, Academic Integrity Board
Mary Beth Simmons, Senior Director, Villanova Writing Center
Randy Weinstein, Professor of Chemical and Biological Engineering and Vice Provost for Teaching and Learning
Note: Some of these require you to be on campus or the VPN to access.
Goldberg, E. (2023, July 19) “Training My Replacement’: Inside a Call Center Worker’s Battle With A.I.,” The New York Times.
Brunelle, Gayle K. (2023, June 25) “Letters: AI’s Impact on Gen Ed Will Be Profound.” The Chronicle of Higher Education.
McMurtrie, Beth and Beckie Supiano (2023, June 13) “Caught Off Guard by AI.” The Chronicle of Higher Education.
Lauritzen, Pia (2023, June 1) “Letters: How to Cultivate a Learning Environment Fit for the AI Age.” The Chronicle of Higher Education.
D’Agostino, Susan (2023, January 30) “Designing Assignments in the ChatGPT Era.” Inside Higher Ed.
Martar, Jake (2023, February 1) “Beating ChatGPT.” University of Arkansas for Medical Sciences, Office of Educational Development website.
Prochaska, Eric (2023, January 23) “Embrace the Bot: Designing Writing Assignments in the Face of AI.” Faculty Focus.
McFarlane, Lydia. (2023, February 8) “University Members React to ChatGPT on Campus.” The Villanovan.
Lang, James M. (2023, April 4) “How to Create Compelling Writing Assignments in a ChatGPT Age.” The Chronicle of Higher Education.
Next Steps
Villanova will continue to monitor AI developments and are discussing possible changes to our academic integrity policy to more explicitly address its use. The Academic Policy Committee (APC) will work on addressing this challenge. We invite faculty to share information and strategies with each other.
In the meantime please feel free to reach out to Gabriele Bauer in VITAL for assistance or Mary Beth Simmons in the Villanova Writing Center as they are both prepared to help you.