DIGIT 210: Large Scale Text Analysis

• Join our DIGIT Slack Group and Hypothes.is group (if you did not already)
• ChatGPT and Git Review Exercise 1: Organize a directory in your GitHub repo (or create a new GitHub repo + directory for this): Curate your experimental prompts and responses as text or XML files in your GitHub repo.

• Read and practice with Dr. B's Shell and Git slide deck: review and learn some new shell and git commands: forking, branching, merging. Watch these two videos paired with my slide deck. (Note: if you're used to git from DIGIT 100 or 110, there is new material here.)
  1. Shell Commands for Navigation and Intel
  2. Git Command Tutorial

• Read and annotate in class Hypothes.is group:
  • Jay Alammar, The Illustrated Word2Vec
  • Shane Lynn, An introduction to word embeddings for text analysis
• Prep for the Python unit: Install Jetbrains Pycharm Edu (Pycharm Community Edition) on your computer, Find the Pycharm Edu training tutorial and start working on Introduction through Variables

• ChatGPT + Git Exercise 2: Push this assignment to your branch of the textAnalysis-Hub repository and issue a pull request assigned to Dr. B (ebeshero).
• Pycharm Edu tutorials: through Strings (submit evidence of completion via screen capture on Canvas).

• Read and annotate in our class Hypothes.is group:
  • Data Annotation in 2023: Why it matters & Top 8 Best Practices by Karatas
  • The Exploited Labor Behind Artificial Intelligence by Williams, Miceli, and Gebru
• Pycharm Edu Community tutorials: Complete the Tutorial through the Condition expressions Unit (submit evidence of completion via screen capture on Canvas).
• Review git branching, catch up/fix git homeworks if necessary.

• Read, annotate with Hypothes.is, and run executable cells in the Google Colab Notebook cellblocks in Tutorial: Exploring Gender Bias in Word Emedding
• Work on the Pycharm Edu tutorials: Get at least partway through Classes and Objects.

• Finish Pycharm Edu Intro to Python tutorials: Classes and objects, Modules and packages, File input and output. Submit evidence of completion via screen capture on Canvas.

• Python NLP Exercise 1
• (By the end of the day): Five Days of Git: Part 1: Record completion on Canvas as part of GitHub Test

• Five Days of Git: Part 2: Record completion on Canvas as part of GitHub Test

• Python NLP Exercise 2: comparing a set of files using word vector data: (words similar to a word of interest)
• Five Days of Git: Part 3: Record completion on Canvas as part of GitHub Test

• Discussion assignment of Parts 3 and 4 of W. J. B. Mattingly's online book, Introduction to Python for Humanists (2022)
• Five Days of Git: Part 4: Record completion on Canvas.

• Python NLP Exercise 3: Build a web scraper to collect text files.
• Five Days of Git: Part 5: Record completion on Canvas to finish the GitHub Test.

• Read (and indicate on Canvas that you have read) our Requirements for initiating semester projects (by end of the day).
• Post Project Ideas (Slack / GitHub)
• Python NLP Exercise 5 (Top2Vec Topic modeling)

• Read and annotate with Hypothes.is The XML FAQ, all of Section 1: Basics (all the short sections) about what XML is for.

• XML Exercise 1
• Project Milestone 1: Launch the project GitHub repo and invite your teammates and me to join (using Settings > Manage settings). Launch Slack channel for project and invite teammates and Dr. B. Post (in your Slack project thread or on GitHub) your available meeting times to help determine a regular meeting time for your group.

• XML and/or RNG Exercise 2

• XML Exercise 3
• Watch Regex Orientation Videos:
  • 20+ min. video: Introducing Regular Expressions in the oXygen XML Editor
  • How to do your Regex Homework in oXygen and keep a record of your Regex expressions

• Finish Regex Exercise 1 As you work on this, consult our Intro to Regular Expressions and the Regular Expressions Quick Start.

• Regex Exercise 2: Sonnets

• Regex Exercise 3

• Regex Exercise 4
• Project Milestone 2 due:
  • Create a file directory structure for the project GitHub repo(s): Initiate the project website within the docs directory with an index.html page and some CSS. Consult with your team and Dr. B to decide on a place to work on the text files (in its own directory, or in a separate private repo?) and create that space. Create a directory for XML files. Begin populating those file directories (even with placeholder Readme.md files to describe what belongs where).
  • Assemble the text files you want to work with on the project. As a team, work on document analysis to plan for how you want these to be marked for structure. What XML structure do you want to use to contain meaningful units of text data? Aim for a clear, simple structure that distinguishes the kind of info you want to be able to track.

• Work with our Follow the XPath tutorial and work through XPath Orientation Exercise

• Read and check out some things you can try in our newtFire eXist environment at Michael Kay’s Learn XQuery in 10 Minutes tutorial and our Introduction to XQuery and the eXist XML Database.
• (Due before class) XQuery Exercise 1: XPath over a collection
• End of the day: Project Milestone 3 due

• XQuery Exercise 2: Writing a FLWOR
• End of the day: Take-home Regex Test.

• XQuery Ex 3

• XQuery Exercise 4: Storing data in HTML
• End of the day: Project Milestone 4 due: exploring project files with XQuery

• XQuery Exercise 5

• Python NER Exercise
• Extra Credit Opportunity: Attend the World-Wide Climate Justice Teach-In from 5:30 - 8pm in Reed Bldg., or come at least at 7pm for the Technology panel (where Dr. B is presenting).

• Prepare for Network Analysis from XML data: Install Cytoscape on your computer. Begin familiarizing yourself with the Cytoscape interface, working with Cytoscape session files (with .cys extension) in the textAnalysis-Hub in Class Examples >> XQuery-NetworkAnalysis. Try opening one of the Cytoscape session files (.cys) found in one of the project directories there following our tutorial instructions.
• Read An Introduction to Network Analysis and Cytoscape for XML Coders

End of Day:

• XQuery to Network Analysis: Exercise 1 prepare a TSV from class today
• Project Milestone 5 due:

Complete take-home Python and Documentation Test (end of the day).

• Project Milestone (due end of day)

• Work through the w3schools SVG tutorial to complete the SVG basic and SVG shapes pages. Filters, gradients, examples, and reference are optional.
• SVG Exercise 1
• Project Milestone 4 (due end of day)

• SVG Exercise 2 (from XQuery): Plot a clear, simple, legible, labelled graph.

• SVG Exercise 3

• Project Milestone due: Visualization and Documentation Development