Overarching goal. Explore persuasion in r/changemyview, where the delta (∆) marks a comment that changed the OP’s view.

This subproject. Examine evaluative practices in delta and non-delta comments, focused on the assessment of the OP’s argumentation.

Research questions

1. How do commenters evaluate the OP’s argument, and how explicit or implicit is that evaluation?
2. Does it differ across two contrasting topics, movie_character (taste) and god_religion (value)?
3. Does assessing the argument relate to persuasive success (the delta)?

• Source: the r/changemyview corpus, submissions and comments from 2013 to 2020.
• Tagged for part of speech, lemma and dependency with spaCy. A first pass with en_core_web_sm encoded the corpus into NoSketch Engine; we then re-tagged the same token streams with en_core_web_trf, a transformer model built on RoBERTa, for higher accuracy while keeping the tokenisation fixed. All queries use the transformer tags.
• Three sub-corpora: submissions (cmvsub), delta winning comments (cmvdelta), and a general comment reference (cmv500k).

• We ran BERTopic over the submission texts (title plus body) to group threads by what they are about. Embeddings from all-MiniLM-L6-v2, with min_topic_size = 200 so the model returns a few dozen well populated topics rather than hundreds of tiny ones.
• That produced 75 fine grained topics, which we merged hierarchically and curated by hand into broader labels: movie_character, god_religion, us_politics, relationship_sex, animals_meat and school_college, among others.
• We drew a stratified sample by topic and role, and for this talk compare the two that sit at opposite ends: movie_character (taste) and god_religion (value).

• Stage 1, corpus. Slot alternation patterns in NoSketch Engine (Hanks, 2013) surface candidate evaluative structures, but reach only about 2.7% of sentences.
• Stage 2, annotation. Assessing an argument is semantic, not surface detectable, so we hand code it in INCEpTION (Klie et al., 2018) on the Evaluation layer.
• Stage 3, triage. A cue and LLM pre-classifier surfaces candidates at high recall; the analyst adjudicates every one.

Coded for polarity and explicitness, the assessments are overwhelmingly negative and implicit.

• In god_religion, commenters go after the argument’s logical validity: assumptions, contradictions, fallacies.
• In movie_character, they argue over taste: whether the view is just opinion or a question of quality.
• The two topics sit at opposite ends, value on one side and subjective taste on the other, which is why we picked them.

• The argument flaw words (straw man, circular, fallacy, assumption) are all rarer in the comments that won a delta than in ordinary comments.
• Comments that win a delta tend to soften their criticism rather than confront the OP directly.
• Potential future focus: are challenging questions doing confrontational work? The aim is to tell genuinely undermining rhetorical questions (Ilie, 1994) apart from ordinary ones. (Descriptively, questions are somewhat commoner in non-delta comments, 6.0% versus 4.8%.)

• In movies, argument assessment is more common in delta winning comments (41% versus 21%); in religion the pattern reverses (32% versus 47%).
• We make no claim either way: assessing the OP’s argument neither reliably wins nor loses a delta. The pattern depends on the topic.
• (exploratory, document level, small n)

• god_religion stands alone on the logical-validity side. It is the one topic where commenters routinely name assumptions, contradictions and fallacies.
• The other four topics, us_politics, relationship_sex, animals_meat and school_college, all sit closer to movie_character on the taste side: they argue more about opinion and quality than about the argument’s logical validity.
• Value disputes look like the exception. Most topics lean toward the subjectivity end of the spectrum.

• Across both topics, commenters only sometimes evaluate the OP’s argument itself. When they do, the evaluation is mostly negative and implicit: the weakness is implied rather than spelled out.
• The topic decides the register. In value disputes (god_religion) commenters use the language of logical validity, naming assumptions, contradictions and fallacies. In taste disputes (movie_character) they argue instead about whether a view is just opinion or a question of quality.

• Open confrontation does not look persuasive. The words that name argument flaws, and questions used to undermine, are both more common in comments that did not win a delta. The comments that did win lean on softer, implicit criticism.
• Whether evaluating the argument goes with persuasion depends on the topic: it goes with winning in movie disputes, but not in religion disputes (exploratory, small sample).
• Methodologically, the corpus and annotation cycle, with a high recall cue and LLM triage, is a workable way to find a rare and semantic phenomenon at scale.

• Complete the balanced sample and inter-coder reliability.
• The sequential organisation of evaluation across turns.
• Extend the persuasion analysis to all six topics.
• A corpus of OP edit statements, where OPs meta-engage with the comments.