Evaluative practices and persuasion in r/changemyview

Overarching goal. Explore persuasion in r/changemyview, where the delta (∆) marks a comment that changed the OP’s view.

This subproject. Examine evaluative practices in delta and non-delta comments, focused on the assessment of the OP’s argumentation.

Research questions

1. How do commenters evaluate the OP’s argument, and how explicit or implicit is that evaluation?
2. Does it differ across two contrasting topics, movie_character (taste) and god_religion (value)?
3. Does assessing the argument relate to persuasive success (the delta)?

• Source: the r/changemyview corpus, submissions and comments from 2013 to 2020.
• Re-tagged for part of speech, lemma and dependency, then loaded into NoSketch Engine.
• Three sub-corpora: submissions (cmvsub), delta winning comments (cmvdelta), and a general comment reference (cmv500k).

• We ran BERTopic over the submissions to group threads by what they are about.
• The micro-topics were curated into readable labels, among them movie_character, god_religion, us_politics, relationship_sex, animals_meat and school_college.
• We drew a stratified sample from these, and for this talk compare the two that sit at opposite ends: movie_character (taste) and god_religion (value).

• Stage 1, corpus. Slot alternation patterns in NoSketch Engine (Hanks, 2013) surface candidate evaluative structures, but reach only about 2.7% of sentences.
• Stage 2, annotation. Assessing an argument is semantic, not surface detectable, so we hand code it in INCEpTION (Klie et al., 2018) on the Evaluation layer.
• Stage 3, triage. A cue and LLM pre-classifier surfaces candidates at high recall; the analyst adjudicates every one.

Coded for polarity and explicitness, the assessments are overwhelmingly negative and implicit.

• In god_religion, commenters go after the logic: assumptions, contradictions, fallacies.
• In movie_character, they argue over taste: whether the view is just opinion or a question of quality.
• The two topics sit at opposite ends, value on one side and subjective taste on the other, which is why we picked them.

• The argument flaw words (straw man, circular, fallacy, assumption) are all rarer in the comments that won a delta than in ordinary comments.
• A preliminary indicator points the same way: the share of sentences phrased as questions is higher in non-delta comments (6.0%) than in delta comments (4.8%).
• Comments that win a delta tend to soften their criticism rather than confront the OP directly.
• Future trajectory: tell genuinely undermining rhetorical questions (Ilie, 1994) apart from ordinary ones.

• In movies, commenters who engage the argument win more often (delta 41% versus 21%).
• In religion, it is the opposite (delta 32% versus 47%); the effect flips (interaction p = .054).
• We do not claim that assessing the argument wins. If anything we find the reverse: in religion, non-delta comments carry more of it. The movie subset points the other way, but is small and exploratory.
• (exploratory, document level, small n)

• god_religion stands alone on the logic side. It is the one topic where commenters routinely name assumptions, contradictions and fallacies.
• The other four topics, us_politics, relationship_sex, animals_meat and school_college, all sit closer to movie_character on the taste side: they argue more about opinion and quality than about formal logic.
• Value disputes look like the exception. Most topics lean toward the subjectivity end of the spectrum.

• Across both topics, commenters only sometimes evaluate the OP’s argument itself. When they do, the evaluation is mostly negative and implicit: the weakness is implied rather than spelled out.
• The topic decides the register. In value disputes (god_religion) commenters use the language of logic, naming assumptions, contradictions and fallacies. In taste disputes (movie_character) they argue instead about whether a view is just opinion or a question of quality.

• Open confrontation does not look persuasive. The words that name argument flaws, and questions used to undermine, are both more common in comments that did not win a delta. The comments that did win lean on softer, implicit criticism.
• Whether evaluating the argument goes with persuasion depends on the topic: it goes with winning in movie disputes, but not in religion disputes (exploratory, small sample).
• Methodologically, the corpus and annotation cycle, with a high recall cue and LLM triage, is a workable way to find a rare and semantic phenomenon at scale.

• Complete the balanced sample and inter-coder reliability.
• The sequential organisation of evaluation across turns.
• The full persuasion test across all six topics.
• A corpus of OP edit statements, where OPs meta-engage with the comments.