Korean Accounting Enforcement Dataset

Overview

A structured dataset of Korean accounting fraud enforcement decisions, extracted from Financial Supervisory Service (FSS) and Securities & Futures Commission (SFC) publications. The dataset covers 240 violations across three source document types, with a bias-validated six-type taxonomy applied consistently across both regulators. 86 named companies are matched to DART corp_codes, enabling cross-reference to financial statement data. 60 company-years have Beneish ratios computed from DART filings. The full enrichment pipeline is reproducible — LLM-enriched from PDF/HWP source documents, validated through cohort splitting, blind prompt stripping, and cross-model replication.

Problem

Korean accounting fraud enforcement is unusually well-documented and unusually hard to use. The FSS publishes quarterly enforcement reports as PDFs in Korean. The SFC publishes its quarterly decisions in ZIP files containing PDFs. Neither is structured. Anyone wanting to do empirical research on Korean accounting fraud has to extract the violations themselves — once for the PDF parsing, once for Korean text normalization, once for the violation taxonomy, and once again for the cross-reference to DART if Beneish ratios are needed. The aggregate cost across research groups is enormous. The marginal benefit of doing it well once and publishing it is high. A secondary problem: LLM enrichment of structured classification tasks is unreliable without validation. The natural failure mode is a prompt that scaffolds the answer — the model applies a label because the description matches the task surface, not because the source text provides specific evidence. Detecting this requires running the same cases with different prompts and different models, not just checking inter-model agreement on a shared prompt.

Constraints

• FSS Source 3 (229 cases): companies anonymized as 사A, 사B — no DART link possible; taxonomy work is limited to anonymized cohort
• 50 pre-2022 FSS PDFs are binary-corrupted at the source level — pdfplumber and pypdfium2 both fail; the 65-ok / 50-failed split is a permanent ceiling
• DART match ceiling ~90%: some named companies have restructured, delisted, or use name variants that cannot be resolved to active corp_codes
• Beneish coverage limited to 60 company-years: computation requires two consecutive years of comparable DART financials; many companies have gaps or use nature-of-expense reporting
• Dataset covers three of eight identified enforcement sources — data.go.kr structured CSV, CaseNote, auditor-side findings, FSC press releases, and three others are documented as v2.0 candidates

Approach

Three CSV outputs, each with a distinct analytical role. violations.csv (240 rows) captures the violation type from a closed-set taxonomy, the scheme type, forensic signals linked to the jfia-forensic detectlet vocabulary, and the source document. beneish_ratios.csv (60 rows) joins named-company violations to DART financial statements for the named cohort. dart_matches.csv (86 rows) maps named companies to DART corp_codes at roughly 90% match rate. The taxonomy went through five bias validation phases before the final production run. The enrichment uses a tiered model delegation: Haiku for high-volume structured classification with tool-use enum constraints, Sonnet for nuanced review and comparison, Opus reserved exclusively for the SFC final synthesis — one call interpreting ambiguous empirical findings against expected taxonomy patterns. Total cost through prompt repair: under $2.

Key Decisions

Tech Stack

• Python ≥3.11, uv
• pdfplumber, pypdfium2 (PDF extraction)
• Anthropic SDK (claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4-6)
• DART OpenAPI (financial statements for Beneish computation)
• kr-beneish (Beneish ratio computation for matched companies)
• pandas, pathlib

Result & Impact

• 240
  Total violations coded
• 86
  Named companies DART-matched
• 60
  Company-years with Beneish ratios
• 3 (FSS named, FSS anonymized, SFC)
  Source regulators
• 5
  Bias validation phases
• 95%
  SGI → revenue_fabrication precision
• 90%
  AQI → asset_inflation precision

The first open, reproducible Korean accounting enforcement dataset with a bias-validated taxonomy. The five-phase validation protocol demonstrates that LLM-enriched classification datasets can be made defensible — the TATA finding (100% assignment collapsing to 20% under blind stripping) is a concrete, falsifiable result that shows how generic prompt descriptions masquerade as empirical signal. The dataset's value is as a living artifact: when FSS publishes its next quarterly report, re-running the pipeline updates the dataset with one command.

Learnings

• LLM prompt scaffolding is undetectable through inter-model agreement alone. TATA fired at 95% for both Haiku and Sonnet with the full prompt — high agreement, but both models were reading the description, not the case text. The blind test (same cases, stripped descriptions) is the only design that detects a universal scaffold. Standard inter-annotator agreement checks catch differential biases; they miss shared ones.
• Prompt specificity is the primary quality lever for structured extraction. Rewriting TATA's description from 'large unexplained total accruals' to 'total accruals are materially large relative to assets with no clear business explanation; assign only when the case text specifically references accrual magnitude or reversal patterns' reduced the assignment rate from 100% to 34% concentrated in plausible violation types. The fix was one sentence. The validation to find it required five phases.
• DART match rate has a hard ceiling determined by company lifecycle, not data quality. Restructured, merged, or delisted companies produce name variants that cannot be resolved to active corp_codes. For Korean enforcement research, the effective labeled sample size for ML work is the 60 Beneish rows, not the 240 violations — the constraint is financial history availability, not extraction quality.
• Model delegation routing matters operationally. Sonnet batch ran 1.5 hours for 20 cases; sequential ran 3 minutes. The batch API's completion time scales poorly for review-grade inference tasks where the model processes complex Korean legal text. Route bulk classification to Haiku with tool-use enums; route analytical review to Sonnet sequential; reserve Opus for one-shot synthesis where the interpretation determines credibility.