Brand Equity Validation Tool

Sampling strategy Β· Expert agreement metrics Β· All parameters auto-determined from corpus Β· Use CSV for large files (>20 MB)

What would you like to do?

Step 1
🎯
Review Corpus Manager
Upload your annotated dataset. All sampling parameters are auto-determined from corpus characteristics. The random seed is the only required input. Produces a reproducible sample + expert template.
Step 2
✍️
Expert annotation
Load the expert template and annotate reviews one by one β€” blind to LLM output. Sessions are saved to the server and can be resumed at any time from any device on this network.
Step 3
πŸ“Š
Compute agreement metrics
Upload the LLM reference file and one or more expert annotation files or saved sessions. The appropriate metric is auto-selected per dimension. Supports multi-expert comparison.
Large files (>20 MB): export your Excel as CSV before uploading. CSV files parse significantly faster in the browser and avoid timeout issues with large XLSX files.
Datasets Load one or more files Β· CSV recommended for >20 MB
πŸ“„ Add dataset
XLSX or CSV
Required columns
Company / enterprise name Stratification by company size
Star rating Numeric 1–5, rating-group stratification
Additional stratification columns Optional β€” each column added ensures proportional representation of its values in the sample (e.g. category, subcategory, country)
No additional strata selected.
Language filter Optional β€” select a column then auto-detect the values present
Language column (none = no filter)
Exclusion filters Optional β€” rows matching these conditions are excluded
Error flag Rows where this column is not empty are excluded
e.g. country = USA Β· verified = true
Dimensions Auto-detected from columns matching {name}_n_mentions + {name}_sentiment, or manually configured
For each dimension you can choose which associated columns to include in the sample output. n_mentions and sentiment are always included (used for sampling and metrics).
Or map a custom dimension manually:
Row identifier Optional β€” used only for the SHA-256 reproducibility checksum
Unique row ID Auto-detected if present
Additional columns to include in sample output Non-dimension columns β€” e.g. review text the expert needs to read
Load a file first.
Run analysis in step 2 first.
Complete steps 1–3 first.
How this works
Step 1 β€” Upload files. Upload the LLM annotations file and one or more expert annotation files. Each expert file must have the same column structure as the LLM file, with the same review identifiers and filled-in {dim}_n_mentions / {dim}_sentiment columns.

Step 2 β€” Metrics are auto-selected per dimension based on the sentiment category structure in the LLM file: binary β†’ Cohen"s ΞΊ Β· 3 ordered categories β†’ Krippendorff's Ξ± ordinal Β· 4+ unordered categories β†’ Krippendorff"s Ξ± nominal.

Step 3 β€” With multiple experts, all pairwise comparisons are computed: LLM vs each expert, and expert vs expert (human baseline). The human baseline tells you how close LLM performance is to human-level agreement.
Upload files
LLM annotations file (.xlsx or .csv)
πŸ€–LLM output file
The full dataset or just the validation sample β€” both work. Must contain {dim}_n_mentions and {dim}_sentiment columns.
Expert annotation files β€” one per expert
πŸ‘€Add expert file
Same structure as LLM file. Expert fills in {dim}_n_mentions and {dim}_sentiment columns for each review.
Load or create session

Sessions are saved on the server and can be resumed at any time. Each expert should use a unique session ID.

 
Load a session in step 1 first.
Complete annotation in step 2 first.