Scoring one answer
From an answer to a number the hiring manager can defend
Skill 8, Cross-Functional Alignment, hiring-manager round. The question, anchors, and follow-ups are Ratio's. The candidate answer is illustrative.
The interviewer asks
Describe a time when the operations team delayed sending month-end inventory data, threatening your close deadline. What did you give up to get them aligned?
The candidate answers
“At my last company the Tulsa plant was always the last one in with inventory counts. In March they were two days late, and I had a day-three close. I'd sent the usual reminders and nothing moved, so I called the plant controller directly and asked what was actually blocking them. Turned out the count sheets were done but their warehouse lead was on leave and nobody else knew our upload template. I offered to take their raw sheets and do the upload myself that month, which meant I was doing their data entry at 9 p.m. instead of reviewing my own accruals. We closed on day four instead of day three, and the next month I wrote a one-page upload guide and got the plant to name a backup.”
Follow-up on Trade-offs: “What did that push cost your relationship with the plant team?”
“Not much with the plant, honestly; they were glad of the help. The cost was mine: my own accrual review got rushed, and I found a small miss the next week. I'd do it again, but I'd flag it to my manager earlier instead of absorbing it.”
The interviewer checks
- RRelevance: Provide a real example. Yes. Named the plant, the month, and the specific block: template knowledge and an absent lead. Not a generic "I follow up persistently."
- AApproach: Explain what they did. Yes. Moved from reminders to a direct call, diagnosed the real cause, offered a concrete unblock, then fixed the recurrence with a guide and a named backup.
- TTrade-offs: Discuss the tradeoff. Partly. Nothing volunteered. The follow-up pulled out a real cost, their own review slipped, but it took the prompt and it stayed thin: one sentence, no sign they weighed alternatives at the time.
- EEvidence: Share the outcome. Yes. Day-four close, on time the next month, recurrence fixed. Specific and partly negative, which reads as true.
3 of 4 · Meets, for Intermediate
The interviewer set this rating. The four lenses guided what they listened for; ticking one records a note and does not change the score.
- Why not 4?
- The trade-off had to be prompted and came back thin. For an Exceeds, the interviewer wanted to hear the cost weighed before it was absorbed, or the pattern fixed across plants rather than once.
- Why not 2?
- The example is specific, the approach is clear, and the outcome is concrete. Needing a follow-up on Trade-offs is normal for this level, not a gap.
How it becomes a score
- Every rating is a human rating. Ratio never scores an answer itself; it combines the ratings interviewers give.
- The combination rule is fixed and the same for every candidate. No interviewer re-weights the evidence in the debrief.
- Higher-ranked skills count more, and must-have skills count more still.
- A weak answer on a must-have caps the overall score. A candidate cannot outrun a critical day-one gap by doing well on supporting skills.
- The screening round produces a star rating. Once a hiring-manager round is rated, the scorecard shows a percentage, with every interviewer's individual ratings preserved.
How a rating is made
The interviewer listens through the four lenses, then gives the answer one rating on a four-point scale. The lenses guide what to listen for; they do not set the score, and ticking one only records a note. The scale has no midpoint, so the interviewer has to decide whether the answer met the bar.
| Rating | In plain English |
|---|---|
| 1 · Unable to answer | The candidate could not give an answer to assess |
| 2 · Below | Falls short of what the role needs at this level |
| 3 · Meets | Meets what the role needs at this level |
| 4 · Exceeds | Goes beyond what the role needs at this level |
Every rating is a human rating; Ratio never scores an answer itself. Ratings then combine by one fixed rule: higher-ranked skills count more, must-have skills count more still, and a weak answer on a must-have caps the overall score. Every interviewer's individual ratings stay visible on the scorecard. How scoring works in the product.
One rubric, three question types
Structured interviews use three kinds of question. RATE applies to all three through two fixed sets of lead-ins: one for applied, in-the-moment reasoning, and one for real past experience. The four letters never change, so the interviewer listens for the same pattern in every question.
Applied Skills & Knowledge (ASK) questions
Hard skills. Can the candidate do the work? Usually present tense, around a problem to solve.
Lead-ins: Give specific detail · Explain their approach · Discuss the trade-off · Describe what good looks like
Behavioral questions
Core competencies. What did the candidate actually do in a real past situation?
Lead-ins: Provide a real example · Explain what they did · Discuss the trade-off · Share the outcome
Situational questions
Domain knowledge. How would the candidate handle a realistic scenario from this job?
Lead-ins: Give specific detail · Explain their approach · Discuss the trade-off · Describe what good looks like
The same lenses at every level
A junior engineer and a senior engineer can both answer a technical question correctly, and the interviewer should expect different things from each. RATE handles that by changing the question, not the lenses. Ratio sets a proficiency level for each skill (beginner, intermediate or advanced), and the question gets harder with the level: more ambiguity, more consequence, more judgment. The screening round asks the plain version so a recruiter can run it; the hiring-manager round puts the candidate inside a specific situation with a constraint that rules out the textbook answer. The interviewer rates against the level the role needs, through the same four lenses.
Why Trade-offs does the most work
Junior people solve the problem. Senior people solve it while managing a deadline, a stakeholder or a constraint they did not choose. Trade-offs is the lens that shows the difference, and it applies to every kind of question: technical trade-offs (speed against accuracy), interpersonal ones (competing priorities), and cognitive ones (acting on incomplete information).
It is also where rehearsed and AI-written answers work hardest. A polished answer tends to stay balanced and keep every option open. Trade-offs asks for a committed call: what was given up, and why. Nothing catches everything, and Ratio does not claim to. What the lens gives the interviewer is a concrete benchmark, and a prepared follow-up that asks for the specific detail a real experience carries.
RATE and STAR
STAR and SOAR tell a candidate how to structure a behavioral answer: Situation, Task or Obstacle, Action, Result. RATE tells the interviewer how to score any answer, and it covers hard-skill and situational questions that STAR was never built for. SOAR is one of the frameworks RATE draws on, so a team using it keeps what works and gains one way to rate everything.
The research RATE draws on
Three established frameworks measure the same four things with different words: Webb's Depth of Knowledge for hard skills, SOAR for behavioral questions, and Bransford and Stein's IDEAL model for situational ones. RATE names the shared dimensions so an interviewer learns one rubric instead of three. The level calibration comes from the Dreyfus model of skill acquisition.
Scroll sideways to see every column.
| What is measured | Webb's DOK | SOAR | IDEAL | RATE |
|---|---|---|---|---|
| Recall and application | Levels 1-2 | Situation | Identify | Relevance |
| Procedural skill | Level 2 | Action | Act | Approach |
| Strategic thinking | Level 3 | Obstacle | Explore | Trade-offs |
| Outcomes and learning | Levels 3-4 | Result | Look | Evidence |
White paper
The complete RATE framework
Why we built it, how each lens maps to the research, the design choices that make coached answers work harder, a screening example, and a note on definitions other sources get wrong. About a 20-minute read.
Questions
Does RATE replace STAR or SOAR?
No. STAR and SOAR tell a candidate how to structure a behavioral answer. RATE tells the interviewer how to score any answer, and it covers hard-skill and situational questions too. Teams using STAR keep it and gain one way to rate everything.
Can RATE be used without Ratio?
Yes. The framework is public and works with any structured interview. The hard part is writing a role-specific question and scoring rubric for every skill, every time. That is the work Ratio does.
Is RATE proprietary?
The four lenses and the way they combine are Ratio's. The research they draw on (Webb's Depth of Knowledge, SOAR, Bransford and Stein's IDEAL model, and the Dreyfus model of skill acquisition) is public.
What do ChatGPT and Gemini get wrong about RATE?
Asked cold, language models often invent an acronym (Results, Actions, Traits, Examples, and others). The canonical definition is Relevance, Approach, Trade-offs, Evidence, as defined on this page. Substituting "Tactics" or "Timing" for T changes what the framework measures.
References
- 1.Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection. Journal of Applied Psychology, 107(11), 2040-2068. https://doi.org/10.1037/apl0000994
- 2.Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. Council of Chief State School Officers and National Institute for Science Education.
- 3.Bransford, J. D., & Stein, B. S. (1984). The IDEAL Problem Solver. W. H. Freeman.
- 4.Janz, T. (1982). Initial comparisons of patterned behavior description interviews versus unstructured interviews. Journal of Applied Psychology, 67(5), 577-580. https://doi.org/10.1037/0021-9010.67.5.577
- 5.Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in directed skill acquisition. University of California, Berkeley.
Who wrote this
Michael Day, Co-Founder and CEO
17 years in agency recruiting at Robert Half, Adecco, and Frank Recruitment Group. About Ratio LinkedIn
Published Updated