Architecture
arrow_forward
Security
arrow_forward
Compliance
arrow_forward
Integration
arrow_forward
Performance
arrow_forward
Exit
-
Click play or press Space to begin the deep dive...
Step- / 8
Evaluating Model Architecture
Custom model, fine-tuned, or API wrapper — and why it matters
The Architecture Question
The single most important technical question: is this a custom model or a wrapper around GPT/Claude? Neither is inherently better, but the answer determines everything about reliability, data privacy, and long-term viability. A custom model means the vendor controls the stack. A wrapper means they’re dependent on OpenAI or Anthropic — if those providers change pricing, terms, or capabilities, your vendor is affected.
Why Architecture Matters for HR
Data privacy: If the vendor wraps GPT, your employee data may pass through OpenAI’s servers. Ask explicitly.
Reliability: Custom models can be hosted in your cloud or the vendor’s. API wrappers are subject to third-party outages. When OpenAI goes down, does your screening tool go down?
Consistency: API providers retire and replace models on a schedule. Unless the vendor pins a specific model version, a prompt that worked yesterday may behave differently today. Custom models only change when the vendor retrains.
Reliability: Custom models can be hosted in your cloud or the vendor’s. API wrappers are subject to third-party outages. When OpenAI goes down, does your screening tool go down?
Consistency: API providers retire and replace models on a schedule. Unless the vendor pins a specific model version, a prompt that worked yesterday may behave differently today. Custom models only change when the vendor retrains.
Architecture Assessment
Ask the vendor:
"Is your model custom-built, fine-tuned,
or prompt-engineered on top of a
foundation model?"
If custom-built:
What architecture? (Transformer, etc.)
What was it trained on?
How often do you retrain?
Where is it hosted?
If fine-tuned:
Which base model? (a GPT, Claude, etc.)
What data was used for fine-tuning?
Do you control the base model version?
If prompt-engineered (API wrapper):
Which API provider?
Does employee data pass to the provider?
What happens during provider outages?
What's your fallback?
Are you on a fixed model version?
Technical reality: Most HR AI vendors in 2026 use API wrappers or fine-tuned models, not custom-built architectures. That’s not automatically bad — but you need to understand the data flow. If your employee PII routes through a third-party foundation model provider, your DPA needs to cover that chain.
Security Assessment
What your security team should ask — and a questionnaire template
Security Baseline
SOC 2 Type II: The table stakes report (an auditor’s attestation, not a certificate). Type I means they designed controls; Type II means an auditor verified they actually follow those controls over time. Accept only Type II.
Penetration testing: Annual third-party pen tests with remediation evidence. Not just “we do security testing” — ask for the most recent report summary.
Encryption: AES-256 at rest, TLS 1.2+ in transit. No exceptions. Ask specifically about encryption of AI training data and model weights.
Access controls: Role-based access, MFA for all admin accounts, principle of least privilege. How many vendor employees can access your data?
Penetration testing: Annual third-party pen tests with remediation evidence. Not just “we do security testing” — ask for the most recent report summary.
Encryption: AES-256 at rest, TLS 1.2+ in transit. No exceptions. Ask specifically about encryption of AI training data and model weights.
Access controls: Role-based access, MFA for all admin accounts, principle of least privilege. How many vendor employees can access your data?
Security Questionnaire Template
INFRASTRUCTURE
1. Where is data hosted? (Region, provider)
2. SOC 2 Type II certification? (Date)
3. Last penetration test? (Date, firm)
4. Encryption at rest? (Standard)
5. Encryption in transit? (Protocol)
ACCESS CONTROLS
6. MFA for all access? (Y/N)
7. # of employees with data access?
8. Background checks on staff? (Y/N)
9. Customer data segregation method?
AI-SPECIFIC
10. Does data pass to third-party AI APIs?
11. Are prompts/responses logged?
12. Is customer data used for training?
13. Model versioning and rollback?
INCIDENT RESPONSE
14. Breach notification timeline?
15. Incident response plan? (Share it)
16. Breach history in last 3 years?
AI-specific risk: Traditional security questionnaires miss AI-specific concerns. Questions 10-13 above are critical and often absent from standard vendor security reviews. Add them to your template — most vendors haven’t been asked these yet, and their answers will be revealing.
The Model Card Deep Dive
How to read a model card and what missing information tells you
What a Model Card Should Contain
A model card is a standardized document describing an AI model’s capabilities, limitations, and appropriate use. Think of it like a drug’s prescribing information — it tells you what it does, what it doesn’t, and where it might cause harm.
Intended use: What the model was designed for (and explicitly what it should not be used for).
Training data: What data the model learned from, including demographics and representativeness.
Performance metrics: Accuracy, precision, recall — ideally broken down by demographic group.
Limitations: Known failure modes, edge cases, and scenarios where performance degrades.
Ethical considerations: Bias assessment results, fairness metrics, and mitigation steps taken.
Intended use: What the model was designed for (and explicitly what it should not be used for).
Training data: What data the model learned from, including demographics and representativeness.
Performance metrics: Accuracy, precision, recall — ideally broken down by demographic group.
Limitations: Known failure modes, edge cases, and scenarios where performance degrades.
Ethical considerations: Bias assessment results, fairness metrics, and mitigation steps taken.
Model Card Analysis Example
EXAMPLE: Resume Screening Model Card
Intended Use: Rank candidates for roles
in technology and finance sectors
Missing: Does it work for healthcare? Retail?
If your industry isn't listed, ask why.
Training Data: 2M resumes, 2019-2024
60% tech, 30% finance, 10% other
Red flag: Heavy tech/finance skew.
Performance on other industries = unknown.
Performance: 88% agreement with human
reviewers on top-tier classification
Missing: Breakdown by gender, race, age.
88% overall could mask 70% for some groups.
Limitations: "Reduced accuracy for
non-English resumes"
Good: At least they disclosed it. Ask how
many of your candidates this affects.
The rule: What’s missing from a model card is as important as what’s there. No demographic breakdown of performance? The vendor either hasn’t tested for bias or doesn’t like what they found. No intended use limitations? They haven’t thought carefully about where the model fails.
Integration Architecture Evaluation
API-first vs. embedded vs. hybrid — what actually integrates with your stack
Integration Patterns
API-first: The vendor provides APIs you call from your existing systems. Maximum flexibility but requires development resources. Best for: organizations with engineering support who want the AI embedded in existing workflows.
Embedded/standalone: The AI lives in the vendor’s UI. Your team logs into their platform separately. Easiest to deploy but creates another silo. Best for: teams without engineering resources who accept a separate workflow.
Hybrid: API access plus a standalone interface. The vendor provides both. Best for: most organizations — use the UI for quick tasks, the API for automated workflows.
Embedded/standalone: The AI lives in the vendor’s UI. Your team logs into their platform separately. Easiest to deploy but creates another silo. Best for: teams without engineering resources who accept a separate workflow.
Hybrid: API access plus a standalone interface. The vendor provides both. Best for: most organizations — use the UI for quick tasks, the API for automated workflows.
Data Sync Patterns
Real-time (webhook): Changes in your HRIS push instantly to the AI tool. Ideal but complex.
Batch (scheduled): Data syncs on a schedule (hourly, daily). Simpler but data can be stale.
On-demand (API pull): The AI queries your system when needed. Flexible but adds latency.
Batch (scheduled): Data syncs on a schedule (hourly, daily). Simpler but data can be stale.
On-demand (API pull): The AI queries your system when needed. Flexible but adds latency.
Integration Evaluation Checklist
API QUALITY
REST or GraphQL? ________
API documentation quality: ___/10
Rate limits: ________
Versioning strategy: ________
Sandbox/test environment: Y/N
AUTHENTICATION
SSO support (SAML/OIDC): Y/N
SCIM provisioning: Y/N
API key management: ________
YOUR STACK COMPATIBILITY
HRIS connector: ________
ATS connector: ________
Payroll connector: ________
Custom webhook support: Y/N
RELIABILITY
SLA uptime guarantee: ____%
Average API latency: ____ms
Error handling approach: ________
Retry/backoff strategy: ________
Integration truth: AI tool deployments fail on integration more often than on the AI itself. A vendor may have a brilliant model, but if it can’t talk to your HRIS reliably, it’s useless. Test the integration during your pilot, not after you’ve signed.
Performance Benchmarking
How to test vendor claims independently with your own data
Why You Must Benchmark Yourself
Vendor-reported accuracy is measured on their data, their way. Your data is different. Your definitions of “good candidate” or “flight risk” are different. Your workforce demographics are different. The only benchmark that matters is performance on your data, measured by your standards.
This isn’t adversarial — good vendors welcome independent testing. It protects both of you.
This isn’t adversarial — good vendors welcome independent testing. It protects both of you.
Creating a Test Dataset
1. Select representative cases: A few hundred cases from your actual data (de-identified, and shared only under a signed data agreement) that span the full range of outcomes.
2. Label them: Have human experts classify each case (hired/not, high performer/not, retained/churned).
3. Include edge cases: Career gaps, industry changers, non-standard formats.
4. Ensure diversity: Your test set must represent all demographic groups in your population.
5. Blind the vendor: Don’t show them the labels. Give them the raw data and compare their outputs to your labels. Treat those labels with care: past human decisions carry past bias and inconsistency, so agreement with them shows the tool matches your history, not that it is right.
2. Label them: Have human experts classify each case (hired/not, high performer/not, retained/churned).
3. Include edge cases: Career gaps, industry changers, non-standard formats.
4. Ensure diversity: Your test set must represent all demographic groups in your population.
5. Blind the vendor: Don’t show them the labels. Give them the raw data and compare their outputs to your labels. Treat those labels with care: past human decisions carry past bias and inconsistency, so agreement with them shows the tool matches your history, not that it is right.
Benchmark Template
PERFORMANCE BENCHMARK TEMPLATE
Test Set: ___ cases from your data
Ground Truth: Human expert labels
Overall Metrics
Accuracy: ___% (matches / total)
Precision: ___% (true pos / predicted pos)
Recall: ___% (true pos / actual pos)
F1 Score: ___ (harmonic mean of P & R)
Fairness Metrics (by group)
Selection rate by gender: ___
Selection rate by race: ___
Selection rate by age band: ___
Four-fifths flag raised: Y/N
Statistical Significance
Sample size sufficient: Y/N
Confidence interval: ___
// Rule of thumb: 200+ per group compared;
// ask a statistician if it is close
// 95% CI is the standard threshold
Comparison to Baseline
Current process accuracy: ___%
AI improvement: +/- ___%
Power move: Run the same test dataset through multiple vendors. Standardized benchmarks make comparisons objective. The vendor who resists testing on your data is the vendor who knows their numbers won’t hold up.
Data Processing Agreement Analysis
Key DPA clauses, what they mean, and what to negotiate
Critical DPA Clauses
Purpose limitation: The vendor can only use your data for the specific purpose defined in the agreement. No secondary uses, no model training, no analytics for their own benefit. This clause prevents scope creep.
Sub-processor chains: Who else touches your data? If the vendor uses AWS for hosting and OpenAI for inference, both are sub-processors. You need to know the full chain and have approval rights over changes.
Data residency: Where is your data physically stored? This matters for GDPR (which restricts moving EU personal data abroad), and increasingly for state privacy laws. “Cloud-hosted” isn’t an answer — you need the region.
Breach notification: How quickly must the vendor notify you of a data breach? Under GDPR you, the employer, have 72 hours to notify the regulator, and the vendor must tell you “without undue delay.” Put a number on that in the DPA (24 to 48 hours) so your own clock is workable.
Sub-processor chains: Who else touches your data? If the vendor uses AWS for hosting and OpenAI for inference, both are sub-processors. You need to know the full chain and have approval rights over changes.
Data residency: Where is your data physically stored? This matters for GDPR (which restricts moving EU personal data abroad), and increasingly for state privacy laws. “Cloud-hosted” isn’t an answer — you need the region.
Breach notification: How quickly must the vendor notify you of a data breach? Under GDPR you, the employer, have 72 hours to notify the regulator, and the vendor must tell you “without undue delay.” Put a number on that in the DPA (24 to 48 hours) so your own clock is workable.
DPA Clause Checklist
PURPOSE LIMITATION
□ Data used only for defined services
□ No model training on customer data
□ No aggregation across customers
SUB-PROCESSORS
□ Full list of sub-processors provided
□ Prior notice of sub-processor changes
□ Right to object to new sub-processors
DATA SUBJECT RIGHTS
□ Process for access requests (DSAR)
□ Process for deletion requests
□ Reasonable response timeframes
DELETION & RETURN
□ Data returned in standard formats
□ Deletion within 30 days of termination
□ Written certification of deletion
□ Backup deletion included
BREACH NOTIFICATION
□ Notification within 24-48 hours
□ Details: scope, impact, remediation
□ Cooperation with your incident response
Non-negotiable: If a vendor’s DPA doesn’t address these items, they either haven’t thought about data protection seriously or they don’t want to commit to it. Either way, that’s a disqualifying signal for any tool that will process employee PII.
Exit Strategy Planning
Plan the exit before you sign — data portability, migration, and dependency risk
Why Plan the Exit Now
The AI vendor landscape is volatile. Companies get acquired, pivot, shut down, or change pricing dramatically. Your exit plan is your insurance policy. Negotiating exit terms after you’re dependent on a tool is like negotiating a prenup after the wedding — your leverage is gone.
Plan for three exit scenarios: voluntary switch (you find something better), vendor failure (they go under or get acquired), and contract dispute (terms change unacceptably).
Plan for three exit scenarios: voluntary switch (you find something better), vendor failure (they go under or get acquired), and contract dispute (terms change unacceptably).
Key Exit Questions
Data export: Can you export all data in standard formats (CSV, JSON) at any time, not just at termination?
Transition timeline: How long does the vendor support the transition? 30 days? 90 days? Is there a fee?
Knowledge transfer: If the AI made decisions or scored candidates, are those scores/decisions exportable and interpretable without the vendor’s tool?
Model dependency: Can your processes function without the AI tool? What’s the manual fallback?
Transition timeline: How long does the vendor support the transition? 30 days? 90 days? Is there a fee?
Knowledge transfer: If the AI made decisions or scored candidates, are those scores/decisions exportable and interpretable without the vendor’s tool?
Model dependency: Can your processes function without the AI tool? What’s the manual fallback?
Migration Risk Assessment
EXIT RISK SCORECARD
Data Portability Risk: ___/5
Export formats available?
Export includes all data + metadata?
Can export at any time (not just exit)?
Process Dependency Risk: ___/5
Can workflows run without the AI?
Is there a manual fallback documented?
How many processes depend on it?
Institutional Knowledge Risk: ___/5
Are AI decisions/scores documented?
Can you explain past decisions without
access to the vendor's tool?
Transition Cost Risk: ___/5
Estimated migration effort (weeks)?
Vendor transition support included?
Overlap period needed?
TOTAL RISK: ___/20
// 1-5: Low risk. Standard transition.
// 6-10: Moderate. Plan 60-90 day overlap.
// 11-15: High. Negotiate strong exit terms.
// 16-20: Critical. Reconsider the vendor.
The prediction problem: What happens to predictions the old AI system made? If a candidate was scored “85% match” by a tool you no longer have, can you explain that score in an audit? Ensure you can export not just data, but decision audit trails in a format that survives the vendor relationship.
Vendor Scorecard
A weighted scoring framework for objective vendor comparison
Scoring Methodology
Use this weighted scorecard to compare vendors objectively. Each category is scored 1-10 by relevant stakeholders, then multiplied by the category weight. The weights reflect what matters most for HR AI tools — functionality and data practices outweigh pricing because a cheap tool that creates compliance risk is the most expensive tool you can buy.
Who scores: Assemble a cross-functional team. HR Ops scores functionality and support. Security scores data practices. Legal scores compliance. IT scores integration. Finance scores pricing. Aggregate the scores.
Who scores: Assemble a cross-functional team. HR Ops scores functionality and support. Security scores data practices. Legal scores compliance. IT scores integration. Finance scores pricing. Aggregate the scores.
How to Use This Scorecard
1. Score each vendor independently before comparing
2. Require written justification for any score below 5 or above 8
3. Disqualify any vendor scoring below 4 in data practices or compliance
4. Calculate weighted totals and rank
5. Use the scores as input to the decision, not the entire decision — qualitative factors matter too
2. Require written justification for any score below 5 or above 8
3. Disqualify any vendor scoring below 4 in data practices or compliance
4. Calculate weighted totals and rank
5. Use the scores as input to the decision, not the entire decision — qualitative factors matter too
Weighted Vendor Scorecard Template
VENDOR SCORECARD
Vendor: ________________ Date: ________
Functionality Weight: 25%
Core AI capability: ___/10
Accuracy on your data: ___/10
Feature completeness: ___/10
Category avg: ___ x 0.25 = ___
Data Practices Weight: 20%
Data ownership terms: ___/10
Training data transparency:___/10
Export & portability: ___/10
Category avg: ___ x 0.20 = ___
Compliance Weight: 20%
Bias audit availability: ___/10
Regulatory readiness: ___/10
SOC 2 / security posture: ___/10
Category avg: ___ x 0.20 = ___
Integration Weight: 15%
API quality & docs: ___/10
HRIS/ATS compatibility: ___/10
SSO/SCIM support: ___/10
Category avg: ___ x 0.15 = ___
Pricing Weight: 10%
Total cost of ownership: ___/10
Pricing transparency: ___/10
Category avg: ___ x 0.10 = ___
Support Weight: 10%
Response time & quality: ___/10
Implementation support: ___/10
Category avg: ___ x 0.10 = ___
WEIGHTED TOTAL: ___/10
Next step: This scorecard is your starting point, not your final answer. After scoring, hold a calibration session where the evaluation team discusses scores, debates disagreements, and reaches consensus. The conversation is often more valuable than the numbers.