FREQUENTLY ASKED QUESTIONS

Which Industries Require Continuous AI Validation?

Industries that require ongoing AI validation include those that use AI to make high-stakes decisions and calculations that affect human health and safety, financial security, and legal compliance. As real-world data is constantly changing, continuous evaluation is required to monitor algorithms for model drift, hallucinations, accuracy, and bias.

How Do Enterprises Test AI Models?

Organizations test AI models through a combination of LLM-as-a-judge, RAG Validation (Retrieval-Augmented Generation) and human verified “golden datasets”. Testing aims to evaluate accuracy and safety, as well as check for hallucinations, bias and data leakage.

Why Do AI Systems Fail in Production?

AI systems frequently fail in production due to the massive gaps between how they are tested and how they are expected to perform in real-world environments. During testing, AI systems are evaluated using structured, static data sets, whereas in production, data is dynamic and unpredictable.

‍

What Is Human-in-the-Loop AI Validation?

Human-in-the-loop AI (HITL) AI validation is an evaluation framework that involves human judgement, auditing and editing of automated decisions to ensure accuracy. This hybrid AI-human approach reduces errors and hallucinations and maintains regulatory compliance.

To do this, companies set up a safety net around the AI. When the model gets confused or handles a high-risk task, the system automatically escalates that query to a human for review. Randomized audits are also run on successful tasks to catch hidden mistakes and keep operations working safely while helping the model learn over time.

‍

What Metrics Matter Most in AI Model Evaluation?

The metrics that matter most in production include escalation rate, operational consistency, failure severity and recovery, and business impact metrics. Today, successful AI model evaluation focuses on the metrics organizations use in production.

How Do Companies Evaluate LLMs in Production?

Companies evaluate Large Language Models (LLMs) in production environments using a combination of human in the loop metrics, automated evaluations and LLM-as-a-judge frameworks. AI evaluations are used to measure accuracy and safety, as well as the prevalence of hallucinations and bias in outputs.

‍

Why Are Benchmark Scores Insufficient for AI Evaluation?

AI benchmark scores are insufficient in evaluating AI models as they tend to be based on clean, static data rather than the unstructured, dynamic data used in real-world environments. This means that AI models are not trained to handle the edge cases, unpredictable user behaviour and context ambiguity that often arise outside of controlled test environments.

What is AI evaluation?

AI evaluation is the process of measuring how reliably, safely and accurately an AI model can perform the tasks it is designed to carry out. Because AI model outputs are probabilistic, evaluation needs to be rigorous to flag hallucinations, bias, inaccuracies and user safety issues.

How Are Teams Structured and Managed?

Outsourced teams are structured into units including specialists, a team lead, and quality assurance analysts.

These teams are overseen by a Service Delivery Manager (SDM) who handles daily operations and acts as the primary point of contact in the client's time zone. To support global operations, teams work across offices in the US, Europe, and Asia.

How Long Does It Take to Launch a Team?

Launching an outsourced team with BUNCH typically takes between 24 hours and four weeks from the date the contract is signed.

The exact timeline will depend on the scope of work, the level of specialization required, and the availability of client guidelines and training materials.

What Does the Onboarding Process Look Like?

The onboarding process includes defining workflows, setting performance targets, preparing training materials, and configuring the team before operations go live.

How Does BUNCH Build and Manage Outsourced Teams?

BUNCH builds outsourced teams by recruiting specialists, assigning dedicated management layers, and creating workflows around each client’s specific operation.

We manage everything from hiring and training to quality assurance, reporting, and day-to-day team performance across functions like data labeling, content moderation, customer support, and community management.

‍

How Does Outsourcing Work in Practice?

Outsourcing works by delegating a specific business process to an external partner who supplies the people, management, and infrastructure to run it.

What Problems Does Outsourcing Solve?

Outsourcing partners help growing companies solve three core problems: access to specialized processes and expertise, maintaining quality, and scaling teams.

‍

What Types of Tasks Can Be Outsourced?

Companies benefit most from outsourcing tasks that are repetitive, require human judgment and rely on high-quality data.

While outsourcing partners used to focus on speed and cost-cutting, modern companies prioritize data quality and accuracy to ensure the best results.

‍

What Is the Difference Between Managed Services and Staff Augmentation in Outsourcing?

In a managed services model, an outsourcing partner takes complete ownership and accountability for a core business function, whereas in staff augmentation, a business uses experts employed and managed administratively by the partner, but integrated into the client’s internal team.

‍

What Is the Managed Services Model?

A managed services model is a partnership where a business hands over complete ownership and accountability of a core function to an outsourcing house.

The partner takes care of everything from daily operations and staff management to compliance and client escalations on behalf of the business.

‍

Why Do Companies Choose to Outsource Operations?

Companies outsource operations to access specialized talent and established processes, reduce overall costs, and scale faster without building large internal teams.

How Fast Can We Start?

After a short scoping call, we can propose a workflow and staffing plan quickly.

‍

Can You Validate AI Outputs in Regulated Industries?

Yes, this is where structured rubrics, escalation paths, and documented processing practices matter most. BUNCH documents GDPR-aligned processing practices.

‍

What’s the Difference Between Data Verification and Validation?

Validation checks rules/requirements; verification confirms correctness vs trusted sources. We can implement both depending on your use case.

‍

Do You Provide Data Validation Tools?

We are a managed services provider: we run trained human validation teams and integrate into your existing stack. If you already use tools (testing, monitoring, labeling platforms), we complement them with human judgment and consistent QA.

‍

What Are the Hidden Costs of Outsourcing?

Most outsourcing companies have good entry prices, but include hidden costs like onboarding fees, management overhead, software licensing, and ongoing staff turnover costs.

With a managed services model like BUNCH, however, you pay a fixed monthly price that's all-inclusive, with no unexpected costs or set up fees.

Can I pay using Crypto?

Yes, we offer payment options in stablecoins for our clients. Please be aware that there may be regulatory requirements prior to engaging in crypto payments depending on the client's jurisdiction.

What If I’m Not Satisfied With Your Service?

Clients can always cancel the contract during the one-month probation period if they are not satisfied with our services. However, as of 2024, it's worth noting that no client has yet exercised this option.

What’s the Minimum Contract Duration?

The minimum contract duration is one year for recurrent campaigns. However, most of our services include a one-month probation period during which either party can cancel the contract at any time without further liabilities if they are not satisfied with the service.

What Is a Managed Services Model?

At BUNCH, we provide a "Managed Services" model, focusing on operational excellence and efficiency without requiring client oversight. Unlike the common industry practice of staff augmentation, which involves client involvement in hiring and management, we handle all aspects of service delivery. This includes talent selection, training, productivity management, and data security.By assuming full responsibility for these functions, we allow our clients to concentrate on their core business activities, offering them peace of mind regarding operations.

Our approach not only ensures high-quality service and compliance but also maintains strict data privacy and security standards, contributing to better overall business outcomes.

What Is the Story Behind the Name “BUNCH”?

When we founded BUNCH, we studied in detail the BPO market in major developing countries like the Philippines, Indonesia. We were living in South-East Asia for years and were very aware of the unmatched talent and human quality of Filipinos and Indonesians. Truly unique in this world.

‍

We were actually shocked to find out how most BPOs marketed their services. It was common to find pictures of "agents" in cubicles wearing low-end headsets and portrayed as affordable overseas labor. The entire sales pitch was to show how much companies could save by outsourcing talent to the Philippines, detailing salaries, potential savings, and even how frequently a Filipino gets sick, or when to trust them.

‍

Employee engagement was often about fast-food birthday parties and company mascots. Somehow, most considered their own talent as a bunch of kids.

‍

We found that approach deeply condescending.

‍

BUNCH was born as a hub to connect the new generation of skilled tech talent with tech jobs in global tech hubs. We opened colorful offices without cubicles. Our talent is addressed as Tech Specialists and everyone is treated as the professionals working in AI, SaaS, Fintech, and Social Media that they actually are.

‍

We are very particular about the glorification of our talent. BUNCH was then born as "A bunch of tech specialists doing crazy things with high tech."

I Want to Invest in BUNCH. How Do I Do It?

Currently, BUNCH is a self-funded, independent private company and we are not actively seeking investments. However, we are open to discussions with institutional investors. If you are interested, please reach out to us via email at founders@meetbunch.com.

When Was BUNCH Founded?

BUNCH was founded in 2017 by Carlos Puig and Rodrigo Cardenete.

Are You an AI Company?

We specialize in labeling training data at scale, which is crucial for the machine learning programs behind all AI models today. In fact, some of the models we train are likely used in products you use every day. We also utilize AI to optimize our processes, oversee quality assurance tasks, and assist our agents with various tasks. These AI-enabled processes are integral to our operations. Whether this qualifies us as an AI company may depend on your perspective.

Is Your Team In-House or Crowdsourced?

All our labelers, moderators, and agents are either full-time employees or full-time independent contractors. We do not use crowdsourced talent because maintaining high levels of accuracy, commitment to volume, and meeting tight deadlines are crucial for our clients. Additionally, employing full-time talent ensures ethical working conditions, a standard we uphold for everyone at BUNCH and proudly extend to our clients.

Are You a Call Center?

While calls constitute a small portion of our operations, much of our customer service is conducted via chat or ticket systems. In the Philippines, BPO companies are often referred to as "call centers", a term familiar to older generations. Our primary focus is on data labeling and trust & safety—complex areas that are sometimes simplified in casual conversations as "call center work."

Do You Work Remotely?

Our workforce setup is diverse, with some specialists working remotely and others on-premises. We have team members in Indonesia and Vietnam, though the majority are based in the Philippines.

Do You Have an Office?

Yes. Currently, we have a headquarters office in BGC, Manila, and a production center in Bacoor, Cavite, the Philippines. Some teams work remotely, while other members work on premises, depending on the scope of work, data security, and type of activity.

Where Is Your Team Based?

Most of our team members are based in the Philippines, including our top management and founders.

Frequently Asked Questions

|

AI Evaluation

What Is Human-in-the-Loop AI Validation?

Human-in-the-loop AI (HITL) AI validation is an evaluation framework that involves human judgement, auditing and editing of automated decisions to ensure accuracy. This hybrid AI-human approach reduces errors and hallucinations and maintains regulatory compliance.

To do this, companies set up a safety net around the AI. When the model gets confused or handles a high-risk task, the system automatically escalates that query to a human for review. Randomized audits are also run on successful tasks to catch hidden mistakes and keep operations working safely while helping the model learn over time.

‍

‍

Last Update:

June 25, 2026

Related Content

No items found.

Know more about us

Share your challenge with us and we will send you a quote personally in less than 24 hours.

Get a proposal in less than 24 hours

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form.