WhyLabs

WhyLabs was a hosted AI observability platform for monitoring data quality, model behavior, and LLM applications. The hosted service is no longer the current product, but its source code remains available.

WhyLabs open-sourced the AI Control Center on January 23, 2025. The same notice said hosted SaaS access for existing customers would continue only until March 9, 2025.

The open-source stack includes the AI Control Center, whylogs , and LangKit . This page covers those self-hosted projects rather than presenting WhyLabs as an active managed service.

What is available now?

The WhyLabs AI Control Center contains Observe and Secure components. Its repository documents model monitoring, data-quality checks, drift detection, LLM traces, policy controls, and dashboards.

The platform is Apache-2.0 licensed. Some dashboard packages depend on Highcharts, so operators must obtain the appropriate Highcharts license and restore those dependencies before building the affected interfaces.

The repository also says the project is not accepting outside contributions or feature requests. Source availability should not be confused with an actively supported hosted product.

Project status

whylogs

whylogs creates mergeable statistical profiles from data. Its documented uses include tracking distributions, setting data constraints, detecting drift, validating data quality, and monitoring model performance.

A profile contains aggregated metrics rather than individual source records. Depending on configuration and data type, those metrics can include counters, summary statistics, cardinality estimates, histograms, and up to 128 frequent items by default.

That distinction matters for privacy. Teams should audit the configured metrics and tokenize or encrypt sensitive categorical values when needed instead of assuming every profile is automatically safe for every dataset.

The basic Python flow is:

import whylogs as why

result = why.log(dataframe)
profile = result.profile()

whylogs also documents integrations with pandas, Spark, MLflow, Airflow, Amazon S3, and Google Cloud Storage.

LangKit

LangKit adds LLM-focused text metrics to whylogs. It can extract signals from prompts and responses without defining a full observability backend on its own.

Documented metric groups include:

• Text quality — Readability, complexity, and grade scores
• Relevance — Prompt-response similarity and similarity to user-defined themes
• Security and privacy — Regex patterns, jailbreak and prompt-injection similarity, hallucination consistency, and refusal similarity
• Sentiment and toxicity — Sentiment and toxicity analysis

LangKit’s README uses llm_metrics.init() to add these metrics to a whylogs schema:

import whylogs as why
from langkit import llm_metrics

result = why.log(
    {"prompt": "Hello!", "response": "World!"},
    schema=llm_metrics.init(),
)

These signals are measurements, not guarantees that a prompt is malicious or a response is safe. Teams still need thresholds, review rules, and validation against their own traffic.

Self-hosting considerations

Self-hosting restores access to the platform code, not the former managed-service experience. Operators own deployment, upgrades, storage, alert routing, and incident handling.

The Highcharts requirement is another practical constraint. The repository is Apache-2.0, but selected UI packages need a separately licensed charting dependency.

Repository activity is also limited: the three principal projects had no code pushes after January 2025 when checked on July 31, 2026. Evaluate dependency freshness and operational ownership before adopting them for a new production system.

For maintained managed platforms, review current vendor documentation and run a proof of concept against your telemetry, privacy, and deployment requirements. See the AI security tools category for other options.

Frequently Asked Questions