PBS DataGen Bridge

Synthetic data generation with referential integrity for SQL Databases.

"Realistic test data, without exposing real data."

Executive summary

Development, QA and migration teams need data that looks like production. Copying the real data into a test environment exposes sensitive information and runs against data-protection policy. The usual alternative, writing insert scripts table by table, does not scale: in a schema with dozens of tables chained by foreign keys you have to respect types, lengths and relationships, and a single value that breaks referential integrity breaks the test.

PBS DataGen Bridge is an accelerator in the AIAL layer, built by PBS Studio and delivered through PBS Advanced Services, that generates realistic synthetic data directly against the client's schema, on Oracle and SQL Server. It reflects the schema, infers which value belongs in each column, respects primary and foreign keys, and loads in parallel at scale. In real use it has generated and loaded on the order of billions of rows in minutes. It is not a catalog product: it was born populating test environments in real PBS projects and matured in use.

Its strongest privacy guarantee is structural. The generation engine is deterministic and runs locally, and it never moves a single real record outside the client's network. Generating synthetic data instead of copying real data is, in itself, a data-minimization control. Behind it stands PBS Group, present in 24 countries.

The challenge

Populating a test environment has two usual answers, and neither is enough. The first is to copy production data: fast, but it moves sensitive information into less protected environments and is often unworkable under regulation. The second is to generate random values with hand-written scripts: it avoids real data, but it does not scale on large schemas and produces data that neither resembles the business nor respects the relationships between tables.

The hard part is referential integrity. In a data warehouse of clients, accounts, campaigns and calls, every child row must point to a parent that exists. Generating table by table without coordinating the keys produces data the engine rejects or, worse, data that loads but leaves orphaned records and ruins any test that crosses tables.

At PBS we take a third path: automatic, schema-aware generation that is deterministic and integrity-preserving. The tool reads the real structure of the database, orders the tables by their dependencies and produces data that is consistent across them, without anything leaving the client's network.

PBS Advanced Services, PBS Studio and the AIAL layer

PBS Group — Productive Business Solutions, part of Grupo Musson — is a regional technology integrator present in 24 countries, with in-house software and artificial intelligence engineering capability. Within its offering, PBS Advanced Services brings together the advanced engineering services — migration, modernization, data and analytics — enabled by an AI acceleration layer. This is what the client contracts: solutions with outcomes, not boxed software.

PBS Studio is the innovation and research-and-development area where the company's artificial intelligence products are born; an in-house team that designs, prototypes and matures accelerators under the PBS/Grupo Musson AI Governance Framework. Its first portfolio result is AIAL — Artificial Intelligence Acceleration Layer: a unified layer of accelerators that mounts on top of the client's data projects.

PBS DataGen Bridge is the AIAL accelerator dedicated to populating test environments with synthetic data faithful to the schema, without using real data.

PBS DataGen Bridge

What it does

• Reflects the target schema: it reads columns, types, precision, lengths and primary keys from the engine without the user declaring them.
• Detects foreign keys: it recognizes the relationships defined in the database and lets you complete them from the interface.
• Assigns catalogs by name inference: an engine analyzes the name and type of each field to automatically assign it realistic data catalogs (names, amounts, dates, domain values) instead of meaningless random text.
• Feeds on the client's own catalogs: it can use the client's own codes, naming and domains as a source of values, so that the generated data reflects their business rather than generic values.
• Generates catalogs from context: it produces value catalogs from the business type, table or transaction, ranging from deterministic generators to language-model-assisted generation (names, transaction types, addresses and other domains), at the breadth — number of records — the user chooses.
• Respects type and precision: it generates valid values for each column, without the overflows the engine would reject.
• Preserves referential integrity: it orders tables from parents to children and draws each foreign-key value from the valid range of the referenced table, even if it already held prior data.
• Loads in bulk and in parallel: it uses several CPU cores to reach high insertion throughput.
• Runs from configuration: the user chooses which tables to populate and how many rows, from a web interface or in unattended runs, without touching code.
• Reports progress live: rows inserted, speed and an event log in real time.

Engineering approach

The tool combines deterministic schema reflection, an inference engine that analyzes the name and type of each field to assign it realistic data catalogs, and vectorized batch generation. The generation engine is deterministic and runs locally, which makes its behavior reproducible and keeps the client's data inside their network. The flow is multi-stage (schema discovery, dependency ordering, generation and load) with a producer/consumer scheme that overlaps generation with insertion so the database is never left idle.

Two decisions define the product. For parallelism we chose real multi-process execution over threads, so that generation, which is CPU-intensive, scales across several cores without contention. And rather than generate value by value, we generate whole columns per batch in vectorized form, which multiplies throughput; columns backed by a semantic catalog, which do not vectorize well, fall back automatically to a scalar path.

For large tables, the work is split into partitions with non-overlapping key ranges, coordinated from the real state of the database so that two processes never collide on the same key.

Figure 1. The stages of the process. The final step feeds back to the engineer with traceability and the patterns flagged for review.

What it automates and what the PBS team resolves

The tool automates reading the schema, types and keys; detecting foreign relationships; assigning a plausible value generator per column; ordering the tables by their dependencies; and bulk generation and load respecting types, precision and lengths, with live tracking.

The residual work is handled by the PBS team as part of the service: reviewing the semantic inference when a particular business domain needs the assigned catalog adjusted, defining scope and volume, expressing consistency rules between columns that cannot be inferred from the name (for example, that an end date should be later than a start date) and resolving composite-uniqueness scenarios or business-specific value distributions.

Governance and quality

The governance differentiator is data sovereignty at run time. Bulk generation is deterministic and local, connected only to the database the client points it to. When the tool generates catalogs with the help of a language model, that model receives only context descriptors — business type, table or transaction — and produces fictional values; it is never sent real client records, and it can run on local models when data sovereignty requires it.

Producing synthetic data instead of using the real data is, on top of that, a data-minimization control.

Quality rests on the PBS/Grupo Musson AI Governance Framework, consistent with the NIST AI Risk Management Framework, the ISO/IEC 42001:2023 standard and the EU AI Act. Human oversight is built into the flow: the engineer reviews and adjusts the configuration and the semantic inference before loading, and the tool is explicit about which part of the work remains in their hands.

Application case

A regional financial institution needed a test environment for its data warehouse of clients and campaigns ahead of a deployment. The schema chained dozens of tables by foreign keys (clients, accounts, campaigns, calls, incidents), and populating them by hand while guaranteeing consistency would have taken weeks, on top of exposing real data if it were copied from production.

The team pointed PBS DataGen at the schema, chose tables and volumes, and obtained millions of internally consistent synthetic rows: every account points to a client that exists, every call to a valid account. Without a single real record outside the client's network.

Working with PBS Advanced Services

We offer this capability as a continuum within PBS Advanced Services, not as boxed software. First, as an AIAL accelerator applied within our migration and modernization services. Second, as a starting point for custom AI tools: the same methodology and governance framework with which PBS Studio built this accelerator can build the one your specific problem requires. And third, as a multiplier of the speed and quality of our data engineering and consulting services. In every case, the client does not buy a binary: they buy a verified outcome.

About PBS

PBS Group (Productive Business Solutions), part of Grupo Musson, is a technology integrator with in-house software and artificial intelligence engineering capability and a presence in 24 countries across Central America, the Caribbean and South America. Through PBS Advanced Services, PBS Studio and the AIAL layer, we help organizations in the region modernize their data platforms with accelerators born from real projects and governed under a proprietary AI framework.