Introduction

Dedomena Edge is the on-premise edition of the Dedomena.AI platform: a Python library you install on your own servers to check the quality of your datasets, detect sensitive data, anonymize it and train synthesizers for synthetic data, without your data ever leaving your infrastructure.

This guide explains how to download and install Dedomena Edge, how it is organized, what you need to run it and how to use each of its features. At the end you will find a reference manual with every command, variable and error message.

> Your data stays in your environment. Edge only connects to Dedomena.AI to download the package and the models. Detection, anonymization, training and validation always run on your machine, and the licence is verified locally, offline.

What Dedomena Edge includes

Edge brings three libraries together in a single Python package, dedomena:

Together they cover every kind of asset:

Anonymization method names, sensitive element types and synthesis algorithms are the same as in the Dedomena.AI cloud platform. A configuration written for the cloud works the same way in Edge.

Architecture and requirements

How an installation is organized

An Edge installation has three parts, and all three live in your infrastructure:

• The library is the code you run from Python. It is installed in a virtual environment and processes your data locally.

• The models are the AI models used by detection. They are downloaded separately from the code and stored in a local folder. You can choose their size, or use none at all (see Model size: small, full or none).

• The licence is a signed token you get from the Edge page of the platform. The library verifies it locally, without connecting to anything.

The Dedomena.AI platform is only involved in downloading the package, the models and the token from the Edge page, and in giving you the results once you upload to it the encrypted files your workflows produce: for example, the synthetic data generated from a synthesizer you trained with Edge (see Generating data).

What leaves your machine and what does not:

Modules

The package is delivered in modules: you choose the ones you need and only those are installed. You can choose several at once (for example, Text + Vision).

A few things to keep in mind:

• Axon is always included, whatever you choose.

• Text includes Tables: the language model is also used on the free-text columns of tables.

• If you try to use something your package does not include, the library does not fail with a Python error: it tells you that module is not part of your installation and which modules it does include.

Model size: small, full or none

The code package is the same for every size. The size only decides which models are downloaded:

Models downloaded for each module and size:

• With several modules, the union of their models is downloaded, without repeats. When the full model of a type is included, the lightweight model of that same type is not downloaded as well.

• Vision and audio always need a model. Without one there is no image detection and no transcription, so the none size is not offered with those modules.

• Generative image editing (optional). An additional model of about 57.7 GB that replaces the data in an image with other data that looks natural, instead of drawing it on top. Only available with the Vision module and GPU hardware.

System requirements

Operating system and Python

CPU and memory

GPU (only for gpu hardware)

Before loading any model you can check whether they fit on your card with models.report() (see Which engine will be used).

Disk

• Library and dependencies: a few GB (more in the GPU variant, which includes the NVIDIA libraries).

• Models: as shown in the table in Model size: small, full or none. Leave extra room for your data and results.

Network

• Installing with pip: outbound HTTPS to api.dedomena.ai and to the public Python package repositories during installation.

• Downloading models with dedomena-edge setup: outbound HTTPS to api.dedomena.ai and to the download servers it redirects to. Ask Dedomena.AI for the list of domains if your firewall works with an allowlist.

• Installing from the full download: the target machine needs no network.

• Usage: no network needed.

Which combination to choose

Download and installation

There are two ways to install Edge. Both start from the Edge page of the Dedomena.AI platform:

> You need a platform account with a plan that includes Dedomena Edge. If your plan does not include it, the download replies that you have no access to Edge: contact Dedomena.AI.

The Edge page

The page has three blocks, from top to bottom:

1. The package options: operating system, modules, hardware and model size.

2. Your licence token.

3. The two ways to install: Download and Install with pip.

Choosing the options

1. Operating system: Linux.

2. Modules: select one or more. Modules included with another one are selected automatically and locked (for example, selecting Text selects Tables). Axon is always selected. All installs everything.

3. Hardware:

   • CPU: for machines without a GPU. Installs the lightweight CPU build of the computing libraries.

   • GPU: for machines with an NVIDIA GPU (CUDA 13, driver 580 or later).

4. Model size: small, full or none (see Model size: small, full or none). The page disables the combinations that are not possible:

   • full with CPU hardware, if you choose Text, Vision or All (those models need a GPU);

   • none, if you choose Vision, Audio or All (they need a model).

5. Generative image editing (optional): only shown with Vision and GPU hardware.

Below the download button, the page shows what the download contains and how much the models weigh.

Your licence token

The token is your Edge licence. You need it to install, and the library uses it every time you work with it. It is personal: do not share it or commit it to a code repository.

• Show displays the token and Copy copies it.

• Generate new token generates a new one. Use it when the current one expires.

• Valid until shows the expiry date.

The installation commands on the page already include your token. More details in The licence.

Option A: full download (no internet on the target machine)

1. Click Download and choose a folder on your computer.

   • In Chrome and Edge, the browser writes the files directly into that folder, checks each one and, if the download is interrupted, you can resume it where it stopped.

   • In Firefox and Safari, the download is saved as a single .zip. If it is interrupted, it has to start over. With full models (tens of GB) we recommend Chrome or Edge.

2. When it finishes you have a dedomena-edge folder with these contents:

   Item	Contents
   install.sh	The installer.
   wheels/	The Edge package and all its dependencies, for Python 3.10, 3.11 and 3.12.
   models/	The chosen models. Not present with the none size.

3. Copy the folder to the machine where you will use Edge, by whatever means you normally use (SFTP, disk, internal repository…). If you downloaded a .zip, extract it there.

4. Install:

   bash

   
   cd dedomena-edge
   
   bash install.sh \<TOKEN\>
   
   . ./env.sh          # in every new session, before using the library
   
   

What install.sh does:

1. Chooses where to install (see the table below) and checks that the Python version is 3.10, 3.11 or 3.12.

2. Installs Edge from ./wheels, without an internet connection.

3. Writes env.sh, which activates the virtual environment (if it created one) and points the library to the ./models folder (if there are models).

4. Saves your licence on the machine, so the library finds it without you having to pass it again.

5. Runs dedomena-edge status, which shows the installed modules and models.

Where it installs:

Additional options:

• PYTHON=/path/to/python3.11 bash install.sh \<TOKEN\> uses a specific interpreter.

• The token can also be passed in the DEDOMENA_TOKEN variable instead of as an argument.

• bash install.sh --help shows the help.

> Watch out with conda. If your terminal activates conda's base environment automatically, the installer treats it as the "active environment" and installs there. Use --venv if you prefer a separate environment.

> Do not move the folder after installing. The virtual environment and env.sh use absolute paths, and the models stay in ./models. If you need another location, copy the folder to its final place before running install.sh.

If something fails during installation:

Option B: install with pip

1. Create and activate a virtual environment for Edge:

   bash

   
   python3.11 -m venv /opt/dedomena/venv
   
   . /opt/dedomena/venv/bin/activate
   
   

2. Install the package with the command shown on the page. It already carries your modules, your hardware and your token:

   bash

   
   pip install --index-url "https://licence:\<TOKEN\>@api.dedomena.ai/edge/\<modules\>/\<hardware\>/simple/" dedomena-edge
   
   

   • \<modules\>: all, one module (tables, text, vision, audio, documents, axon) or several joined with + (for example text+vision). The order does not matter.

   • \<hardware\>: cpu or gpu.

   • The token is the index password; the user name (licence) is ignored.

   With gpu hardware and full models, also install the GPU engine used by the full models:

   bash

   
   pip install --index-url "https://licence:\<TOKEN\>@api.dedomena.ai/edge/\<modules\>/gpu/simple/" "dedomena-edge[gpu]"
   
   

3. Download the models and save the licence on the machine:

   bash

   
   dedomena-edge setup --size small --token "\<TOKEN\>"     # or --size full / --size none
   
   

   setup reads which modules your installation has and downloads the models they need for that size. If a download is interrupted, run the same command again: it resumes where it stopped. Each file is verified when it finishes, and a half-downloaded model is never used.

4. Check the installation:

   bash

   
   dedomena-edge status
   
   

dedomena-edge setup options:

setup messages you may see:

Common pip errors:

Checking the installation

dedomena-edge status

dedomena-edge 5.6.2

Modules    : axon, small, tables, text

Models dir : /home/user/.local/share/dedomena/models

GPU        : no

  small-text       1.0.0

• Modules shows the modules of the installed package. small is the code for the lightweight models, which comes with every package that includes detection.

• Models dir is the folder where the library looks for the models.

• Below, one line per installed model, with its version.

To see the licence:

python -c "import dedomena; print(dedomena.auth_status())"

And to see which detection engine each modality will use and whether the models fit on your GPU:

python -c "from dedomena.detection import models; print(models.report())"

Updating, extending or changing the installation

The licence

The licence is the token from the Edge page. It is a signed token that carries your identifier, your permissions and its expiry date. The library verifies it locally, without connecting to anything, so it works on machines without internet access.

Saving the licence on the machine

install.sh and dedomena-edge setup --token already save it for you. If you need to do it by hand (for example, when renewing it):

import dedomena

dedomena.login("\<TOKEN\>", persist=True)   # verifies and saves it

dedomena.auth_status()

# Session(subject='...', scopes=('axon', 'detection', 'nucleus.anonymize', ...), expires=..., source=...)

From then on you do not need to pass the token to any function: they all find it on their own.

• persist=True saves the token in ~/.config/dedomena/token, readable only by your user. Without persist=True, login() applies to the current Python session and writes nothing to disk.

• dedomena.logout() closes the session.

Where the library looks for the token

In this order:

1. The token= argument of the function you call.

2. The session opened with dedomena.login(...).

3. The DEDOMENA_TOKEN environment variable.

4. The saved licence file (~/.config/dedomena/token).

Containers, CI and secret managers. If your policy does not allow credentials to be stored on disk, pass the token through the DEDOMENA_TOKEN variable and set DEDOMENA_NO_PERSIST=1: the library will not write anything to the user's home directory. DEDOMENA_CONFIG_DIR changes the folder of the licence file.

Permissions

The token states what you can do:

If you try something your licence does not cover, the library says so, naming the activity: Your Dedomena licence does not cover training a synthesizer. It covers: …. In that case, ask Dedomena.AI for the corresponding permission.

Expiry and renewal

• The expiry date is shown on the page (Valid until) and in dedomena.auth_status().

• 15 days before it expires, the library warns: Warning: Your Dedomena token expires on … (N day(s) left).

• Once it expires, functions stop running with the message Your Dedomena token expired on …. Generate a new token on the page and save it with dedomena.login("\<NEW_TOKEN\>", persist=True). No need to reinstall anything.

• Turning the system clock back does not extend the licence.

Getting started with the library

Before you start

If you installed with the full download, load the environment in every new session:

. /path/to/dedomena-edge/env.sh

Then:

import dedomena

print(dedomena.__version__)

from dedomena.nucleus import anonymize            # anonymization

from dedomena.nucleus import Auto, Generic        # synthesizers

from dedomena.detection import detect             # detection

from dedomena.axon import Inspector, clean        # data quality

import dedomena loads nothing heavy: each library is loaded the first time you use it.

Data sources

Every function that takes data accepts the same sources:

Databases. PostgreSQL, MySQL/MariaDB, SQL Server, Oracle, SQLite and IBM Db2 are supported:

anonymize(

    "postgresql://user:password@server:5432/crm",

    source_format="DATABASE",

    query="SELECT * FROM customers WHERE signup \>= '2026-01-01'",

    config="auto",

    output="output/customers_anonymized.parquet",

    output_format="PARQUET",

)

If query is a single word, it is read as a table name (SELECT * FROM "customers").

IBM Db2. Download IBM's JDBC driver (db2jcc4.jar) and copy it into the DB2Driver folder inside the installed package. You also need a Java runtime on the machine:

python -c "import dedomena, os; print(os.path.join(os.path.dirname(dedomena.__file__), 'DB2Driver'))"

The URL looks like this:

jdbc:db2://\<host\>:\<port\>/\<database\>:sslConnection=true;user=\<user\>;password=\<password\>;

Db2 JDBC drivers: <https://www.ibm.com/support/pages/db2-jdbc-driver-versions-and-downloads\>

The asset (Asset)

Almost every function returns an asset (dedomena.core.Asset): an object that carries the data and what is known about it.

You can pass an asset from one function to the next: Axon prepares, detection finds, Nucleus anonymizes or synthesizes, and the history is kept. To save it:

asset.save("output/", data_format="CSV")   # output/data.csv + output/meta.json

Messages, logs and support

The library does not show Python tracebacks. When something fails, you see a single line:

Dedomena error [3f9a1c02]: Failed to train the synthesizer.

Details were written to /path/logHistory.log

• DEDOMENA_LOG_FILE=/path/my_log.log changes the location of the log. Useful with several jobs in parallel.

If you need help, send support the error identifier ([3f9a1c02]), logHistory.log and, if present, dedomena-support.enc.

Sensitive element detection

Detection finds the sensitive data in an asset and says where it is. Anonymization runs it on its own when needed, but you can also use it directly: to audit what data you hold, to review the findings before anonymizing, or to feed your own processes.

detect()

from dedomena.detection import detect

detect("documents/id_card.png")       # image

detect("documents/contract.pdf")      # PDF, DOCX or TXT

detect("calls/support.mp3")           # audio

detect(dataframe)                     # table: which column is of which type

detect("My DNI is 12345678Z")         # free text

It returns a list of elements with the same shape for every modality: the value, its type and where it is.

It requires the detection permission. If your package does not include the asset's modality, the function tells you so with the message Detection for '…' is not part of this build.

How detection works

Edge combines several layers, and they always work together:

• Patterns and validations (always on): emails, phone numbers, Spanish DNI/NIE/CIF with their check letter, IBAN with its check digits, cards with their verification algorithm, licence plates, IP addresses… They win any overlap: a number that passes its validation is a fact.

• AI models (depending on the installed size): they find what no pattern can reach, such as names, companies, free-form addresses, special categories of data (health, religion, trade union membership…), faces, signatures, logos and fingerprints.

• Reading text in images (in Vision and in scanned PDFs): the text in the image is read and analysed like any other text, in Spanish, English, Basque, Catalan and Galician.

• Transcription (in Audio): the audio is transcribed with per-word timestamps, and detection runs on the transcript. Numbers dictated in words ("six, one, two…") are also recovered when they pass their validation.

The library automatically picks the best engine installed for each modality (see Models: automatic, registered and your own). The shape of the result does not change with the engine; only the coverage does.

> No image model, no image detection. If your package includes Vision and you have not downloaded any model, the first image fails with a message that gives the command: dedomena-edge setup --size small. That way an image is never delivered after being only half-checked.

Sensitive element types

Each element has a type: the code you use in the anonymization configuration. The types are the same as in the cloud platform.

To limit text detection to specific types:

from dedomena.detection import detect_in_text

detect_in_text("Call 612 345 678 or write to ana@example.com", types=["phone_number"])

detect_in_text(text, certain_only=True) returns only the values that passed a validation (check letter, check digits…).

Tables: which column is of which type

from dedomena.detection import detect_columns

detect_columns(dataframe)

# [\{'value': None, 'type': 'email', 'column': 'mail'\}, \{'value': None, 'type': 'dni', 'column': 'document'\}, ...]

It needs the Tables module. Columns are classified by their name and by a sample of their values.

Reviewing and reusing a detection

You can detect, review the result (or have a person review it) and then anonymize exactly what was reviewed, without detecting again:

from dedomena.core import Asset

from dedomena.detection import detect

from dedomena.nucleus import anonymize

asset = Asset("documents/id_card.png")

asset.sensitive_elements = detect(asset)

asset.save_elements("review/id_card.elements.json")      # JSON to review or edit

# ... review ...

reviewed = Asset.load_elements("review/id_card.elements.json")

anonymize(reviewed, config="auto", output="output/id_card_anonymized.png")

An asset with its elements assigned is considered already reviewed: anonymization does not detect again and does not overwrite the review.

Audio in batches

To process many audio files, use the batch functions. Each file writes its own result, so a long run can be resumed:

from dedomena.detection import audio

paths = audio.discover("/data/calls", recursive=True)

results = audio.detect_many(

    paths,

    output_dir="/data/output/detections",     # one \<audio\>.detection.json per file

    types=\{"iban", "credit_card_number", "credit_card_security_code", "credit_card_expire"\},

    resume=True,                              # skips files already processed

    return_transcript=True,

    workers=1,

)

audio.write_summary(results, "/data/output/detections")

For a single file:

aligned, unaligned, transcript = audio.transcribe_and_detect(

    "call.mp3", language="es", return_transcript=True)

• aligned are the elements with their time range.

• unaligned are the detected values that could not be placed in time. They are returned separately so they are never silently lost.

• If you keep the transcript and its timestamps, you can repeat detection with other types without transcribing again: audio.detect_in_transcript(transcript, words, types=…).

• device="auto" (default) uses the GPU for lightweight transcription if available. device="cpu" forces the CPU.

Anonymization

Anonymization modifies the sensitive parts of your real data and leaves everything else exactly as it was. The result is a new anonymized file. It requires the nucleus.anonymize permission; the detection it runs internally does not need the detection permission.

First example

import dedomena

from dedomena.nucleus import anonymize

result = anonymize(

    "data/customers.csv",

    config="auto",                                   # detects and applies each element's default method

    output="output/customers_anonymized.csv",

    output_dir="results",                            # mapping, run log and conformance report

)

print(result.content)    # 'output/customers_anonymized.csv'

anonymize() works the same way with a table, an image, a document or an audio file: the type is inferred from the source.

anonymize() parameters:

It returns an Asset whose content is the path written (or the DataFrame, with output_format="DATAFRAME") and whose history records the anonymization.

The configuration

The configuration says what to do with each column (in tables) or with each element type (in images, documents and audio).

Automatic mode. config="auto" detects the sensitive elements and applies each one the default method of its type (see Which methods each type accepts).

Short form. The method name:

config = \{

    "email":  "mask",

    "salary": "perturb",

    "notes":  "drop",

\}

Extended form. A dictionary with the method and, if it takes one, its parameter (param):

config = \{

    "dni":         \{"method": "pseudonym", "param": r"CUS-\d\{6\}"\},

    "name":        \{"method": "simulation", "param": "name"\},

    "email":       \{"method": "simulation", "param": "email", "derive_from": "name"\},

    "postcode":    \{"method": "mask", "param": 3\},

    "signup_date": \{"method": "date_shift", "param": "customer_id"\},

    "code":        \{"method": "mask", "type": "string"\},

\}

• In tables, the keys are the column names. Columns not listed in the configuration are left untouched.

• In images, documents, audio and video, the keys are the element type codes (see Sensitive element types). Upper case, spaces and underscores are all accepted.

• Method names accept synonyms: simulate is simulation, tokenize is coding, blackout is redact, mute is silence, and so on. The names used by the cloud platform are also accepted.

Anonymization methods

Methods that need the language model. text_regenerate needs the Text module with a registered model (see Registering a model explicitly). Without it, anonymization stops with an error instead of delivering the column empty or untouched. It cannot be used on columns that are not free text either: for a column of URLs or codes, use simulation or coding.

Which methods each type accepts

The method must be valid for the type of the column or element. The first one in each list is the one config="auto" applies.

date_shift is also accepted on text columns, because dates read from a CSV usually arrive as text.

Tables

from dedomena.nucleus import anonymize

anonymize(

    "data/employees.parquet",

    config=\{

        "dni":        \{"method": "pseudonym", "param": r"EMP-\d\{6\}"\},

        "name":       \{"method": "simulation", "param": "name"\},

        "email":      \{"method": "simulation", "param": "email", "derive_from": "name"\},

        "phone":      "mask",

        "salary":     \{"method": "perturb", "param": 0.1\},

        "birth_year": \{"method": "generalize", "param": 10\},

        "comments":   \{"method": "coding", "param": "tokens"\},

    \},

    output="output/employees_anonymized.parquet",

    output_format="PARQUET",

    output_dir="results",

    country="Spain",

)

• Tables are anonymized on CPU. They only need a model if you use text_regenerate, or if you want coding to find names and companies inside free text (with the lightweight or the full model).

• To get the result in memory: output_format="DATAFRAME", and it is in result.content.

• If you do not set output, the file is written to the current folder as \<name\>_anonymized.\<ext\>.

Images

anonymize(

    "documents/id_card.png",

    config=\{

        "face":      "blur",

        "signature": "redact",

        "name":      \{"method": "simulation", "param": "name"\},

        "dni":       "mask",

        "birthdate": "mask",

    \},

    output="output/id_card_anonymized.png",

)

• Visual elements (face, signature, logo, fingerprint): the whole region is hidden, including any text on top of it.

• Text inside the image: the original value is erased and the new value is drawn to fit its space, with colours taken from the image itself.

• MRZ zone of passports and ID documents: replaced by a valid synthetic MRZ (with its check digits), consistent with the document's already anonymized data. If it cannot be rebuilt safely, it is pixelated.

• Generative editing (optional, requires its model and a GPU): if installed, text is replaced with a natural look instead of being drawn on top. If not installed, the drawing method is used, which is always predictable.

• If a configured element has no location, anonymization fails instead of delivering an image that looks anonymized but is not.

Image options (image_options):

Documents

anonymize(

    "documents/contract.pdf",

    config=\{"name": "pseudonym", "address": "mask", "iban": "hash", "signature": "redact"\},

    output="output/contract_anonymized.pdf",

)

• PDF: sensitive text is removed from the document, not just covered (it cannot be recovered by selecting or copying it), and the replacement is written with the original font, size and colour. Form fields, embedded images and visual regions (signatures, faces, logos) are anonymized too.

• DOCX: the body, tables, headers and footers are processed. A modified paragraph may lose any character formatting it had inside.

• TXT: direct text replacement.

Audio

anonymize(

    "calls/customer_support.mp3",

    config=\{"name": "beep", "phone_number": "beep", "iban": "silence", "address": "beep"\},

    output="output/customer_support_anonymized.mp3",

    audio_options=\{"language": "es"\},

)

Audio options (audio_options):

• If two segments overlap, the most destructive method is applied (remove > silence > beep).

• The result keeps the input format.

Deliveries of several assets

When you anonymize several related assets (several tables from the same system, or a table and the documents of those same customers), use the Anonymization class. It guarantees that:

• the same person receives the same replacement in every asset;

• the dates of the same entity are shifted equally in every table;

• foreign keys (references) still match after anonymization.

from dedomena.nucleus import Anonymization

policy = \{

    "customer_id": \{"method": "pseudonym", "param": r"C-\d\{6\}"\},

    "name":        \{"method": "simulation", "param": "name"\},

    "email":       \{"method": "simulation", "param": "email", "derive_from": "name"\},

    "date":        \{"method": "date_shift", "param": "customer_id"\},

\}

tables = [

    \{"name": "orders",    "data": "data/orders.csv",

     "config": \{**policy, "customer_id": \{"method": "pseudonym", "references": "customers.customer_id"\}\}\},

    \{"name": "customers", "data": "data/customers.csv", "config": policy\},

]

anon = Anonymization(config=policy, output_dir="delivery_october", country="Spain")

for table in anon.ordered(tables):          # referenced tables first

    anon.run(table["data"], name=table["name"], config=table["config"],

             output=f"delivery_october/\{table['name']\}.csv")

anon.conformance()                          # conformance report as PDF and JSON

print(anon.findings)                        # anything left untreated

• ordered() sorts the tables so that a referenced table is anonymized before the tables that reference it.

• run() accepts the same sources as anonymize(), plus name (the table's name in the delivery, which references point to) and config (to override the policy for that asset).

Files generated

The conformance report describes what was detected, what was treated and with which method, which values changed, which entities are shared between assets, and what each treatment protects against (and what it does not). If anything was left untreated (a configured column that does not exist, an audio value that could not be placed in time…), it appears in the report and is also printed on screen when that asset finishes, even with verbose=False.

Generated-content marking

When a method writes new values (simulation, text_regenerate or coding in invented mode) and has treated at least one element, Edge marks the result as AI-generated content, as required by Article 50 of the EU AI Act:

• the asset's history says so (result.ai_generated is True);

• a \<file\>-metadata.json file with the provenance information is written next to the file.

Methods that hide, remove or transform what was there (mask, hash, pseudonym, blur, pixelate, redact, beep, drop, perturb, shuffle, generalize, date_shift) do not mark the result. Synthetic data is always marked.

Reversal and incremental deliveries

Reversible methods (mask, pseudonym, simulation, generalize, coding, date_shift, text_regenerate) save their mapping in output_dir. With it, an authorized person can reverse an anonymized table:

from dedomena.nucleus import deanonymize

original = deanonymize("output/customers_anonymized.csv",

                       mapping_dir="results",

                       output="reversed/customers.csv")

hash, shuffle, perturb, drop and the visual and audio methods cannot be reversed. Reversal applies to tables.

Incremental deliveries. To anonymize the next month's data with exactly the same replacements as the previous delivery (the same person gets the same pseudonym, their dates are shifted the same way), load the previous delivery:

from dedomena.nucleus import Anonymization

anon = Anonymization.load("delivery_october")

anon.run("data/customers_november.csv", name="customers",

         output="delivery_november/customers.csv")

Synthetic data

A synthesizer learns the statistical properties of your real data, so that new synthetic data can be generated from it without copying the original records. The workflow is always the same:

1. Train the synthesizer on your data, on your machine (train).

2. Evaluate its quality, privacy and utility (evaluate).

3. Save it to a file (save).

4. Upload the synthesizer to the Dedomena.AI platform.

5. Generate the synthetic data from the platform, or through the API.

Training requires the Tables module and the nucleus.synthesis permission. Synthetic data is generated on the Dedomena.AI platform, from the synthesizer you upload (see Generating data).

Choosing the algorithm

from dedomena.nucleus import Auto, choose_algorithm

choose_algorithm("data/sales.csv")            # ('transactional', 'reason…'), without training anything

model = Auto(epochs=150)

model.train("data/sales.csv")

print(model.algorithm, model.reason)

Auto never chooses TimeSeries: if your data is a time series, use that class directly.

Full example (Generic)

import dedomena

from dedomena.nucleus import Generic

# 1. Configure

model = Generic(

    synthesizer_name="customers_v1",

    synthesizer_description="Retail customers 2026",

    epochs=200,

    batch_size=256,

    configuration=\{

        "categorical_columns": ["province", "segment"],

        "integer_columns": ["age"],

        "float_columns": ["income"],

        "date_columns": \{"signup_date": "%Y-%m-%d"\},

        "id_columns": \{"customer_id": r"CUS-\d\{6\}"\},

        "target": "segment",

    \},

    sensitive=\{"name": "name", "email": "email", "dni": "dni"\},

    constraints=["age\>=18", "income\>=0"],

)

# 2. Train

model.train("data/customers.csv")

# 3. Evaluate (writes report/report.pdf and report/evaluation.json)

privacy, quality, utility = model.evaluate(output_dir="report")

# 4. Save (writes synthesizers/synthesizer.zip, with the report inside)

model.save("synthesizers")

# 5. Upload synthesizers/synthesizer.zip to the Dedomena.AI platform and generate the data there

Common options

These options are passed when creating Generic, Transactional, TimeSeries or Auto:

Column types (inside configuration):

Columns you do not declare are classified automatically, and the library tells you how it classified them. No column is discarded: if you do not want a column in the model, remove it from the data before training.

> Column types go inside configuration. Passing them as separate arguments (categorical_columns=[…]), as in previous versions, raises an error when creating the model.

Transactional

For an entity's movements over time. Use columns_mapping to say which column of your data plays each role:

from dedomena.nucleus import Transactional

model = Transactional(

    columns_mapping=\{"user_id": "account", "txn_date": "date", "amount": "amount_eur",

                     "concept": "description", "balance": "balance_eur", "cat_id": "category"\},

    balance_updated=True,

    transform_descriptions="level2",

    configuration=\{"categorical_columns": ["category"], "date_columns": \{"date": "%Y-%m-%d"\}\},

    datasets_country="Spain",

    epochs=250,

    synthesizer_name="transactions_v1",

)

model.train("data/transactions.parquet")

model.save("synthesizers/transactions")

• balance_updated: True if each row's balance already includes that row's amount; False if it includes the previous row's. The balance of the generated data is recomputed from the amounts.

• With a cat_id column, consistency between description and category is preserved.

Transaction descriptions (transform_descriptions, default "level2"): the generated descriptions are rewritten so they do not repeat real texts and are consistent with their row's amount and date.

Levels 2 and 3 make use of the detection and text models if they are installed. Without them, they rely on patterns and fictitious values.

Time series

For periodic measurements of several entities. Use columns_mapping to give the entity (user_id), the date (txn_date) and the measurement (amount). At least 5 distinct entities are needed.

from dedomena.nucleus import TimeSeries

model = TimeSeries(

    columns_mapping=\{"user_id": "meter", "txn_date": "date", "amount": "kwh"\},

    static_columns=["tariff", "province"],      # constant for each entity

    epochs=300,

    batch_size=64,

)

model.train("data/readings.csv")

model.save("synthesizers/readings")

• Series are modelled with one observation per day. Date columns other than txn_date are not used.

• When you generate data from a time-series synthesizer, the amount you request is the number of series (entities), not of rows.

Relational

For several related tables. Describe each table in datasets_config and pass the data of all of them to train():

from dedomena.nucleus import Relational

config = \{

    "customers": \{

        "primary_key": "customer_id",

        "categorical_columns": ["segment", "province"],

        "integer_columns": ["age"],

        "sensitive": \{"name": "name"\},

        "amplify": "default",

    \},

    "orders": \{

        "primary_key": "order_id",

        "foreign_key": \{"customer_id": "customers"\},

        "date_columns": ["order_date"],

        "float_columns": ["total"],

        "amplify": "default",

    \},

\}

model = Relational(datasets_config=config, epochs=200, datasets_country="Spain",

                   synthesizer_name="shop_v1")

model.train(\{"customers": "data/customers.csv", "orders": "data/orders.parquet"\})

model.save("synthesizers/shop")

Per-table keys in datasets_config:

• Relationships must form a graph without cycles, and every foreign key must point to a primary key. Rows whose keys point nowhere are discarded during training, with a warning.

• Relational training requires 150 or more passes.

• When you generate data from a relational synthesizer, the amount you request is the number of complete copies of the schema. Keys still match across the generated tables.

• A relational synthesizer cannot be retrained: train a new one with the updated data.

Constraints

Rules the generated data always meets. They are written without spaces:

A malformed constraint raises an error during training.

Generating data

Synthetic data is generated on the Dedomena.AI platform, from the synthesizer you train with Edge:

1. Save the synthesizer with model.save(folder). It writes synthesizer.zip in that folder.

2. Upload synthesizer.zip to the Dedomena.AI platform, from the web application or through the API.

3. Generate the synthetic data from the platform, or through the API.

The generation options (number of records, conditional generation…) are described in the platform and API documentation. Synthetic data is always marked as AI-generated content.

Saving, loading and retraining

model.save("synthesizers/customers")              # → synthesizers/customers/synthesizer.zip

model = load_synthesizer("synthesizers/customers/synthesizer.zip")

model.metadata                                    # name, algorithm, columns, scores…

• save() takes a folder and always writes synthesizer.zip inside it. If you ran evaluate() before, the report is included.

• load_synthesizer(path) loads a saved synthesizer, for example to retrain it.

• Protect synthesizer.zip as sensitive data: it contains the trained model and may contain samples of the training data.

Retraining with new data (Generic, Transactional and TimeSeries only):

model = load_synthesizer("synthesizers/customers/synthesizer.zip")

model.retrain(data="data/customers_november.csv", epochs=100)

model.save("synthesizers/customers_v2")

• Always set epochs.

• With amplify="quality", forgetfulness_factor (between 0 and 1) decides how much of the previous data is forgotten: 0 keeps everything, 1 starts from scratch.

Evaluation and report

privacy, quality, utility = model.evaluate(output_dir="report")

It measures the synthesizer against the real data it was trained on and returns three scores from 0 to 100:

The following are written to output_dir:

• report.pdf: the report, with the scores, each metric explained and its charts;

• evaluation.json: the same results as data, to integrate into your processes;

• logHistory.log.

Each score comes with a rating: critical (up to 40), poor (up to 60), fair (up to 80) or excellent.

• Control data. To measure singling-out and linkability risk, set aside, before training, a part of the real data the model does not see, and pass it as control_data: model.evaluate(output_dir="report", control_data="data/control.csv").

• Evaluation needs at least 3 typed columns that are neither sensitive nor identifiers.

• Run evaluate() before save() so the report is included in the synthesizer.

• In relational, evaluate(output_dir=…) returns the average scores and includes referential integrity.

Evaluating any two datasets. To compare a real dataset with synthetic data you generated on the Dedomena.AI platform:

from dedomena.nucleus.evaluations import synthesizer_evaluations

privacy, quality, utility = synthesizer_evaluations(

    real_data="data/real.parquet",

    synthetic_data="data/synthetic.parquet",

    categorical_columns=["province", "segment"],

    integer_columns=["age"],

    float_columns=["income"],

    date_columns=["signup_date"],

    target="segment",

    output_dir="report",

    synthesizer_name="customers_v1",

)

For several tables, synthesizer_evaluations_multitable(configuration=\{table: \{...\}\}, output_dir="report") produces a single combined report.

Free-text columns and privacy

Free-text columns (text_columns) are modelled as categories: the generated texts are taken from the real training texts. If those columns contain personal data, anonymize them before training, for example with coding or text_regenerate (see Anonymization), or remove them from the data.

The exception is transaction descriptions in Transactional with transform_descriptions, which are rewritten when generating (see Transactional).

Starting from data prepared with Axon

If you pass train() an asset prepared with Axon, the synthesizer uses the column types Axon detected. What you declare in configuration takes precedence. See From Axon to Nucleus.

Axon: dataset quality and preparation

Axon checks and prepares a dataset before you anonymize or synthesize it: it infers column types, detects outliers and missing-value patterns, cleans the text, applies transformation rules and produces a quality report in PDF. It is included in every package and requires the axon permission.

Each step returns a new asset and leaves the original unchanged; the history accumulates.

Inspecting

from dedomena.axon import Inspector

asset = Inspector().run("data/customers.csv", name="customers")

asset.schema["types"]        # categorical_columns, integer_columns, float_columns, boolean_columns, text_columns, date_columns

asset.schema["semantic"]     # identifier, phone, email/URL, coordinate columns…

asset.schema["outliers"]     # \{column: [rows with outliers]\}

asset.info["rows"], asset.info["columns"]

asset.info["axon"]["unavailable"]    # \{\} if everything could run

The short form is inspect_dataset(source, **options).

Cleaning

from dedomena.axon import clean

cleaned = clean("data/customers.csv", name="customers")

cleaned.info["axon"]["cleaning"]     # duplicate rows, repeated headers, missing values, spaces fixed…

cleaned.quality["cleaning_after"]    # indicative score from 0 to 1

Cleaning:

• replaces tabs and line breaks with spaces, collapses repeated spaces and trims each cell;

• removes typographic or unbalanced quotes;

• converts to numbers the columns whose decimals are written with a comma ("1,5").

It also counts duplicate rows and headers repeated inside the data, but does not remove them. Remove them yourself if needed, for example with df.drop_duplicates().

After cleaning, the columns are left as text. To give them back their type, apply cast_type rules (next section) and inspect again.

Applying rules

from dedomena.axon import ApplyRules, Inspector

rules = \{

    "email":  \{"type": "normalize_nulls"\},

    "name":   \{"type": "normalize_text_casing", "params": \{"proposed_case": "title"\}\},

    "age":    [\{"type": "cast_type", "params": \{"target_type": "integer", "mode": "force"\}\},

               \{"type": "nullify_out_of_range", "params": \{"min": 0, "max": 120\}\}],

    "amount": \{"type": "cast_type", "params": \{"target_type": "float", "mode": "force"\}\},

    "city":   \{"type": "enforce_category", "params": \{"allowed": ["Madrid", "Barcelona", "Valencia"]\}\},

\}

prepared = ApplyRules(rules).run(cleaned, name="customers")

prepared.lineage[-1].details["applied"]    # rules applied per column

prepared.lineage[-1].details["failed"]     # rules that could not be applied (e.g. missing column)

prepared = Inspector().run(prepared, name="customers")   # final types

Each column takes one rule or a list of rules, applied in order:

For anonymization and synthesis to see a column's new type, use cast_type with "mode": "force".

Quality report in PDF

from dedomena.axon import report

path = report(prepared, "reports/customer_quality", lang="en", name="Customers 2026")

# → 'reports/customer_quality.pdf'

• lang: "en" or "es".

• The report is built from what the asset already knows: inspect it before generating the report.

• The report includes example values from each column. Treat it with the same confidentiality as the dataset. The same applies to meta.json when you save an inspected asset.

Risk and executive summary

These two functions need the language model: the Text module with a model installed.

from dedomena.detection import detect, llm

from dedomena.axon import risk, insight, report

llm.register()                                       # needed for insight()

prepared.sensitive_elements = detect(prepared)       # detect before computing the risk

with_risk = risk(prepared, name="customers")

summary = with_risk.info["risk"]["summary"]

summary["label"], summary["level"], summary["sensitivity_index"], summary["avg_risk_level_sensitive"]

with_insight = insight(with_risk, language="english", context="Online channel sign-ups", rows=200)

print(with_insight.info["insight"]["text"])

report(with_insight, "reports/customer_risk.pdf", lang="en")   # adds the risk and insights sections

• risk() computes the re-identification risk level from the detected sensitive columns: run detect() first; without detection, the result is "no risk".

• insight() writes an executive summary of what the data says, from the first rows rows. language accepts "spanish" or "english".

From Axon to Nucleus

The recommended workflow:

1. Inspect the original data with Inspector, to get a diagnosis.

2. Clean with clean.

3. Apply rules with ApplyRules, using cast_type with "mode": "force" to fix the types.

4. Inspect again with Inspector, so the schema describes the prepared data.

5. Anonymize or synthesize with Nucleus.

from dedomena.nucleus import anonymize, Generic

anonymized = anonymize(prepared, config="auto", output="output/customers_anon.csv")

model = Generic()

model.train(prepared)            # uses the types Axon detected

• Synthesis: uses Axon's column types for everything you have not declared in configuration.

• Anonymization: uses the cleaned content and, if the asset already has detected elements, uses them without detecting again.

• Always set output= when anonymizing an Axon asset (or output_format="DATAFRAME").

Checking the Axon engine

from dedomena.axon import engine_report

engine_report()      # \{'basic_cleaner': None, 'profiling': None, ...\}   None = works

If part of the engine cannot run on your machine (for example, because of a dependency), Inspector skips it, warns you and records it in asset.info["axon"]["unavailable"] with the reason. The rest of the analysis continues.

Models: automatic, registered and your own

How the library chooses

You do not need to configure anything: the library uses the best engine it finds installed for each modality.

• Patterns and validations are always applied, with any engine.

• When loaded, each engine writes a line to the log with what was chosen and the device (text detection engine selected: …, small-text device selected: cpu…).

• If an engine cannot be loaded, the library moves on to the next one and warns you. If what is missing are the models, the warning includes the command to download them:

  code

  
  text detection engine selected: patterns (...).
  
  Download the models with your licence: `dedomena-edge setup --size small`.
  
  

• Lightweight models run on CPU even when there is a GPU, so they do not take up card memory. You can move them to the GPU (see Registering a model explicitly).

Which engine will be used

from dedomena.detection import models

print(models.report())          # what will be used for each modality, and whether it fits on the GPU

GPU: 96 GB

residency: shared

runs on:

  audio: small-audio on cpu

  images: …

  text: …

• models.runtime_report() returns the same as a dictionary.

• models.plan() tells you, for the full models, whether they fit on your GPU: fits, tight (they fit with a reduced context) or no.

• None of this loads the models.

Per modality:

from dedomena.detection import llm, small, audio

llm.model_report()                     # (path, engine, problem): problem None = it can be loaded

small.report()                         # available lightweight models, and what is missing

audio.transcriber_report(device="auto")

Registering a model explicitly

Registering a model loads it into memory and connects it to detection and anonymization. The library does this on its own when it needs to, but registering it yourself lets you choose which capabilities to enable, on which device, and when to pay the loading time.

Language model (Text module):

from dedomena.detection import llm

llm.register()        # with the installed models

llm.unregister() hands text detection back to the patterns. llm.available() tells you whether it can be loaded.

> text_regenerate needs the model to be registered. Automatic detection loads the language model for detection only. To use text_regenerate, call llm.register() first.

Lightweight models:

from dedomena.detection import small

small.register(device="cuda")    # moves the lightweight text and image models to the GPU

small.upgrade()                  # swaps the lightweight models for the full ones, if installed

Full image model:

from dedomena.detection.images import vlm

vlm.register(confirm=True)       # also reviews each proposed region by cropping it

Using a model you already have

If your organization already has its own language model, Edge can use it for free-text detection, text_regenerate and Axon's executive summaries. You need the Text module.

Model requirements:

• A conversational language model (instruction-tuned), with a context of at least 8,192 tokens.

• Multilingual, if your data is not in English.

• Edge sends it its own instructions. Detection quality depends on the model: test it with your data before using it in production.

Option 1: your weights, on your disk

Give the path of the model:

export DEDOMENA_LLM_MODEL=/models/my-model

or when registering it:

from dedomena.detection import llm

llm.register(model="/models/my-model")

If the folder contains both a model folder and a .gguf file, the folder wins.

Option 2: your own model server

If you already serve a model on your internal network with an API compatible with OpenAI's chat API (/v1/chat/completions):

export DEDOMENA_LLM_ENDPOINT=http://models.internal:8000      # without /v1

export DEDOMENA_LLM_ENDPOINT_MODEL=model-name-on-your-server

from dedomena.detection import llm

llm.register()

• Edge calls \<DEDOMENA_LLM_ENDPOINT\>/v1/chat/completions.

• The server must be on your network: that way the data still does not leave your infrastructure.

• Edge does not send an authentication header. If your server requires one, expose it behind an internal proxy that adds it.

• With this variable set, Edge uses the server instead of a local model.

Your own image model

It works the same way with DEDOMENA_VISION_MODEL (a model folder) or with DEDOMENA_VISION_ENDPOINT and DEDOMENA_VISION_ENDPOINT_MODEL (a compatible server). Edge asks it to locate regions (faces, signatures, logos, fingerprints) and values within the image and to return their coordinates, so it must be a vision-language model capable of doing so. We recommend using Dedomena.AI's image model, unless you validate yours with your documents.

Forcing an engine

The library chooses on its own how to run the model, based on its format and your hardware. If you need to force a different way of running it (for example, because the GPU is busy with another process), ask Dedomena.AI support about the DEDOMENA_LLM_BACKEND and DEDOMENA_VISION_BACKEND variables.

Registering your own detector

If you have your own detector (for example, an internal model for a document type of your own), you can plug it in and every function in the library will use it instead of the built-in engine:

from dedomena.detection import register_detector, detect

def my_text_detector(text, types=None):

    """Receives a text and returns the elements it finds."""

    results = []

    for number in find_policy_numbers(text):               # your logic

        results.append(\{"value": number, "type": "health_card"\})

    return results

register_detector("text", my_text_detector)

detect("Policy AX-99812 is in the name of…")

• Patterns still run before your detector and win overlaps.

• A text value that does not appear verbatim in the text is discarded.

• If your detector fails, the library falls back to the patterns for that batch and warns you in the log.

• To remove it: from dedomena.detection.registry import unregister_detector; unregister_detector("text").

In the same way, register_generator("text_regenerate", function) lets you plug in your own text generator. The function receives (prompts, max_tokens=...) and returns one string per prompt.

Where the models are

The library looks for each model in this order:

1. The path you give when registering it (model=).

2. Its environment variable: DEDOMENA_LLM_MODEL, DEDOMENA_VISION_MODEL, DEDOMENA_AUDIO_MODEL, DEDOMENA_SMALL_TEXT_MODEL, DEDOMENA_SMALL_VISION_MODEL, DEDOMENA_SMALL_AUDIO_MODEL…

3. The models folder: DEDOMENA_MODELS_DIR, or ~/.local/share/dedomena/models if it is not set.

dedomena-edge setup downloads into the same folder where the library looks, so you do not need to configure anything. With the full download, env.sh sets DEDOMENA_MODELS_DIR to ./models.

Integrity. Each model carries a manifest with the size and fingerprint of each file. The fingerprint is checked on download, and the size every time the model is loaded: a half-copied model is not used.

GPU: memory and sharing

If your package has the full text and image models, both share the GPU:

• if they fit at the same time, both are loaded (shared);

• if not, they are loaded in turns (exclusive): each one gets the whole card and the other is unloaded when switching. Each switch costs a load, so order the work by modality (tables first, then images).

The library decides by measuring the card. You can force it with DEDOMENA_GPU_POLICY=shared|exclusive, and set each model's share of memory with DEDOMENA_LLM_VRAM and DEDOMENA_VISION_VRAM (for example, 0.45).

Reference manual

Commands

Environment variables

All of them are optional: without any, the library decides on its own.

Common errors