What has been measured, and how to test it yourself.

Measured to date: 18 workloads on one AMD GPU, plus NVIDIA runs on three clouds.

This brief is for infrastructure and performance engineers who want to know what Chameleon does, what has been measured, and how to test it on their own workloads before changing anything in production. The Phase 1 pilots were not benchmarks. Their purpose was to show, independently and across different architectures, what Composite Job Designs make possible. In plain terms:

• The results agree across independent environments. A university team (DEHub, on-premises AMD integrated GPU), runs recorded by PREDICTif Solutions on AWS, and our own runs on Oracle Cloud and Google Cloud (NVIDIA) all show the same pattern: large speedups and large energy reductions against the first run of the same job, in overlapping ranges. The DEHub and AWS figures are not our numbers, and they track with the OCI and GCP figures. What carries the conclusion is that consistency across different architectures, not any single run or report.
• Measured by a university team. Rowan University’s Digital Engineering Hub (DEHub) ran 18 workloads on one integrated AMD GPU. We supplied the executable and had no access to the system. The resulting report is a draft and has not been signed. We proposed the Phase 1 scope, DEHub management agreed to it, and we delivered what was agreed. We said at the outset that two milestones, Buffer Feeds and Synergy, were unfunded at the time and outside that scope. Management then requested a Phase 2 proposal, reviewed it and accepted it, pending completion of those two milestones.
• Run on cloud NVIDIA GPUs. The AWS runs were independently recorded by PREDICTif Solutions (since acquired by nClouds), in an AWS-funded project. Amazon has approved funding for the current AWS phase (AWS Phase 2), which is active. We also report runs on Oracle Cloud and Google Cloud instances (T4, A10G, A10, A100). The Oracle and Google runs are our own results.
• Evaluation scope. The evaluation binaries are built for evaluation. Single-GPU runs are the measured scope; multi-GPU operation and long-running use are next steps. Buffer Feeds and Synergy were unfunded during Phase 1, so they are not part of what was measured.

The two milestones

Two milestones remain, and Phase 2 is pending their completion. Neither was funded during Phase 1, and neither is part of what was measured.

• Buffer Feeds (Milestone 01, Distribution). Distributes work across all components within a single node, including multi-GPU configurations such as the NVIDIA A100 and B100, with no hypervisor dependency at the node level. It enables single-node, multi-GPU Composite Job Designs. It does not by itself run work across multiple nodes: cross-node distribution is a separate networking step that follows. Every measurement in this brief is single GPU, so none of it is attributable to Buffer Feeds.
• Synergy Expansion (Milestone 02, Intelligence). Expands Essence’s capacity to understand, create, edit and materialize Aptivs across the full range of what computing can do, assimilating software’s historical workarounds as governed, direct capability. It includes natural-language creation of workloads, with generative AI proposals governed by Meaning Coordinates.

What is available today

The sections that follow set out what was measured and under what conditions, then propose a one-month evaluation that your own team runs and controls.

From intent to SPIR-V, executed through Vulkan.

Chameleon takes a statement of intent, expresses it as a Composite Job Design (CJD), and uses Morpheus® to generate SPIR-V instructions for the target GPU at runtime. It is not a compiler pass, a fixed kernel library, a prompt-to-code assistant or a profiler.

• Inputs. The current pilot build runs a fixed set of problem sets. Our pilot page describes the input forms the platform is designed to take (natural language, existing GLSL, kernels, or a workload harness with a stable execution boundary) and states no CUDA, ROCm, oneAPI or PyTorch dependency.
• Composite Job Design. A CJD describes a set of valid, semantically equivalent execution designs for one intent, bounded by declared constraints such as power or latency. Per the DEHub report, any adaptation selects among pre-validated paths; intent and constraints do not change at runtime.
• Two execution modes. Fixed: a single design is selected and used unchanged. Bounded adaptive: several realizations are evaluated under controlled conditions and converge to a stable design. Chameleon generates the SPIR-V at runtime in both modes. Exporting SPIR-V as a file is not available yet.
• What it restructures. Kernel boundaries, instruction ordering, parallelism granularity, memory access patterns, scheduling and synchronization.
• Footprint in the DEHub evaluation. One Linux executable of about 14 MB. No agents, background services or SDKs. Ubuntu 24.04, kernel 6.11 or newer (needed for that AMD platform), the open AMDGPU driver, Vulkan 1.3 or newer.
• Hardware path. SPIR-V through Vulkan targets NVIDIA, AMD and Intel GPUs. NVIDIA T4, A10G, A10, A100 and H100 runs are reported on cloud instances (H100 on Google Cloud only); the DEHub evaluation covers only an AMD Radeon 8060S.

Why SPIR-V and not CUDA, ROCm or oneAPI?

CUDA, ROCm and oneAPI each lead to one vendor's hardware, so work written for one has to be ported and tuned again for another. SPIR-V is the cross-vendor format that NVIDIA, AMD and Intel drivers accept, and each driver lowers it to the native instructions of its own chip. Because Chameleon generates the instructions instead of asking an engineer to write them, it can target that shared layer and tune the output for each machine. This path does not replace vendor libraries, and DirectX 12 and Metal use their own intermediate formats. The Why SPIR-V page explains the stack for engineers who work in CUDA, with diagrams.

Tuning that is usually done by hand

Chameleon uses Morpheus® to generate the machine instructions from the CJD. Performance engineers usually tune the tasks below by hand, workload by workload, and the result holds for one workload on one chip at one moment. When the hardware or the conditions change, the work starts again. Here the same tasks are generated for the hardware present.

Output carried as a bitstream

Output from a CJD is carried in the Wantverse [.wv] format, which the technical docs describe as a stream and not a file in the conventional sense. The bitstream is designed to be laid out contiguously so memory does not fragment, to be headerless so the payload is the whole stream, and to be encrypted per stream by StreamWeave®, which shifts algorithm combinations continuously. Jobs are also woven together along the stream with no fixed start or end point, which adds a form of obfuscation. The job details needed for processing are carried by Meaning Coordinates inside the Aptivs, not by a header. File-based output for pipelines that need a file is not running today. It is re-established during the first 30 days of funded milestone work. The docs list StreamWeave per-stream encryption as dependent on the Synergy milestone.

Essence adapts. A benchmark assumes the opposite.

A standard benchmark assumes a fixed tool and a fixed hardware environment: a defined program, run on a named machine, scored against another program on the same machine. That design suits the first two eras of software. It does not suit a system whose purpose is to adapt.

What this means for reading the results

• Demonstration points, not scores. The DEHub, AWS, Google Cloud and OCI figures show Essence adapting on specific workloads and specific hardware. They are not benchmark scores, and no fixed-workload ranking is claimed.
• Same intent, different execution, by design. On other hardware, or with other declared priorities, Essence resolves the same intent differently. Two runs on two machines compare two resolutions, not one program twice.
• The first-run baseline follows from this. The speedup compares Essence’s first resolution of a job (Composite Job Design 1) with its adapted result, because there is no fixed program to hold constant.
• A fixed comparison is still a useful reference. Buyers need one, so the evaluation below includes a comparison against your own frozen build once your own workloads can run. It measures one slice. The questions that fit better are whether the outcome meets the declared priority on the hardware present, and whether it holds when the hardware or conditions change, without rewriting.

Why Phase 2 uses the evaluator’s own workloads

Because a benchmark scores a fixed program, a benchmark suite cannot show what a Composite Job Design does. DEHub’s Phase 2 therefore takes a different form. It is not a request for a score, and it does not signal doubt about CJDs. DEHub wants to run its own workloads as CJDs, so that the result answers the question the institution cares about: what happens on its own work, on its own hardware, against success criteria it fixes in advance. We expect the same to hold for other evaluators. A fixed comparison against a frozen build becomes possible once your own workloads can run. Until the milestones are complete, the current build runs a fixed set of problem sets. The CJD run on your own workload is the measurement that matters.

Seventeen of 18 workloads ran 22× to 114× faster than their first run. One ran 14× faster.

Read these as demonstration points rather than benchmark scores (see the previous section). The table lists each result with its source and hardware. In every case, speedup is measured against the first run of the same job.

Results by cloud and GPU

Each row summarizes the tests in our pilot log (revision 11) for one GPU on one cloud. A test is one workload at one resolution. Speedup is the best of runs 2 to 6 against run 1 of the same job, shown as a range across tests with the median in brackets. Resolution is 3840 × 2160 unless noted; some AWS T4 tests use 1920 × 1080, and the H100 tests span several resolutions.

• H100 results are from Google Cloud only. They span four resolutions, 3840 × 2160 to 7680 × 4320, over four workloads. Red Nebula holds 30.7× to 31.4× at all four resolutions. Firework Pop holds 32.3× to 32.4× through 6016 × 3384 and falls to 15.6× at 7680 × 4320, with energy reduction of 77%. Smoke_Riser ranges from 31.4× down to 14.0× (energy reduction 91% at the low end), and the SDF workload from 34.0× to 18.4×.
• No speedup is reported at 15360 × 8640. The four H100 tests at that resolution record the first run only.
• B100. No B100 run is recorded. The H100 setup carries over only in part: NVIDIA documents that Blackwell GPUs require the open GPU kernel modules, and the driver branch, the Vulkan driver package and the Vulkan version exposed must be confirmed on the B100 nodes against a working H100 node. A B100 figure itself would come from the evaluation. An A5000 and an AWS-hosted H100 are also not yet recorded.
• Not tabulated. Two single Google Cloud tests (an A10 and an A100 with no variant stated), because one test does not make a range.
• What the log measures. Time, power and energy per run. It has no output-comparison column. Where the log notes mention fidelity or quality, that is a description, not a measurement.

The rows agree with each other

The DEHub rows (14× to 114× speedup, 97.8% to 99.7% energy reduction), the cloud rows (3.9× to 53.6× speedup with a median near 30× on every platform, and 75.3% to 99.6% energy reduction) and the earlier CPU result all show the same direction, with overlapping magnitudes. They come from different people and different architectures (AMD integrated, and NVIDIA T4, A10G, A10, A100 and H100 cloud GPUs), and the DEHub and AWS measurements were not made by us. The purpose of the pilots was never a score. It was to show, across architectures, that the same Composite Job Design approach engineers a solution for each. Read together, the rows show that. No single report has to carry the conclusion.

Our bandwidth (WarpSpeed) and video (illumin8) results concern signal processing, not GPU execution, and are outside this brief. Our illumin8 technology forms a unified pipeline for signal processing, which enables adaptive quality, without loss. It derives detail mathematically from the signal, based on Meaning Coordinates, and the original can be reconstructed when required. illumin8 is not running as a build today. It returns when the milestones are complete, so it is outside the first evaluation. The pilot log behind this section records speed, power and energy only, with no output-comparison column. A quality measure for your workloads would therefore be agreed with you in advance and run once illumin8 is deployed again, and a reconstruction check (rebuild the original from the output and compare it with the input) can be included then.

Check run on the Phase 1 table

Each energy figure equals 1 minus (optimized power × optimized time) over (baseline power × baseline time), for all 18 rows. The energy column is therefore calculated from the recorded time and power readings, not read from a separate energy meter.

Where the measurements apply, and where your evaluation extends them.

Each item below defines the conditions the results were measured under. Your evaluation extends them to your own system.

• Workload type. The DEHub workloads are shader-style kernels run at 1920 × 1080, chosen as representative of data-pipeline patterns. The report relates them to simulation, imaging, robotics and AI tensor work. AI inference and training workloads are not yet part of the measured set.
• Baseline. The baseline is the first run of the same job. In our pilot log, run 1 of each test is Composite Job Design 1, described as the default implementation with the fewest permutations, and runs 2 to 6 are successive refinements. A speedup therefore measures how far Essence’s own refinement moves from its first output. Comparison with your existing code, a hand-tuned shader or a vendor compiler path is what the client-run evaluation below adds. The DEHub evaluation uses the same first-run baseline.
• Hardware. The headline 114× comes from a 40-compute-unit integrated GPU with shared memory. Discrete datacenter GPUs were measured on the cloud NVIDIA runs. No B100, A5000 or AWS-hosted H100 run is recorded yet. The H100 setup carries over to B100 only in part, because Blackwell needs the open kernel modules and a driver, Vulkan package and Vulkan version that are confirmed at scoping.
• Repeatability. The report states results were stable across runs. Our pilot log records one pass per test, up to six successive variants. The evaluation below specifies repeated runs with spread reported.
• Independence. The tests were run by DEHub personnel on their own system, and we had no access to it. We proposed the Phase 1 scope and DEHub management agreed to it. The Phase 2 proposal was prepared at management’s request, then reviewed and accepted by management, pending the two milestones. The results do not depend on that report alone: the AWS runs were recorded by a third party, and the OCI and GCP runs fall in the same range.
• Edge and physical AI. Edge computing and physical AI, such as robots and vehicles, run on many small machines with tight limits, and the company sees both as massive opportunities for businesses. Results on edge hardware have not yet been measured.
• Scope. Single GPU. Multi-GPU and cluster operation follow later milestones. Our guarantee is not a particular number. It is real time optimization of data ordering, scheduling, memory, networking and hardware parallelization for the workload and hardware at hand, which is what highly skilled engineers do by hand and cannot do in real time. Composite Job Designs are how it is done. Results depend on memory bandwidth, compute-to-memory balance, control-flow irregularity and tensor topology.
• Cloud runs. The AWS runs were recorded independently by PREDICTif Solutions, in a project funded by Amazon. The Google Cloud (including all H100) and OCI runs are our own results.

Adoption is staged, and every stage after the first evaluation depends on the funded milestones.

The documented path expands only where a workload justifies it. Teams can adopt per workload, per product or per team.

1. Export Mode. The design lets Essence hand standard artifacts to an existing pipeline unchanged: Git, Jenkins, GitHub Actions, GitLab CI, Azure DevOps, SAST and DAST scanners, artifact registries and current deploy targets. Essence is not a code generator, and these artifacts are an interface to existing tooling, not its purpose. File-based output is not running today. It is re-established during the first 30 days of funded milestone work.
2. Hybrid. Fixed, scan-ready builds are kept for compliance, while controlled runtime behavior is introduced for chosen workloads. Policy gates decide which workloads are fixed and which are adaptive.
3. Essence Runtime Mode. Execution happens dynamically within Essence, with policy, traceability and audit controls. Fixed builds can still be exported when certification or offline deployment requires them, once file-based output is re-established. The OS-hosted runtime returns when the milestones are complete.

These phases describe the platform’s design. Today only the GPU optimization build runs, and the measurements in this brief come from it on a single GPU, so the evaluation below comes before any rollout planning.

One month, your hardware, your criteria. Your own workloads follow the milestones.

Your team operates the evaluation and sets the success criteria in advance. The current pilot build runs a fixed set of problem sets, so an evaluation today is run on those, on your hardware. Running your own workloads and comparing them with your frozen production build becomes possible once the two milestones are complete. Both stages follow the same method. The structure follows the Phase 2 proposal prepared at DEHub management’s request (about one month, single GPU, performance and energy analysis). Run counts and tolerances are proposals for your engineers to adjust. If your question is whether efficiency can be gained without regression on a live system, the form that answers it is a live side-by-side (shadow) evaluation of your full system, described below. It depends on the milestones, so it cannot be run on the current build.

1. Scope (week 0). You supply the GPU model, the driver version and kernel module type, the Vulkan version reported by vulkaninfo, the OS and version, and the objective (performance, energy, throughput or latency). Today the workloads are our fixed problem sets. After the milestones, you add one to three of your own.
2. Confirm what can run. We state in writing which problem sets the current pilot build runs. The build is not modifiable for individual evaluations, so an existing CUDA kernel or PyTorch workload is not accepted as submitted. Changes to the build depend on funding to complete the two milestones.
3. Baseline. Today the baseline is the first run of the same job, the default design. It shows how far adaptation moves on your hardware. A comparison with your production build is not part of the current build’s scope. Once your own workloads can run, you freeze and record your current production build, compiler, flags and driver before any Chameleon run, and both sides run identical inputs.
4. Metrics. Wall time per unit of work, throughput and GPU utilization. Power and energy come from your own telemetry (vendor tools or an external meter) and are reported as measured, not calculated from time and nominal power. Record the time and energy of every refinement run as well as the final one, so that the cost of getting to the final design counts against the final figure.
5. Priority shift. Run the same intent with the declared objective changed (for example from time to energy) and record how the execution and the metrics change. Confirmed at scoping.
6. Hardware change. If your fleet mixes GPU models, run the same package unmodified on each model and report the metrics per model as well as in aggregate. Confirmed at scoping.
7. Runs. Fixed warm-up, then repeated runs. Proposal: at least 30 runs per configuration, with baseline and Chameleon interleaved once a production baseline is in play. Report the median and spread and keep every raw value.
8. Correctness. Outputs are compared with the reference within tolerances you set before testing. The tolerance can be per output or in aggregate across the run (for example, an equivalence threshold of 99.9% judged over the whole workload), and you choose the metric. Our pilot log contains no output-comparison measurement, so this evaluation is where output equivalence is first measured. A tolerance failure counts against the result.
9. Live side-by-side (after the milestones). Your production system keeps serving unchanged. The same live inputs also run through the Chameleon path in shadow, and its outputs are not served. Outputs of the two paths are compared in aggregate against your tolerance, with time, throughput and energy recorded on both. Results are reported per GPU model and overall. Because the full system is the unit under test, this needs your own workloads to run, which depends on the two milestones. No manual-coding bridge is offered to get there sooner.
10. Isolation. You operate the system. We provide the executable and guidance only, with no access to the environment or data, as in the DEHub evaluation.
11. Pre-set criteria. You write the success threshold and stop conditions before the first run.
12. Deliverables. You keep the raw CSV, the configuration record and a short written report. Sharing any of it with us is by agreement.

Suggested rhythm: setup in week 1, runs in weeks 2 and 3, analysis in week 4. Multi-GPU operation (Buffer Feeds) and natural-language creation of workloads (Synergy) depend on two milestones that are not complete, so they are outside this first evaluation. So are the .wv stream and StreamWeave encryption, which are not part of a single-GPU kernel test and depend on an incomplete milestone.

Current answers, and what the evaluation settles.