Skip to content

Allow outcome constraints on objective metrics in MOO (#5234)#5234

Closed
dme65 wants to merge 1 commit into
facebook:mainfrom
dme65:export-D109061377
Closed

Allow outcome constraints on objective metrics in MOO (#5234)#5234
dme65 wants to merge 1 commit into
facebook:mainfrom
dme65:export-D109061377

Conversation

@dme65

@dme65 dme65 commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary:

In multi-objective optimization, the acquisition function can exploit
asymmetric model uncertainty by proposing points that are extreme in
easy-to-predict objectives (e.g., very small model size) while hoping for
favorable outcomes on uncertain objectives (e.g., model accuracy). Users want to
add constraints against the optimization direction (e.g., model size >=
threshold while minimizing model size) to prevent exploring regions that
are known to be uninteresting.

Previously, Ax prevented any outcome constraint on an objective metric
via get_unconstrainable_metric_names(). This restriction made sense
for single-objective optimization (where constraining the sole objective
is reducible to a constrained problem) but was overly restrictive for
MOO. The downstream BoTorch acquisition functions (e.g.,
qLogNoisyExpectedHypervolumeImprovement) already support constraints
on objective outputs via their constraints parameter, which applies
a sigmoid-based feasibility indicator independently of the hypervolume
objective.

This diff relaxes the validation in
MultiObjectiveOptimizationConfig._validate_transformed_optimization_config()
to allow outcome constraints on objective metrics for multi-objective
problems, while preserving the restriction for single-objective and
scalarized-objective cases.

Note: objective thresholds are insufficient for this use case because
(1) they only serve as the hypervolume reference point (soft boundary,
not hard constraint), and (2) they only support bounding from the
direction of optimization -- you cannot set a lower bound on a
minimized objective via thresholds.

Reviewed By: saitcakmak

Differential Revision: D109061377

@meta-cla meta-cla Bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Jun 18, 2026
@meta-codesync

meta-codesync Bot commented Jun 18, 2026

Copy link
Copy Markdown

@dme65 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D109061377.

@codecov-commenter

codecov-commenter commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.56%. Comparing base (9bd6ec8) to head (35e8d51).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5234   +/-   ##
=======================================
  Coverage   96.56%   96.56%           
=======================================
  Files         619      619           
  Lines       70168    70177    +9     
=======================================
+ Hits        67756    67765    +9     
  Misses       2412     2412           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Summary:

In multi-objective optimization, the acquisition function can exploit
asymmetric model uncertainty by proposing points that are extreme in
easy-to-predict objectives (e.g., very small FLOPs) while hoping for
favorable outcomes on uncertain objectives (e.g., NE). Users want to
add constraints against the optimization direction (e.g., FLOPs >=
threshold while minimizing FLOPs) to prevent exploring regions that
are known to be uninteresting.

Previously, Ax prevented any outcome constraint on an objective metric
via `get_unconstrainable_metric_names()`. This restriction made sense
for single-objective optimization (where constraining the sole objective
is reducible to a constrained problem) but was overly restrictive for
MOO. The downstream BoTorch acquisition functions (e.g.,
`qLogNoisyExpectedHypervolumeImprovement`) already support constraints
on objective outputs via their `constraints` parameter, which applies
a sigmoid-based feasibility indicator independently of the hypervolume
objective.

This diff relaxes the validation in
`MultiObjectiveOptimizationConfig._validate_transformed_optimization_config()`
to allow outcome constraints on objective metrics for multi-objective
problems, while preserving the restriction for single-objective and
scalarized-objective cases.

Note: objective thresholds are insufficient for this use case because
(1) they only serve as the hypervolume reference point (soft boundary,
not hard constraint), and (2) they only support bounding from the
direction of optimization -- you cannot set a lower bound on a
minimized objective via thresholds.

Reviewed By: saitcakmak

Differential Revision: D109061377
@meta-codesync meta-codesync Bot changed the title Allow outcome constraints on objective metrics in MOO Allow outcome constraints on objective metrics in MOO (#5234) Jun 18, 2026
@dme65 dme65 force-pushed the export-D109061377 branch from 852e91b to 35e8d51 Compare June 18, 2026 22:30
@meta-codesync meta-codesync Bot closed this in 2724981 Jun 18, 2026
@meta-codesync meta-codesync Bot added the Merged label Jun 18, 2026
@meta-codesync

meta-codesync Bot commented Jun 18, 2026

Copy link
Copy Markdown

This pull request has been merged in 2724981.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. Merged meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants