Stochastic Direct Search Methods for Blind Resource Allocation
Abstract: Motivated by programmatic advertising optimization, we consider the task of sequentially allocating budget across a set of resources. At every time step, a feasible allocation is chosen and only a corresponding random return is observed. The goal is to maximize the cumulative expected sum of returns. This is a realistic model for budget allocation across subdivisions of marketing campaigns, with the objective of maximizing the number of conversions. We study direct search (also known as pattern search) methods for linearly constrained and derivative-free optimization in the presence of noise, which apply in particular to sequential budget allocation. These algorithms, which do not rely on hierarchical partitioning of the resource space, are easy to implement; they respect the operational constraints of resource allocation by avoiding evaluation outside of the feasible domain; and, they are also compatible with warm start by being (approximate) descent algorithms. However, they have not yet been analyzed from the perspective of cumulative regret. We show that direct search methods achieves finite regret in the deterministic and unconstrained case. In the presence of evaluation noise and linear constraints, we propose a simple extension of direct search that achieves a regret upper-bound of the order of $T^{2/3}$. We also propose an accelerated version of the algorithm, relying on repeated sequential testing, that significantly improves the practical behavior of the approach.
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=TxNjmAnunA
Changes Since Last Submission: We revised the manuscript in order to take the comments of the reviewers into account and to higlight the novelty and practical value of the proposed algorithms.
The main modifications with respect to the first submission are the following:
- The structure of the introduction has been modified to make it clearer that the problem we tackle extends beyond blind resource allocation, as advised by reviewer GNrJ.
- Further details have been added in Section 2 on the comparison to UCB on a discretization of the feasible set, as advised by reviewer 7z8h.
- Assumption 4 in Section 3.2 has been removed as it is a consequence of Assumption 1 and 3, as noted by reviewer 7z8h.
- Some details of the implementation of algorithms in Section 2 have been moved to Appendix E, in order to respect length constraints.
- A remark has been added on the necessity of the assumption that the optimal allocation resides in the interior of the feasible sets, to answer a question raised by 7z8h. Further simulations supporting our claim have been added to Appendix F
- Typos found by reviewers CKgM and 7z8h have been corrected.
- The introduction has been remodeled in order to make the difference with cumulative constraints clearer.
- The section explaining the contributions was been rewritten in order to highlight the novelty of our algorithm, and of the analysis.
Code: https://github.com/juliette-achddou/blind-resource-allocation
Assigned Action Editor: ~Lijun_Zhang1
Submission Number: 1821
Loading