On multivariate discrete least squares

Yeon Ju Lee, Charles A. Micchelli, Jungho Yoon

Abstract: For a positive integer n∈N<math><mi is="true">n</mi><mo is="true">∈</mo><mi mathvariant="double-struck" is="true">N</mi></math> we introduce the index set Nn:={1,2,…,n}<math><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">n</mi></mrow></msub><mo is="true">:</mo><mo is="true">=</mo><mrow is="true"><mo is="true">{</mo><mn is="true">1</mn><mo is="true">,</mo><mn is="true">2</mn><mo is="true">,</mo><mo is="true">…</mo><mo is="true">,</mo><mi is="true">n</mi><mo is="true">}</mo></mrow></math>. Let X:={xi:i∈Nn}<math><mi is="true">X</mi><mo is="true">:</mo><mo is="true">=</mo><mrow is="true"><mo is="true">{</mo><msub is="true"><mrow is="true"><mi is="true">x</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub><mo is="true">:</mo><mi is="true">i</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">n</mi></mrow></msub><mo is="true">}</mo></mrow></math> be a distinct set of vectors in Rd<math><msup is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">R</mi></mrow><mrow is="true"><mi is="true">d</mi></mrow></msup></math>, Y:={yi:i∈Nn}<math><mi is="true">Y</mi><mo is="true">:</mo><mo is="true">=</mo><mrow is="true"><mo is="true">{</mo><msub is="true"><mrow is="true"><mi is="true">y</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub><mo is="true">:</mo><mi is="true">i</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">n</mi></mrow></msub><mo is="true">}</mo></mrow></math> a prescribed data set of real numbers in R<math><mi mathvariant="double-struck" is="true">R</mi></math> and F:={fj:j∈Nm},m<n<math><mi mathvariant="script" is="true">F</mi><mo is="true">:</mo><mo is="true">=</mo><mrow is="true"><mo is="true">{</mo><msub is="true"><mrow is="true"><mi is="true">f</mi></mrow><mrow is="true"><mi is="true">j</mi></mrow></msub><mo is="true">:</mo><mi is="true">j</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">m</mi></mrow></msub><mo is="true">}</mo></mrow><mo is="true">,</mo><mi is="true">m</mi><mo is="true">&lt;</mo><mi is="true">n</mi></math>, a given set of real valued continuous functions defined on some neighborhood O<math><mi mathvariant="script" is="true">O</mi></math> of Rd<math><msup is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">R</mi></mrow><mrow is="true"><mi is="true">d</mi></mrow></msup></math> containing X<math><mi is="true">X</mi></math>. The discrete least squares problem determines a (generally unique) function f=∑j∈Nmcj⋆fj∈spanF<math><mi is="true">f</mi><mo is="true">=</mo><msub is="true"><mrow is="true"><mo is="true">∑</mo></mrow><mrow is="true"><mi is="true">j</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">m</mi></mrow></msub></mrow></msub><msubsup is="true"><mrow is="true"><mi is="true">c</mi></mrow><mrow is="true"><mi is="true">j</mi></mrow><mrow is="true"><mo is="true">⋆</mo></mrow></msubsup><msub is="true"><mrow is="true"><mi is="true">f</mi></mrow><mrow is="true"><mi is="true">j</mi></mrow></msub><mo is="true">∈</mo><mtext is="true">span</mtext><mi mathvariant="script" is="true">F</mi></math> which minimizes the square of the ℓ2−<math><msup is="true"><mrow is="true"><mi is="true">ℓ</mi></mrow><mrow is="true"><mn is="true">2</mn></mrow></msup><mo is="true">−</mo></math>norm ∑i∈Nn(∑j∈Nmcjfj(xi)−yi)2<math><munder is="true"><mrow is="true"><mo is="true">∑</mo></mrow><mrow is="true"><mi is="true">i</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">n</mi></mrow></msub></mrow></munder><msup is="true"><mrow is="true"><mrow is="true"><mo is="true">(</mo><munder is="true"><mrow is="true"><mo is="true">∑</mo></mrow><mrow is="true"><mi is="true">j</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">m</mi></mrow></msub></mrow></munder><msub is="true"><mrow is="true"><mi is="true">c</mi></mrow><mrow is="true"><mi is="true">j</mi></mrow></msub><msub is="true"><mrow is="true"><mi is="true">f</mi></mrow><mrow is="true"><mi is="true">j</mi></mrow></msub><mrow is="true"><mo is="true">(</mo><msub is="true"><mrow is="true"><mi is="true">x</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub><mo is="true">)</mo></mrow><mo is="true">−</mo><msub is="true"><mrow is="true"><mi is="true">y</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub><mo is="true">)</mo></mrow></mrow><mrow is="true"><mn is="true">2</mn></mrow></msup></math> over all vectors (cj:j∈Nm)∈Rm<math><mrow is="true"><mo is="true">(</mo><msub is="true"><mrow is="true"><mi is="true">c</mi></mrow><mrow is="true"><mi is="true">j</mi></mrow></msub><mo is="true">:</mo><mi is="true">j</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">m</mi></mrow></msub><mo is="true">)</mo></mrow><mo is="true">∈</mo><msup is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">R</mi></mrow><mrow is="true"><mi is="true">m</mi></mrow></msup></math>. The value of f<math><mi is="true">f</mi></math> at some s∈O<math><mi is="true">s</mi><mo is="true">∈</mo><mi mathvariant="script" is="true">O</mi></math> may be viewed as the optimally predicted value (in the ℓ2−<math><msup is="true"><mrow is="true"><mi is="true">ℓ</mi></mrow><mrow is="true"><mn is="true">2</mn></mrow></msup><mo is="true">−</mo></math>sense) of all functions in spanF<math><mtext is="true">span</mtext><mi mathvariant="script" is="true">F</mi></math> from the given data X={xi:i∈Nn}<math><mi is="true">X</mi><mo is="true">=</mo><mrow is="true"><mo is="true">{</mo><msub is="true"><mrow is="true"><mi is="true">x</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub><mo is="true">:</mo><mi is="true">i</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">n</mi></mrow></msub><mo is="true">}</mo></mrow></math> and Y={yi:i∈Nn}<math><mi is="true">Y</mi><mo is="true">=</mo><mrow is="true"><mo is="true">{</mo><msub is="true"><mrow is="true"><mi is="true">y</mi></mrow><mrow is="true"><mi is="true">i</mi></mrow></msub><mo is="true">:</mo><mi is="true">i</mi><mo is="true">∈</mo><msub is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">N</mi></mrow><mrow is="true"><mi is="true">n</mi></mrow></msub><mo is="true">}</mo></mrow></math>.We ask “What happens if the components of X<math><mi is="true">X</mi></math> and s<math><mi is="true">s</mi></math> are nearly the same”. For example, when all these vectors are near the origin in Rd<math><msup is="true"><mrow is="true"><mi mathvariant="double-struck" is="true">R</mi></mrow><mrow is="true"><mi is="true">d</mi></mrow></msup></math>. From a practical point of view this problem comes up in image analysis when we wish to obtain a new pixel value from nearby available pixel values as was done in [2], for a specified set of functions F<math><mi mathvariant="script" is="true">F</mi></math>.This problem was satisfactorily solved in the univariate case in Section 6 of Lee and Micchelli (2013). Here, we treat the significantly more difficult multivariate case using an approach recently provided in Yeon Ju Lee, Charles A. Micchelli and Jungho Yoon (2015).