Jump to content

Theory of functional connections

From Wikipedia, the free encyclopedia

The Theory of Functional Connections (TFC) is a mathematical framework designed for functional interpolation. It introduces a method to derive a functional— a function that operates on another function—capable of transforming constrained optimization problems into equivalent unconstrained problems. This transformation enables the application of TFC to various mathematical challenges, including the solution of differential equations. Functional interpolation, in this context, refers to constructing functionals that always satisfy given constraints, regardless of the expression of the internal (free) function.

From interpolation to functional interpolation

[edit]

To provide a general context for the TFC, consider a generic interpolation problem involving constraints, such as a differential equation subject to a boundary value problem (BVP). Regardless of the differential equation, these constraints may be consistent or inconsistent. For instance, in a problem over the domain , the constraints and are inconsistent, as they yield different values at the shared point . If the constraints are consistent, a function interpolating these constraints can be constructed by selecting linearly independent support functions, such as monomials, . The chosen set of support functions may or may not be consistent with the given constraints. For instance, the constraints and are inconsistent with the support functions, , as can be easily verified. If the support functions are consistent with the constraints, the interpolation problem can be solved, yielding an interpolant--a function that satisfies all constraints. Choosing a different set of support functions would result in a different interpolant. When an interpolation problem is solved and an initial interpolant is determined, all possible interpolants can, in principle, be generated by performing the interpolation process with every distinct set of linearly independent support functions consistent with the constraints. However, this method is impractical, as the number of possible sets of support functions is infinite.

This challenge was addressed through the development of the TFC, an analytical framework for performing functional interpolation introduced by Daniele Mortari at Texas A&M University.[1] The approach involves constructing a functional that satisfies the given constraints for any arbitrary expression of , referred to as the free function. This functional, known as the constrained functional, provides a complete representation of all possible interpolants. By varying , it is possible to generate the entire set of interpolants, including those that are discontinuous or partially defined.

Function and functional interpolation flowchart

Function interpolation produces a single interpolating function, while functional interpolation generates a family of interpolating functions represented through a functional. This functional defines the subspace of functions that inherently satisfy the given constraints, effectively reducing the solution space to the region where solutions to the constrained optimization problem are located. By employing these functionals, constrained optimization problems can be reformulated as unconstrained problems. This reformulation allows for simpler and more efficient solution methods, often improving accuracy, robustness, and reliability. Within this context, the Theory of Functional Connections (TFC) provides a systematic framework for transforming constrained problems into unconstrained ones, thereby streamlining the solution process.

TFC addresses univariate constraints involving points, derivatives, integrals, and any linear combination of these.[2] The theory is also extended to accommodate infinite and multivariate constraints and applied to solving ordinary, partial, and integro-differential equations. The consistency problem, which pertains to constraints, interpolation, and functional interpolation, is comprehensively addressed in.[3] This includes the consistency challenges associated with boundary conditions that involve shear and mixed derivatives.

The univariate version of TFC can be expressed in one of the following two forms:

where represents the number of linear constraints, is the free function, and are user-defined, linearly independent support functions. The terms are the coefficient functionals, are switching functions (which take a value of 1 when evaluated at their respective constraint and 0 at other constraints), and are projection functionals that express the constraints in terms of the free function.

A rational example

[edit]

To show how TFC generalizes interpolation, consider the constraints, and . An interpolating function satisfying these constraints is,

as can be easily verified. Because of this interpolation property, the derivative of the function,

vanishes at and , for \textit{any} function, . Therefore, by adding to , a functional is obtained that still satisfies the constraints,

no matter what is. Due to this property, this functional is referred to as constrained functional. The key requirement for the functional to work as intended is that the terms and are defined. Once this condition is met, the functional is free to take on any arbitrary values beyond the specified constraints, thanks to the infinite flexibility provided by . Importantly, this flexibility is not limited to the specific constraints chosen in this example. Instead, it applies universally to any set of constraints. This universality illustrates how TFC performs functional interpolation: it constructs a function that satisfies the given constraints while simultaneously allowing complete freedom in behavior elsewhere through the choice of . In essence, this example demonstrates that the constrained functional captures all possible functions that meet the given constraints, showcasing the power and generality of TFC in handling a wide variety of interpolation problems.

Example: A univariate constrained functional animation using 2 absolute constraints and one relative constraint.
Example: A univariate constrained functional animation using 2 absolute constraints and one relative constraint.

Applications of TFC

[edit]

TFC has been extended and employed in various applications, including its use in shear-type and mixed derivative problems, the analysis of fractional operators,[4] the determination of geodesics for BVP in curved spaces,[5] and in continuation methods.[6][7] Additionally, TFC has been applied to indirect optimal control,[8][9] the modeling of stiff chemical kinetics,[10] and the study of epidemiological dynamics.[11] TFC extends into astrodynamics[1], where Lambert's problem is efficiently solved.[12] It has also demonstrated potential in nonlinear programming[13] and structural mechanics[14][15] and radiative transfer,[16] among other areas. An efficient, free Python TFC toolbox is available at https://github.com/leakec/tfc.

Of particular note is the application of TFC in neural networks, where it has shown exceptional efficiency,[17][18] especially addressing high-dimensional problems and in enhancing the performance of physics-informed neural networks[19] by effectively eliminating constraints from the optimization process, a challenge that traditional neural networks often struggle to address. This capability significantly improves computational efficiency and accuracy, enabling the resolution of complex problems with greater ease. TFC has been employed with physics-informed neural networks and symbolic regression techniques[20] for physics discovery of dynamical systems.[21][22]

Difference with spectral methods

[edit]

At first glance, TFC and spectral methods may appear similar in their approach to solving constrained optimization problems. However, there are two fundamental distinctions between them:

  • Representation of solutions: Spectral methods represent the solution as a sum of basis functions, whereas TFC represents the free function as a sum of basis functions. This distinction allows TFC to analytically satisfy the constraints, while spectral methods treat constraints as additional data, approximating them with an accuracy dependent on the residuals.
  • Computational approach in BVP: In linear BVPs, the computational strategies of the two methods differ significantly. Spectral methods typically employ iterative techniques, such as the shooting method, to reformulate the BVP as an initial value problem, which is simpler to solve. Conversely, TFC directly addresses these problems through linear least-squares techniques, avoiding the need for iterative procedures.

Both methods can perform optimization using either the Galerkin method, which ensures the residual vector is orthogonal to the chosen basis functions, or the Collocation method, which minimizes the norm of the residual vector.

Difference with Lagrange multipliers technique

[edit]

The Lagrange multipliers method is a widely used approach for imposing constraints in an optimization problem. This technique introduces additional variables, known as multipliers, which must be computed to enforce the constraints. While the computation of these multipliers is straightforward in some cases, it can be challenging or even practically infeasible in others, thereby adding significant complexity to the problem. In contrast, TFC doesn't add new variables and enables the derivation of constrained functionals without encountering insurmountable difficulties. However, it is important to note that the Lagrange multiplier method has the advantage of handling inequality constraints, a capability that TFC currently lacks.

A notable limitation of both approaches is their propensity to produce solutions that correspond to local optima rather than guaranteed global optima, particularly in the context of non-convex problems. Consequently, supplementary verification procedures or alternative methods may be required to assess and confirm the quality and global validity of the obtained solution. In summary, while TFC does not entirely replace the Lagrange multipliers method, it serves as a powerful alternative in cases where the computation of multipliers becomes excessively complex or infeasible, provided the constraints are limited to equalities.

References

[edit]
  1. ^ Mortari, Daniele (December 2017). "The Theory of Connections: Connecting Points". Mathematics. 5 (4): 57. arXiv:1702.06862. doi:10.3390/math5040057. ISSN 2227-7390.
  2. ^ De Florio, Mario; Schiassi, Enrico; D’Ambrosio, Andrea; Mortari, Daniele; Furfaro, Roberto (September 2021). "Theory of Functional Connections Applied to Linear ODEs Subject to Integral Constraints and Linear Ordinary Integro-Differential Equations". Mathematical and Computational Applications. 26 (3): 65. doi:10.3390/mca26030065. hdl:11573/1568297. ISSN 2297-8747.
  3. ^ Mortari, Daniele (January 2022). "Theory of Functional Connections Subject to Shear-Type and Mixed Derivatives". Mathematics. 10 (24): 4692. doi:10.3390/math10244692. ISSN 2227-7390.
  4. ^ Mortari, Daniele; Garrappa, Roberto; Nicolò, Luigi (January 2023). "Theory of Functional Connections Extended to Fractional Operators". Mathematics. 11 (7): 1721. doi:10.3390/math11071721. ISSN 2227-7390.
  5. ^ Mortari, Daniele (August 2022). "Using the Theory of Functional Connections to Solve Boundary Value Geodesic Problems". Mathematical and Computational Applications. 27 (4): 64. doi:10.3390/mca27040064. ISSN 2297-8747.
  6. ^ Wang, Yang; Topputo, Francesco (1 February 2022). "A TFC-based homotopy continuation algorithm with application to dynamics and control problems". Journal of Computational and Applied Mathematics. 401: 113777. arXiv:1911.04899. doi:10.1016/j.cam.2021.113777. hdl:11311/1183129. ISSN 0377-0427.
  7. ^ Campana, Claudio Toquinho; Merisio, Gianmario; Topputo, Francesco (June 2024). "Low-energy Earth–Moon transfers via Theory of Functional Connections and homotopy". Celestial Mechanics and Dynamical Astronomy. 136 (3): 21. Bibcode:2024CeMDA.136...21C. doi:10.1007/s10569-024-10192-5. ISSN 0923-2958.
  8. ^ D’Ambrosio, Andrea; Schiassi, Enrico; Johnston, Hunter; Curti, Fabio; Mortari, Daniele; Furfaro, Roberto (15 June 2022). "Time-energy optimal landing on planetary bodies via theory of functional connections". Advances in Space Research. 69 (12): 4198–4220. Bibcode:2022AdSpR..69.4198D. doi:10.1016/j.asr.2022.04.009. ISSN 0273-1177.
  9. ^ Schiassi, Enrico; D’Ambrosio, Andrea; Furfaro, Roberto (2023). "An Overview of X-TFC Applications for Aerospace Optimal Control Problems". The Use of Artificial Intelligence for Space Applications. Studies in Computational Intelligence. Vol. 1088. Springer Nature Switzerland. pp. 199–212. doi:10.1007/978-3-031-25755-1_13. ISBN 978-3-031-25754-4.
  10. ^ De Florio, Mario; Schiassi, Enrico; Furfaro, Roberto (1 June 2022). "Physics-informed neural networks and functional interpolation for stiff chemical kinetics". Chaos: An Interdisciplinary Journal of Nonlinear Science. 32 (6). Bibcode:2022Chaos..32f3107D. doi:10.1063/5.0086649. ISSN 1054-1500. PMID 35778155.
  11. ^ Schiassi, Enrico; De Florio, Mario; D’Ambrosio, Andrea; Mortari, Daniele; Furfaro, Roberto (January 2021). "Physics-Informed Neural Networks and Functional Interpolation for Data-Driven Parameters Discovery of Epidemiological Compartmental Models". Mathematics. 9 (17): 2069. doi:10.3390/math9172069. hdl:11573/1566436. ISSN 2227-7390.
  12. ^ Criscola, Franco; Canales, David; Mortari, Daniele (September 2024). "Application of the Theory of Functional Connections to the Perturbed Lambert's Problem". Celestial Mechanics and Dynamical Astronomy. 136 (36). arXiv:2408.03381. Bibcode:2024CeMDA.136...36C. doi:10.1007/s10569-024-10212-4.
  13. ^ Mai, Tina; Mortari, Daniele (1 May 2022). "Theory of functional connections applied to quadratic and nonlinear programming under equality constraints". Journal of Computational and Applied Mathematics. 406: 113912. arXiv:1910.04917. doi:10.1016/j.cam.2021.113912. ISSN 0377-0427.
  14. ^ Yassopoulos, Christopher; Leake, Carl; Reddy, J. N.; Mortari, Daniele (1 November 2021). "Analysis of Timoshenko–Ehrenfest beam problems using the Theory of Functional Connections". Engineering Analysis with Boundary Elements. 132: 271–280. doi:10.1016/j.enganabound.2021.07.011. ISSN 0955-7997.
  15. ^ Yassopoulos, Christopher; Reddy, J. N.; Mortari, Daniele (1 March 2023). "Analysis of nonlinear Timoshenko–Ehrenfest beam problems with von Kármán nonlinearity using the Theory of Functional Connections". Mathematics and Computers in Simulation. 205: 709–744. doi:10.1016/j.matcom.2022.10.015. ISSN 0378-4754.
  16. ^ De Florio, Mario; Schiassi, Enrico; Furfaro, Roberto; Ganapol, Barry D.; Mostacci, Domiziano (1 January 2021). "Solutions of Chandrasekhar's basic problem in radiative transfer via theory of functional connections". Journal of Quantitative Spectroscopy and Radiative Transfer. 259: 107384. Bibcode:2021JQSRT.25907384D. doi:10.1016/j.jqsrt.2020.107384. hdl:11585/779571. ISSN 0022-4073.
  17. ^ Leake, Carl; Mortari, Daniele (March 2020). "Deep Theory of Functional Connections: A New Method for Estimating the Solutions of Partial Differential Equations". Machine Learning and Knowledge Extraction. 2 (1): 37–55. doi:10.3390/make2010004. ISSN 2504-4990. PMC 7259480. PMID 32478283.
  18. ^ Schiassi, Enrico; Furfaro, Roberto; Leake, Carl; De Florio, Mario; Johnston, Hunter; Mortari, Daniele (7 October 2021). "Extreme theory of functional connections: A fast physics-informed neural network method for solving ordinary and partial differential equations". Neurocomputing. 457: 334–356. doi:10.1016/j.neucom.2021.06.015. ISSN 0925-2312.
  19. ^ Raissi, M.; Perdikaris, P.; Karniadakis, G. E. (1 February 2019). "Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations". Journal of Computational Physics. 378: 686–707. Bibcode:2019JCoPh.378..686R. doi:10.1016/j.jcp.2018.10.045. ISSN 0021-9991. OSTI 1595805.
  20. ^ Cranmer, Miles (21 November 2024). "MilesCranmer/PySR". GitHub.
  21. ^ Daryakenari, Nazanin Ahmadi; Florio, Mario De; Shukla, Khemraj; Karniadakis, George Em (12 March 2024). "AI-Aristotle: A physics-informed framework for systems biology gray-box identification". PLOS Computational Biology. 20 (3): e1011916. arXiv:2310.01433. Bibcode:2024PLSCB..20E1916A. doi:10.1371/journal.pcbi.1011916. ISSN 1553-7358. PMC 10931529. PMID 38470870.
  22. ^ De Florio, Mario; Kevrekidis, Ioannis G.; Karniadakis, George Em (1 November 2024). "AI-Lorenz: A physics-data-driven framework for Black-Box and Gray-Box identification of chaotic systems with symbolic regression". Chaos, Solitons & Fractals. 188: 115538. arXiv:2312.14237. Bibcode:2024CSF...18815538D. doi:10.1016/j.chaos.2024.115538. ISSN 0960-0779.