Machine learning force field: Theory - Revision history

Liebetreu: Reverted edit by Liebetreu (talk) to last revision by Karsai

2026-04-15T12:37:32Z

Reverted edit by Liebetreu (talk) to last revision by Karsai

← Older revision		Revision as of 12:37, 15 April 2026
Line 422:		Line 422:

	<math>		<math>
	\mathbf{U}^{T}\mathbf{K} \mathbf{U}= \mathbf{L} = \~~mathrm~~{diag}(l_{1},l_{2},...,l_{N_{B}}),		\mathbf{U}^{T}\mathbf{K} \mathbf{U}= \mathbf{L} = \textrm{diag}(l_{1},l_{2},...,l_{N_{B}}),
	</math>		</math>

Liebetreu: /* Sparsification of local reference configurations */

2026-04-14T11:49:16Z

Sparsification of local reference configurations

← Older revision		Revision as of 11:49, 14 April 2026
Line 422:		Line 422:

	<math>		<math>
	\mathbf{U}^{T}\mathbf{K} \mathbf{U}= \mathbf{L} = \~~textrm~~{diag}(l_{1},l_{2},...,l_{N_{B}}),		\mathbf{U}^{T}\mathbf{K} \mathbf{U}= \mathbf{L} = \mathrm{diag}(l_{1},l_{2},...,l_{N_{B}}),
	</math>		</math>

Karsai: /* Bayesian linear regression */

2026-01-21T11:38:01Z

Bayesian linear regression

← Older revision		Revision as of 11:38, 21 January 2026
Line 229:		Line 229:
	== Bayesian linear regression ==		== Bayesian linear regression ==

	Ultimately for ~~error~~ prediction we want to get the maximized probability of observing a new structure <math>\mathbf{y}</math> on basis of the training set <math>\mathbf{Y}</math>, which is denoted as <math>p \left( \mathbf{y} \| \mathbf{Y} \right)</math>. For this we need to get from the error of the linear fitting coefficients <math>\mathbf{w}</math> in the reproduction of the training data <math>p\left( \mathbf{w} \| \mathbf{Y} \right)</math> to <math>p \left( \mathbf{y} \| \mathbf{Y} \right)</math> which is explained in the following.		Ultimately for uncertainty prediction we want to get the maximized probability of observing a new structure <math>\mathbf{y}</math> on basis of the training set <math>\mathbf{Y}</math>, which is denoted as <math>p \left( \mathbf{y} \| \mathbf{Y} \right)</math>. For this we need to get from the error of the linear fitting coefficients <math>\mathbf{w}</math> in the reproduction of the training data <math>p\left( \mathbf{w} \| \mathbf{Y} \right)</math> to <math>p \left( \mathbf{y} \| \mathbf{Y} \right)</math> which is explained in the following.

	First we obtain <math>p\left( \mathbf{w} \| \mathbf{Y} \right)</math> from the Bayesian theorem		First we obtain <math>p\left( \mathbf{w} \| \mathbf{Y} \right)</math> from the Bayesian theorem

Karsai: /* Threshold for uncertainty of forces */

2026-01-21T09:19:27Z

Threshold for uncertainty of forces

Karsai: /* Sampling of training data and local reference configurations */

2026-01-21T09:11:22Z

Sampling of training data and local reference configurations

← Older revision		Revision as of 09:11, 21 January 2026
Line 31:		Line 31:
	* If there is no force field present, all atoms of a structure are sampled as local reference configurations and a force field is constructed.		* If there is no force field present, all atoms of a structure are sampled as local reference configurations and a force field is constructed.
	* If the Bayesian uncertainty of the force for any atom is above the strict threshold set by {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the local reference configurations are sampled and a new force field is constructed.		* If the Bayesian uncertainty of the force for any atom is above the strict threshold set by {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the local reference configurations are sampled and a new force field is constructed.
	* If the Bayesian ~~error~~ of the force for any atom is above the threshold {{TAG\|ML_CTIFOR}} but below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the structure is added to the list of new training structure candidates. Whenever the number of candidates is equal to {{TAG\|ML_MCONF_NEW}} they are added to the entire set of training structures and the force field is updated. To avoid sampling too similar structures, the next step, from which training structures are allowed to be taken as candidates, is set by {{TAG\|ML_NMDINT}}. All ab initio calculations within this distance are skipped if the Bayesian ~~error~~ for the force on all atoms is below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}.		* If the Bayesian uncertainty of the force for any atom is above the threshold {{TAG\|ML_CTIFOR}} but below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the structure is added to the list of new training structure candidates. Whenever the number of candidates is equal to {{TAG\|ML_MCONF_NEW}} they are added to the entire set of training structures and the force field is updated. To avoid sampling too similar structures, the next step, from which training structures are allowed to be taken as candidates, is set by {{TAG\|ML_NMDINT}}. All ab initio calculations within this distance are skipped if the Bayesian uncertainty for the force on all atoms is below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}.

	=== Threshold for uncertainty of forces ===		=== Threshold for uncertainty of forces ===

Karsai: /* Threshold for error of forces */

2026-01-21T09:10:42Z

Threshold for error of forces

← Older revision		Revision as of 09:10, 21 January 2026
Line 33:		Line 33:
	* If the Bayesian error of the force for any atom is above the threshold {{TAG\|ML_CTIFOR}} but below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the structure is added to the list of new training structure candidates. Whenever the number of candidates is equal to {{TAG\|ML_MCONF_NEW}} they are added to the entire set of training structures and the force field is updated. To avoid sampling too similar structures, the next step, from which training structures are allowed to be taken as candidates, is set by {{TAG\|ML_NMDINT}}. All ab initio calculations within this distance are skipped if the Bayesian error for the force on all atoms is below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}.		* If the Bayesian error of the force for any atom is above the threshold {{TAG\|ML_CTIFOR}} but below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the structure is added to the list of new training structure candidates. Whenever the number of candidates is equal to {{TAG\|ML_MCONF_NEW}} they are added to the entire set of training structures and the force field is updated. To avoid sampling too similar structures, the next step, from which training structures are allowed to be taken as candidates, is set by {{TAG\|ML_NMDINT}}. All ab initio calculations within this distance are skipped if the Bayesian error for the force on all atoms is below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}.

	=== Threshold for ~~error~~ of forces ===		=== Threshold for uncertainty of forces ===
	Training structures and their corresponding local configurations are only chosen if the error in the forces of any atom exceeds a chosen threshold. The initial threshold is set to the value provided by {{TAG\|ML_CTIFOR}} (the unit is eV/Angstrom).		Training structures and their corresponding local configurations are only chosen if the error in the forces of any atom exceeds a chosen threshold. The initial threshold is set to the value provided by {{TAG\|ML_CTIFOR}} (the unit is eV/Angstrom).
	The behavior of how the threshold is further controlled is given by {{TAG\|ML_ICRITERIA}}. The following options are available:		The behavior of how the threshold is further controlled is given by {{TAG\|ML_ICRITERIA}}. The following options are available:

Karsai: /* Sampling of training data and local reference configurations */

2026-01-21T09:10:18Z

Sampling of training data and local reference configurations

← Older revision		Revision as of 09:10, 21 January 2026
Line 24:		Line 24:
	===Sampling of training data and local reference configurations===		===Sampling of training data and local reference configurations===

	We employ a learning scheme where structures are only added to the list of training structures when local reference configurations are picked for atoms that have an ~~error~~ in the force higher than a given threshold. So in the following, it is implied that whenever a new training structure is obtained, also local reference configurations from this structure are obtained.		We employ a learning scheme where structures are only added to the list of training structures when local reference configurations are picked for atoms that have an uncertainty in the force higher than a given threshold. So in the following, it is implied that whenever a new training structure is obtained, also local reference configurations from this structure are obtained.

	Usually one can employ that the force field doesn't necessarily need to be retrained immediately at every step when a training structure with corresponding local configurations is added. Instead, one can also collect candidates and do the learning in a later step for all structures simultaneously (blocking). This way saving significant computational costs. Of course learning after every new configuration or after every block can have different results, but with not too large block sizes the difference should be small. The tag {{TAG\|ML_MCONF_NEW}} sets the block size for learning.		Usually one can employ that the force field doesn't necessarily need to be retrained immediately at every step when a training structure with corresponding local configurations is added. Instead, one can also collect candidates and do the learning in a later step for all structures simultaneously (blocking). This way saving significant computational costs. Of course learning after every new configuration or after every block can have different results, but with not too large block sizes the difference should be small. The tag {{TAG\|ML_MCONF_NEW}} sets the block size for learning.
Line 30:		Line 30:

	* If there is no force field present, all atoms of a structure are sampled as local reference configurations and a force field is constructed.		* If there is no force field present, all atoms of a structure are sampled as local reference configurations and a force field is constructed.
	* If the Bayesian ~~error~~ of the force for any atom is above the strict threshold set by {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the local reference configurations are sampled and a new force field is constructed.		* If the Bayesian uncertainty of the force for any atom is above the strict threshold set by {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the local reference configurations are sampled and a new force field is constructed.
	* If the Bayesian error of the force for any atom is above the threshold {{TAG\|ML_CTIFOR}} but below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the structure is added to the list of new training structure candidates. Whenever the number of candidates is equal to {{TAG\|ML_MCONF_NEW}} they are added to the entire set of training structures and the force field is updated. To avoid sampling too similar structures, the next step, from which training structures are allowed to be taken as candidates, is set by {{TAG\|ML_NMDINT}}. All ab initio calculations within this distance are skipped if the Bayesian error for the force on all atoms is below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}.		* If the Bayesian error of the force for any atom is above the threshold {{TAG\|ML_CTIFOR}} but below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}, the structure is added to the list of new training structure candidates. Whenever the number of candidates is equal to {{TAG\|ML_MCONF_NEW}} they are added to the entire set of training structures and the force field is updated. To avoid sampling too similar structures, the next step, from which training structures are allowed to be taken as candidates, is set by {{TAG\|ML_NMDINT}}. All ab initio calculations within this distance are skipped if the Bayesian error for the force on all atoms is below {{TAG\|ML_CDOUB}}<math>\times</math>{{TAG\|ML_CTIFOR}}.

Karsai: /* Introduction */

2026-01-21T09:09:01Z

Introduction

← Older revision		Revision as of 09:09, 21 January 2026
Line 4:		Line 4:
	Molecular dynamics is one of the most important methods for the determination of dynamic properties. The quality of the molecular dynamics simulation depends very strongly on the accuracy of the calculational method, but higher accuracy usually comes at the cost of increased computational demand. A very accurate method is ab initio molecular dynamics, where the interactions of atoms and electrons are calculated fully quantum mechanically from ab initio calculations (such as e.g. DFT). Unfortunately, this method is very often limited to small simulation cells and simulation times. One way to hugely speed up these calculations is by using force fields, which are parametrizations of the potential energy. These parametrizations can range from very simple functions with only empirical parameters, to very complex functions parametrized using thousands of ab initio calculations. Usually making accurate force fields manually is a very time-consuming task that needs tremendous expertise and know-how.		Molecular dynamics is one of the most important methods for the determination of dynamic properties. The quality of the molecular dynamics simulation depends very strongly on the accuracy of the calculational method, but higher accuracy usually comes at the cost of increased computational demand. A very accurate method is ab initio molecular dynamics, where the interactions of atoms and electrons are calculated fully quantum mechanically from ab initio calculations (such as e.g. DFT). Unfortunately, this method is very often limited to small simulation cells and simulation times. One way to hugely speed up these calculations is by using force fields, which are parametrizations of the potential energy. These parametrizations can range from very simple functions with only empirical parameters, to very complex functions parametrized using thousands of ab initio calculations. Usually making accurate force fields manually is a very time-consuming task that needs tremendous expertise and know-how.

	Another way to greatly reduce computational cost and required human intervention is by machine learning. Here, in the prediction of the target property, the method automatically interpolates between known training systems that were previously calculated ab initio. This way the generation of force fields is already significantly simplified compared to a classical force field which needs manual (or even empirical) adjustment of the parameters. Nevertheless, there is still the problem of how to choose the proper (minimal) training data. One very efficient and automatic way to solve that is to adapt on-the-fly learning. Here an MD calculation is used for the learning. During the run of the MD calculation, ab initio data is picked out and added to the training data. From the existing data, a force field is continuously built up. At each step, it is judged whether to make an ab initio calculation and possibly add the data to the force field or to use the force field for that step and actually skip learning for that step. Hence the name "on the fly" learning. The crucial point here is the probability model for the estimation of ~~errors~~. This model is built up from newly sampled data. Hence, the more accurate the force field gets the less sampling is needed and the more expensive ab initio steps are skipped. This way not only the convergence of the force field can be controlled but also a very widespread scan through phase space for the training structures can be performed. The crucial point for on-the-fly machine learning which will be explained with the rest of the methodology in the following subsections is to be able to predict ~~errors~~ of the force field on a newly sampled structure without the necessity to perform an ab initio calculation on that structure.		Another way to greatly reduce computational cost and required human intervention is by machine learning. Here, in the prediction of the target property, the method automatically interpolates between known training systems that were previously calculated ab initio. This way the generation of force fields is already significantly simplified compared to a classical force field which needs manual (or even empirical) adjustment of the parameters. Nevertheless, there is still the problem of how to choose the proper (minimal) training data. One very efficient and automatic way to solve that is to adapt on-the-fly learning. Here an MD calculation is used for the learning. During the run of the MD calculation, ab initio data is picked out and added to the training data. From the existing data, a force field is continuously built up. At each step, it is judged whether to make an ab initio calculation and possibly add the data to the force field or to use the force field for that step and actually skip learning for that step. Hence the name "on the fly" learning. The crucial point here is the probability model for the estimation of uncertainties. This model is built up from newly sampled data. Hence, the more accurate the force field gets the less sampling is needed and the more expensive ab initio steps are skipped. This way not only the convergence of the force field can be controlled but also a very widespread scan through phase space for the training structures can be performed. The crucial point for on-the-fly machine learning which will be explained with the rest of the methodology in the following subsections is to be able to predict uncertainties of the force field on a newly sampled structure without the necessity to perform an ab initio calculation on that structure.

	== Algorithms ==		== Algorithms ==

Karsai: /* Error estimation */

2026-01-21T09:08:01Z

Error estimation

← Older revision		Revision as of 09:08, 21 January 2026
Line 371:		Line 371:
	The evidence approximation can be done for any of the above-described regression methods, but we combine it only with the solutions from LU factorization since solutions via SVD are calculationally too expensive to be carried out multiple times.		The evidence approximation can be done for any of the above-described regression methods, but we combine it only with the solutions from LU factorization since solutions via SVD are calculationally too expensive to be carried out multiple times.

	== ~~Error~~ estimation ==		== Uncertainty estimation ==

	=== ~~Error~~ estimates from Bayesian linear regression ===		=== Uncertainty estimates from Bayesian linear regression ===

	By using the relation		By using the relation

Kresse: /* Error estimates from Bayesian linear regression */

2026-01-14T14:14:04Z

Error estimates from Bayesian linear regression

← Older revision		Revision as of 14:14, 14 January 2026
Line 396:		Line 396:
	The diagonal elements of <math>\mathbf{\sigma}</math> correspond to the variances in the predicted results. These variances are a measure of the uncertainty of the predicted (dimensionless) total energies, forces, and stress tensors. VASP outputs the square root of these variances, i.e. the standard deviations of the predicted means, to the [[ML_LOGFILE]] ("<code>grep BEEF ML_LOGFILE</code>"). The output is scaled to the appropriate units (eV/atom and eV/A). During on-the-fly training, a first-principles calculation is performed if the standard deviations are above a certain threshold.		The diagonal elements of <math>\mathbf{\sigma}</math> correspond to the variances in the predicted results. These variances are a measure of the uncertainty of the predicted (dimensionless) total energies, forces, and stress tensors. VASP outputs the square root of these variances, i.e. the standard deviations of the predicted means, to the [[ML_LOGFILE]] ("<code>grep BEEF ML_LOGFILE</code>"). The output is scaled to the appropriate units (eV/atom and eV/A). During on-the-fly training, a first-principles calculation is performed if the standard deviations are above a certain threshold.

	Please note that the uncertainty estimate can only account for epistemic uncertainty, which arises from a lack of knowledge or data. Systematic errors caused by the model's approximation can not be measured or determined from the Bayesian uncertainty. In kernel-based methods, model errors usually dominate the total error after few hundred training steps; thus the predicted uncertainty is usually significantly smaller than the true error. This does not ~~point~~ indicate any deficiency in the Bayesian regression!		Please note that the uncertainty estimate can only account for epistemic uncertainty, which arises from a lack of knowledge or data. Systematic errors caused by the model's approximation can not be measured or determined from the Bayesian uncertainty. In kernel-based methods, model errors usually dominate the total error after few hundred training steps; thus the predicted uncertainty is usually significantly smaller than the true error. This does not indicate any deficiency in the Bayesian regression!

	=== Spilling factor ===		=== Spilling factor ===

← Older revision		Revision as of 09:19, 21 January 2026
Line 34:		Line 34:

	=== Threshold for uncertainty of forces ===		=== Threshold for uncertainty of forces ===
	Training structures and their corresponding local configurations are only chosen if the ~~error~~ in the forces of any atom exceeds a chosen threshold. The initial threshold is set to the value provided by {{TAG\|ML_CTIFOR}} (the unit is eV/Angstrom).		Training structures and their corresponding local configurations are only chosen if the uncertainty in the forces of any atom exceeds a chosen threshold. The initial threshold is set to the value provided by {{TAG\|ML_CTIFOR}} (the unit is eV/Angstrom).
	The behavior of how the threshold is further controlled is given by {{TAG\|ML_ICRITERIA}}. The following options are available:		The behavior of how the threshold is further controlled is given by {{TAG\|ML_ICRITERIA}}. The following options are available:
	*{{TAG\|ML_ICRITERIA}} = 0: No update of initial value of {{TAG\|ML_CTIFOR}} is done.		*{{TAG\|ML_ICRITERIA}} = 0: No update of initial value of {{TAG\|ML_CTIFOR}} is done.
	*{{TAG\|ML_ICRITERIA}} = 1: Update of criteria using an average of the Bayesian ~~errors~~ of the forces from history (see description of the method below).		*{{TAG\|ML_ICRITERIA}} = 1: Update of criteria using an average of the Bayesian uncertainties of the forces from history (see description of the method below).
	*{{TAG\|ML_ICRITERIA}} = 2: Update of criteria using gliding average of Bayesian ~~errors~~ (probably more robust but '''not well tested''').		*{{TAG\|ML_ICRITERIA}} = 2: Update of criteria using gliding average of Bayesian uncertainties (probably more robust but '''not well tested''').

	Generally, it is recommended to automatically update the threshold {{TAG\|ML_CTIFOR}} during machine learning. Details on how and when the update is performed are controlled by {{TAG\|ML_CSLOPE}}, {{TAG\|ML_CSIG}} and {{TAG\|ML_MHIS}}.		Generally, it is recommended to automatically update the threshold {{TAG\|ML_CTIFOR}} during machine learning. Details on how and when the update is performed are controlled by {{TAG\|ML_CSLOPE}}, {{TAG\|ML_CSIG}} and {{TAG\|ML_MHIS}}.
Line 44:		Line 44:
	Description of {{TAG\|ML_ICRITERIA}}=1:		Description of {{TAG\|ML_ICRITERIA}}=1:

	{{TAG\|ML_CTIFOR}} is updated using the average Bayesian ~~error~~ in the previous steps. Specifically, it is set to		{{TAG\|ML_CTIFOR}} is updated using the average Bayesian uncertainty in the previous steps. Specifically, it is set to

	{{TAG\|ML_CTIFOR}} = (average of the stored Bayesian ~~errors~~) *(1.0 + {{TAG\|ML_CX}}).		{{TAG\|ML_CTIFOR}} = (average of the stored Bayesian uncertainties) *(1.0 + {{TAG\|ML_CX}}).

	The number of entries in the history of the Bayesian ~~errors~~ is controlled by {{TAG\|ML_MHIS}}. To avoid noisy data or an abrupt jump of the Bayesian ~~error~~ causing issues, the standard error of the history must be below the threshold {{TAG\|ML_CSIG}}, for the update to take place. Furthermore, the slope of the stored data must be below the threshold {{TAG\|ML_CSLOPE}}. In practice, the slope and the standard errors are at least to some extent correlated: often the standard error is proportional to {{TAG\|ML_MHIS}}/3 times the slope or somewhat larger. We recommend to vary only {{TAG\|ML_CSIG}} and keep {{TAG\|ML_CSLOPE}} fixed to its default value.		The number of entries in the history of the Bayesian uncertainties is controlled by {{TAG\|ML_MHIS}}. To avoid noisy data or an abrupt jump of the Bayesian uncertainty causing issues, the standard error of the history must be below the threshold {{TAG\|ML_CSIG}}, for the update to take place. Furthermore, the slope of the stored data must be below the threshold {{TAG\|ML_CSLOPE}}. In practice, the slope and the standard errors are at least to some extent correlated: often the standard error is proportional to {{TAG\|ML_MHIS}}/3 times the slope or somewhat larger. We recommend to vary only {{TAG\|ML_CSIG}} and keep {{TAG\|ML_CSLOPE}} fixed to its default value.

	== Local energies ==		== Local energies ==